-
Stefy Lanza (nextime / spora ) authored
Two models that both offload rented a card each, even when neither came close to filling one. They can now name a shared `pool` — on the model's runpod block or on a burst target — and land on the same pod; each request carries its own model name, and the pool rents a second pod only once the first is carrying scale_up_inflight_per_pod requests. Sharing is refused, loudly, for pod servers pinned to one model at launch: a vLLM pod is `--model X` and a llama.cpp pod `-hf one.gguf`. Allowing it would send requests to a pod that answers with the wrong model's weights, or 404s — a clear error at configuration time beats debugging that. `engine: coderai` (a whole coderai, which picks the model per request) is what makes a shared pool possible. The first model to create a shared pool sets its shape; later joiners do not overwrite it, so whichever model happens to load last cannot silently redefine the budget everyone is sharing. Capability pools now use the same rule instead of their own ad-hoc guard. Settable in the UI on both the model's pod section and its burst target. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
69d944a9