-
Stefy Lanza (nextime / spora ) authored
A pod knew only what it was seeded with at boot, so any request naming another model got "not available" for the rest of that pod's life. That makes a pod a fixed catalogue rather than an extension of the local system, which is the whole point of it. When a remote refuses a model it does not know, the gateway now teaches it and retries once: resolve the model to something the remote can fetch (HuggingFace id, URL, or an upload when it exists nowhere else), POST it to the new /v1/models/register, and replay the request. Registration is in memory — a pod is disposable — and idempotent, so a model already loaded is never disturbed. Taught once per pod, tracked by (base, model). The retry is bounded to one attempt and only fires on a 400/404 whose body actually says the model is unknown: a 500 or an out-of-memory must never turn into a retry loop against a rented GPU. A model that resolves to no fetchable source is refused with that reason rather than registered as something the pod could never load. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
92c264f2