Skip to content
Commit 92c264f2 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

runpod: let a pod learn a model at runtime



A pod knew only what it was seeded with at boot, so any request naming
another model got "not available" for the rest of that pod's life. That
makes a pod a fixed catalogue rather than an extension of the local
system, which is the whole point of it.

When a remote refuses a model it does not know, the gateway now teaches
it and retries once: resolve the model to something the remote can fetch
(HuggingFace id, URL, or an upload when it exists nowhere else), POST it
to the new /v1/models/register, and replay the request. Registration is
in memory — a pod is disposable — and idempotent, so a model already
loaded is never disturbed. Taught once per pod, tracked by (base, model).

The retry is bounded to one attempt and only fires on a 400/404 whose
body actually says the model is unknown: a 500 or an out-of-memory must
never turn into a retry loop against a rented GPU.

A model that resolves to no fetchable source is refused with that reason
rather than registered as something the pod could never load.

Co-Authored-By: default avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
parent ee879bef
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment