-
Stefy Lanza (nextime / spora ) authored
A pod cannot see this machine's disk, so "use my local models on RunPod" needed a way for the pod to GET them. Three, now, picked by `source`: auto — an explicit hf_repo, a path that already is a repo id, or a repo id recovered from the HuggingFace cache path (models--Owner--Repo). Most of a normal catalogue resolves with nothing typed: weights downloaded through coderai live in that cache, and the directory name still says where from. hf — download hf_repo from HuggingFace. url — download model_url directly, which is how most one-off GGUFs are had. upload — send the weights from here. Each server gets only what it can use, refused at configuration time rather than after five minutes of pod boot: vLLM is launched `--model` and takes a repo id only; llama.cpp takes `-hf` or downloads a URL with `-mu`; upload works only on a coderai pod, because vLLM and llama.cpp need the file before their container starts while a coderai pod loads models on demand. Upload is a real option, not a token one — on symmetric gigabit a 30 GB model is a few minutes — but it is the cold cost of every NEW pod, so it is never chosen automatically. /v1/models/upload streams to disk in chunks (never into memory), extracts a tar for multi-file models with every member checked against the destination, and registers the entry in memory only. An existence check by name and size makes a repeat free and refuses a half-copy. keep_warm keeps one pod running so nothing waits for a cold boot. OFF by default and deliberately: a warm pod bills every hour of every day, including the ones nobody uses it. Warm pods start at startup, since a pool created on first use would still pay the boot it exists to avoid. All of it is in the UI: weights source with the right field shown per choice, keep-warm, and an "always run this model locally" pin. Co-Authored-By:Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
249da5af
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| static | ||
| templates | ||
| __init__.py | ||
| auth.py | ||
| download_worker.py | ||
| routes.py |