• Stefy Lanza (nextime / spora )'s avatar
    runpod: resolve, download or upload a model's weights; optional warm pods · 249da5af
    Stefy Lanza (nextime / spora ) authored
    A pod cannot see this machine's disk, so "use my local models on RunPod"
    needed a way for the pod to GET them. Three, now, picked by `source`:
    
      auto   — an explicit hf_repo, a path that already is a repo id, or a
               repo id recovered from the HuggingFace cache path
               (models--Owner--Repo). Most of a normal catalogue resolves
               with nothing typed: weights downloaded through coderai live
               in that cache, and the directory name still says where from.
      hf     — download hf_repo from HuggingFace.
      url    — download model_url directly, which is how most one-off GGUFs
               are had.
      upload — send the weights from here.
    
    Each server gets only what it can use, refused at configuration time
    rather than after five minutes of pod boot: vLLM is launched `--model`
    and takes a repo id only; llama.cpp takes `-hf` or downloads a URL with
    `-mu`; upload works only on a coderai pod, because vLLM and llama.cpp
    need the file before their container starts while a coderai pod loads
    models on demand.
    
    Upload is a real option, not a token one — on symmetric gigabit a 30 GB
    model is a few minutes — but it is the cold cost of every NEW pod, so it
    is never chosen automatically. /v1/models/upload streams to disk in
    chunks (never into memory), extracts a tar for multi-file models with
    every member checked against the destination, and registers the entry in
    memory only. An existence check by name and size makes a repeat free and
    refuses a half-copy.
    
    keep_warm keeps one pod running so nothing waits for a cold boot. OFF by
    default and deliberately: a warm pod bills every hour of every day,
    including the ones nobody uses it. Warm pods start at startup, since a
    pool created on first use would still pay the boot it exists to avoid.
    
    All of it is in the UI: weights source with the right field shown per
    choice, keep-warm, and an "always run this model locally" pin.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    249da5af
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
colibri_worker.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lipsync.py Loading commit data...
log.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
models_transfer.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...