runpod: resolve, download or upload a model's weights; optional warm pods

A pod cannot see this machine's disk, so "use my local models on RunPod"
needed a way for the pod to GET them. Three, now, picked by `source`:

  auto   — an explicit hf_repo, a path that already is a repo id, or a
           repo id recovered from the HuggingFace cache path
           (models--Owner--Repo). Most of a normal catalogue resolves
           with nothing typed: weights downloaded through coderai live
           in that cache, and the directory name still says where from.
  hf     — download hf_repo from HuggingFace.
  url    — download model_url directly, which is how most one-off GGUFs
           are had.
  upload — send the weights from here.

Each server gets only what it can use, refused at configuration time
rather than after five minutes of pod boot: vLLM is launched `--model`
and takes a repo id only; llama.cpp takes `-hf` or downloads a URL with
`-mu`; upload works only on a coderai pod, because vLLM and llama.cpp
need the file before their container starts while a coderai pod loads
models on demand.

Upload is a real option, not a token one — on symmetric gigabit a 30 GB
model is a few minutes — but it is the cold cost of every NEW pod, so it
is never chosen automatically. /v1/models/upload streams to disk in
chunks (never into memory), extracts a tar for multi-file models with
every member checked against the destination, and registers the entry in
memory only. An existence check by name and size makes a repeat free and
refuses a half-copy.

keep_warm keeps one pod running so nothing waits for a cold boot. OFF by
default and deliberately: a warm pod bills every hour of every day,
including the ones nobody uses it. Warm pods start at startup, since a
pool created on first use would still pay the boot it exists to avoid.

All of it is in the UI: weights source with the right field shown per
choice, keep-warm, and an "always run this model locally" pin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY

codai/admin/routes.py

@@ -2710,6 +2710,10 @@ async def api_model_configure(request: Request, username: str = Depends(require_
                 "auto_compact_model", "suppress_reasoning",
                 # Speech-to-text (wav2vec2 / vosk / NeMo-Canary) options.
                 "languages", "supports_translation", "keep_resident",
+                # "local" pins the model here even when its whole capability is
+                # configured remote — the opt-out that lets two models of the
+                # same kind be placed differently.
+                "placement",
                 ):
         if key in data:
             entry[key] = data[key]
@@ -2787,7 +2791,9 @@ async def api_model_configure(request: Request, username: str = Depends(require_
                   "endpoint_id", "cost_period",
                   # engine selection, private registries, and the pod's own token
                   "engine", "hf_gguf", "health_path", "docker_args",
-                  "registry_auth_id", "api_key", "pool"):
+                  "registry_auth_id", "api_key", "pool",
+                  # where the POD gets the weights
+                  "source", "hf_repo", "model_url", "quantization"):
             v = src.get(k)
             if isinstance(v, str) and v.strip():
                 rpo[k] = v.strip()
@@ -2802,7 +2808,7 @@ async def api_model_configure(request: Request, username: str = Depends(require_
         if ct:
             rpo["cloud_types"] = ct
         # bools
-        for k in ("allow_spot", "allow_open_pod", "sticky_sessions"):
+        for k in ("allow_spot", "allow_open_pod", "sticky_sessions", "keep_warm"):
             if k in src and src.get(k) is not None and src.get(k) != "":
                 sv = src.get(k)
                 rpo[k] = (sv.lower() in ("1", "true", "on", "yes")

codai/admin/templates/models.html

@@ -743,6 +743,13 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
             <div class="form-row" style="margin:0"><label class="form-label">Max pods</label><input type="number" id="cfg-rp-max-pods" class="form-input" min="1" placeholder="1"></div>
             <div class="form-row" style="margin:0"><label class="form-label">Idle timeout (s)</label><input type="number" id="cfg-rp-idle" class="form-input" min="0" placeholder="300"></div>
           </div>
+          <div class="form-row" style="margin:.5rem 0 0">
+            <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer">
+              <input type="checkbox" id="cfg-rp-keep-warm" style="width:auto">
+              <span class="form-label" style="margin:0">Always keep a pod running for this model</span>
+            </label>
+            <span class="form-hint" style="font-size:11px">No cold-boot wait (image pull + weight download, minutes) — but it bills every hour of every day, including the ones nobody uses it. Off by default.</span>
+          </div>
           <div style="display:grid;grid-template-columns:1fr 1fr;gap:.5rem;margin-top:.5rem">
             <div class="form-row" style="margin:0">
               <label class="form-label">Boot timeout (s)</label>
@@ -773,6 +780,23 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
               <span class="form-hint" style="font-size:11px">A local .gguf path means nothing on a rented machine — name the HF repo the pod should download.</span>
             </div>
           </div>
+          <div style="display:grid;grid-template-columns:1fr 2fr;gap:.5rem;margin-top:.5rem">
+            <div class="form-row" style="margin:0">
+              <label class="form-label">Weights come from</label>
+              <select id="cfg-rp-source" class="form-input" onchange="_updateRunpodSource()">
+                <option value="auto">Auto — work it out</option>
+                <option value="hf">HuggingFace repo</option>
+                <option value="url">Direct URL</option>
+                <option value="upload">Upload from here</option>
+              </select>
+              <span class="form-hint" id="cfg-rp-source-hint" style="font-size:11px"></span>
+            </div>
+            <div class="form-row" style="margin:0">
+              <label class="form-label" id="cfg-rp-source-label">Repo / URL</label>
+              <input type="text" id="cfg-rp-hf-repo" class="form-input" placeholder="Qwen/Qwen3.5-9B">
+              <input type="text" id="cfg-rp-model-url" class="form-input" style="display:none" placeholder="https://host/model-Q4_K_M.gguf">
+            </div>
+          </div>
           <div class="form-row" style="margin:.5rem 0 0">
             <label class="form-label">Share pod pool <span class="muted">(optional name)</span></label>
             <input type="text" id="cfg-rp-pool" class="form-input" placeholder="blank = this model rents its own pods">
@@ -855,6 +879,14 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
       <div class="form-section" style="margin-top:.75rem">
         <div class="form-section-title">Placement — where this model runs</div>
 
+        <div class="form-row" style="margin:0 0 .6rem">
+          <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer">
+            <input type="checkbox" id="cfg-placement-local" style="width:auto">
+            <span class="form-label" style="margin:0">Always run this model locally</span>
+          </label>
+          <span class="form-hint" style="font-size:11px">Pins it here even when its whole capability is set remote — so one video model can stay local while another runs on a pod.</span>
+        </div>
+
         <!-- (1) Served from another machine entirely. -->
         <div class="form-row" style="margin:0">
           <label class="form-label">Serve from a remote URL <span class="muted">(any OpenAI-compatible endpoint)</span></label>
@@ -4157,6 +4189,11 @@ function _populateRunpod(s){
   set('cfg-rp-engine', rp.engine || 'auto');
   set('cfg-rp-hf-gguf', rp.hf_gguf || '');
   set('cfg-rp-pool', rp.pool || '');
+  set('cfg-rp-source', rp.source || 'auto');
+  set('cfg-rp-hf-repo', rp.hf_repo || '');
+  set('cfg-rp-model-url', rp.model_url || '');
+  chk('cfg-rp-keep-warm', rp.keep_warm);
+  _updateRunpodSource();
   set('cfg-rp-api-key', rp.api_key || '');
   chk('cfg-rp-open', rp.allow_open_pod);
   set('cfg-rp-registry-auth', rp.registry_auth_id || '');
@@ -4174,6 +4211,7 @@ function _populateRunpod(s){
 function _populatePlacement(s){
   const set = (id,v)=>{ const el=document.getElementById(id); if(el!=null) el.value=(v==null?'':v); };
   const chk = (id,v)=>{ const el=document.getElementById(id); if(el) el.checked=!!v; };
+  chk('cfg-placement-local', String((s && s.placement) || '').toLowerCase() === 'local');
   set('cfg-remote-url', (s && s.service_url) || '');
   set('cfg-remote-served', (s && s.served_model) || '');
   set('cfg-remote-key', (s && s.api_key) || '');
@@ -4208,6 +4246,29 @@ function _updateSpillVisibility(){
   if (pd) pd.style.display = (mode === 'serverless') ? 'none' : '';
 }
 
+// A pod cannot see this machine's disk: it downloads the weights itself, or is
+// sent them. This picks which, and shows the field that matters.
+function _updateRunpodSource(){
+  const el = document.getElementById('cfg-rp-source');
+  if (!el) return;
+  const v = el.value || 'auto';
+  const repo = document.getElementById('cfg-rp-hf-repo');
+  const url = document.getElementById('cfg-rp-model-url');
+  const label = document.getElementById('cfg-rp-source-label');
+  const hint = document.getElementById('cfg-rp-source-hint');
+  if (repo) repo.style.display = (v === 'url' || v === 'upload') ? 'none' : '';
+  if (url) url.style.display = (v === 'url') ? '' : 'none';
+  if (label) label.textContent = (v === 'url') ? 'Download URL'
+    : (v === 'upload') ? 'Uploaded from this machine' : 'HuggingFace repo id';
+  const HINTS = {
+    auto: 'Uses the repo id if the model has one — including one recovered from the HuggingFace cache path.',
+    hf: 'The pod downloads it from HuggingFace at datacenter speed. Costs nothing here.',
+    url: 'The pod downloads it directly — how most one-off GGUFs are had.',
+    upload: 'Sent from this machine. Needed only when the weights exist nowhere else; every NEW pod pays the upload again.',
+  };
+  if (hint) hint.textContent = HINTS[v] || '';
+}
+
 function _updateRunpodEngineHint(){
   const el = document.getElementById('cfg-rp-engine');
   const hint = document.getElementById('cfg-rp-engine-hint');
@@ -4293,6 +4354,10 @@ function _collectRunpod(){
     engine:             val('cfg-rp-engine') || 'auto',
     hf_gguf:            (val('cfg-rp-hf-gguf') || '').trim(),
     pool:               (val('cfg-rp-pool') || '').trim(),
+    source:             val('cfg-rp-source') || 'auto',
+    hf_repo:            (val('cfg-rp-hf-repo') || '').trim(),
+    model_url:          (val('cfg-rp-model-url') || '').trim(),
+    keep_warm:          document.getElementById('cfg-rp-keep-warm').checked,
     api_key:            (val('cfg-rp-api-key') || '').trim(),
     allow_open_pod:     document.getElementById('cfg-rp-open').checked,
     registry_auth_id:   (val('cfg-rp-registry-auth') || '').trim(),
@@ -4327,6 +4392,7 @@ async function saveModelConfig(){
     // Placement: serve this model from elsewhere, and/or burst the overflow.
     // Always sent, even when blank: an empty value is how you REMOVE a remote,
     // and a key that is simply absent leaves the old one in place.
+    placement:         document.getElementById('cfg-placement-local').checked ? 'local' : '',
     service_url:       (document.getElementById('cfg-remote-url').value || '').trim(),
     served_model:      (document.getElementById('cfg-remote-served').value || '').trim(),
     api_key:           (document.getElementById('cfg-remote-key').value || '').trim(),

codai/api/app.py

@@ -95,6 +95,18 @@ async def lifespan(app: FastAPI):
     except Exception:
         pass
 
+    # Start any RunPod pod configured to stay warm. Pools are otherwise created
+    # on first use, so the first request after a restart would still pay the cold
+    # boot that `keep_warm` exists to avoid. In a thread: provisioning blocks for
+    # minutes and must not hold up startup.
+    try:
+        import threading as _th
+        from codai.api.runpod_worker import warm_configured_pools
+        _th.Thread(target=warm_configured_pools, daemon=True,
+                   name="runpod-warm").start()
+    except Exception:
+        pass
+
     yield
 
     if broker_service is not None:
@@ -592,6 +604,8 @@ app.include_router(environments_router, tags=["Environments"])
 app.include_router(spatial_router, tags=["Spatial / 3D"])
 app.include_router(ocr_router, tags=["OCR"])
 app.include_router(rerank_router, tags=["Rerank"])
+from codai.api.models_transfer import router as models_transfer_router
+app.include_router(models_transfer_router, tags=["Models"])
 app.include_router(admin_router, tags=["Admin"])
 
 

codai/api/models_transfer.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""Push a model's weights to a coderai that cannot fetch them itself.
+
+A rented pod normally downloads its own weights — from HuggingFace, or from a
+URL — which costs nothing here and runs at datacenter speed. But some models
+have neither: a merge you made locally, a file from a host that is gone, weights
+you are not going to publish. For those, the only way to get them onto a pod is
+to send them.
+
+That is what this is. It is deliberately a plain streaming upload rather than
+anything clever: the receiving side writes the bytes to its model directory and
+registers the entry, so the model becomes loadable exactly as if it had been
+downloaded there.
+
+  POST /v1/models/upload?name=<file>&model_type=<section>[&alias=][&tar=1]
+       body: the raw file (or a tar stream when tar=1), streamed to disk
+  GET  /v1/models/uploaded?name=<file>[&bytes=<n>]
+       200 when the file is already there (and the right size), 404 otherwise
+
+Sending 30 GB is a real decision, not a default: coderai resolves a HuggingFace
+repo id or a URL first and only uploads when told to (`source: "upload"`). The
+existence check makes the second pod cheap — but a pod is disposable storage, so
+every NEW pod pays the upload again. A RunPod network volume is the better answer
+for weights you will use repeatedly.
+"""
+
+import os
+import shutil
+import tempfile
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Request
+
+from codai.api.loras import _require_api_auth
+
+router = APIRouter()
+
+#: models.json sections an upload may target.
+_SECTIONS = {
+    "text_models", "image_models", "audio_models", "gguf_models", "tts_models",
+    "vision_models", "video_models", "audio_gen_models", "embedding_models",
+    "spatial_models",
+}
+
+
+def _models_dir() -> str:
+    """Where uploaded weights land — the same cache the downloader writes to."""
+    try:
+        from codai.models.cache import get_model_cache_dir
+        return get_model_cache_dir()
+    except Exception:
+        return os.path.expanduser(os.environ.get("CODERAI_MODELS_DIR", "~/.coderai/models"))
+
+
+def _safe_target(name: str) -> str:
+    """Resolve an upload name to a path inside the model directory.
+
+    The name comes from another machine, so it is reduced to a bare basename and
+    the result is re-checked against the directory: a crafted name must not be
+    able to write anywhere else.
+    """
+    base = os.path.basename((name or "").strip())
+    if not base or base in (".", "..") or "/" in base or "\\" in base:
+        raise HTTPException(status_code=400, detail="invalid model name")
+    root = os.path.realpath(_models_dir())
+    os.makedirs(root, exist_ok=True)
+    target = os.path.realpath(os.path.join(root, base))
+    if os.path.commonpath([target, root]) != root:
+        raise HTTPException(status_code=400, detail="invalid model name")
+    return target
+
+
+@router.get("/v1/models/uploaded", summary="Check an uploaded model is present")
+async def model_uploaded(name: str = Query(...), bytes: int = Query(0),
+                         _auth=Depends(_require_api_auth)):
+    """200 when the named model is already here, 404 otherwise.
+
+    Lets the sender skip re-uploading to a pod that already has it — the check
+    that makes a warm pod free and a cold one pay once.
+    """
+    target = _safe_target(name)
+    if not os.path.exists(target):
+        raise HTTPException(status_code=404, detail="not uploaded")
+    size = (os.path.getsize(target) if os.path.isfile(target)
+            else sum(os.path.getsize(os.path.join(r, f))
+                     for r, _, fs in os.walk(target) for f in fs))
+    if bytes and size != bytes:
+        # A partial or stale copy is worse than none: report it missing so the
+        # sender replaces it rather than the pod loading half a model.
+        raise HTTPException(status_code=404, detail="size mismatch")
+    return {"name": os.path.basename(target), "path": target, "bytes": size}
+
+
+@router.post("/v1/models/upload", summary="Upload model weights")
+async def upload_model(request: Request, name: str = Query(...),
+                       model_type: str = Query("text_models"),
+                       alias: str = Query(""), tar: int = Query(0),
+                       _auth=Depends(_require_api_auth)):
+    """Stream model weights to this instance and register them.
+
+    Streamed straight to disk in chunks — these are multi-GB files and must never
+    be held in memory. A tar stream (``tar=1``) is extracted, which is how a
+    multi-file model directory travels.
+    """
+    if model_type not in _SECTIONS:
+        raise HTTPException(status_code=400, detail=f"unknown model_type {model_type!r}")
+    target = _safe_target(name)
+
+    tmp = target + ".part"
+    total = 0
+    try:
+        with open(tmp, "wb") as fh:
+            async for chunk in request.stream():
+                if chunk:
+                    fh.write(chunk)
+                    total += len(chunk)
+        if not total:
+            raise HTTPException(status_code=400, detail="empty upload")
+        if tar:
+            extract_to = target + ".incoming"
+            shutil.rmtree(extract_to, ignore_errors=True)
+            os.makedirs(extract_to, exist_ok=True)
+            import tarfile
+            with tarfile.open(tmp, "r:*") as tf:
+                _safe_extract(tf, extract_to)
+            shutil.rmtree(target, ignore_errors=True)
+            os.replace(extract_to, target)
+            os.remove(tmp)
+        else:
+            os.replace(tmp, target)
+    except HTTPException:
+        _cleanup(tmp)
+        raise
+    except Exception as exc:
+        _cleanup(tmp)
+        raise HTTPException(status_code=500, detail=f"upload failed: {exc}")
+
+    _register(target, model_type, alias)
+    return {"path": target, "bytes": total, "model_type": model_type}
+
+
+def _cleanup(path: str) -> None:
+    try:
+        os.remove(path)
+    except OSError:
+        pass
+
+
+def _safe_extract(tf, dest: str) -> None:
+    """Extract a tar, refusing any member that would escape ``dest``.
+
+    The archive comes from another machine; a member named ../../etc/x must not
+    be able to write outside the destination.
+    """
+    root = os.path.realpath(dest)
+    for member in tf.getmembers():
+        out = os.path.realpath(os.path.join(dest, member.name))
+        if os.path.commonpath([out, root]) != root:
+            raise HTTPException(status_code=400,
+                                detail=f"tar member escapes the destination: {member.name}")
+        if member.issym() or member.islnk():
+            link = os.path.realpath(os.path.join(os.path.dirname(out), member.linkname))
+            if os.path.commonpath([link, root]) != root:
+                raise HTTPException(status_code=400,
+                                    detail=f"tar link escapes the destination: {member.name}")
+    tf.extractall(dest)
+
+
+def _register(path: str, model_type: str, alias: str = "") -> None:
+    """Add the uploaded model to the in-memory catalogue so it can be served.
+
+    In memory only, like CODERAI_SEED_MODELS: a pod is disposable, and writing
+    models.json here would edit a real installation's catalogue behind its back.
+    """
+    try:
+        from codai.admin.routes import config_manager
+        md = getattr(config_manager, "models_data", None)
+        if not isinstance(md, dict):
+            return
+        lst = md.setdefault(model_type, [])
+        if not isinstance(lst, list):
+            return
+        if any(isinstance(m, dict) and m.get("path") == path for m in lst):
+            return
+        entry = {"path": path, "model_type": model_type, "model_types": [model_type]}
+        if alias:
+            entry["alias"] = alias
+        lst.append(entry)
+        print(f"[upload] registered {path} in {model_type}", flush=True)
+    except Exception as exc:
+        print(f"[upload] could not register {path}: {exc}", flush=True)

codai/api/remote_gateway.py

@@ -99,8 +99,12 @@ def file_origin(filename: str) -> str:
     return (_FILE_ORIGINS or {}).get(filename, "")
 
 
-#: Handled by RemoteOpenAIBackend instead — see the module docstring.
-_SKIP_PATHS = ("/v1/chat/completions", "/v1/completions", "/v1/models")
+#: Handled by RemoteOpenAIBackend instead — see the module docstring. The two
+#: transfer endpoints are here for a different reason: an upload is addressed to
+#: the instance it is sent to (that is the whole point of sending it), so
+#: forwarding one would bounce the weights straight back out again.
+_SKIP_PATHS = ("/v1/chat/completions", "/v1/completions", "/v1/models",
+               "/v1/models/upload", "/v1/models/uploaded")
 
 #: Orchestration endpoints: a sequence of calls to other endpoints, not a model.
 #: They are NEVER forwarded as a whole — the chain stays here and each step goes
@@ -276,6 +280,9 @@ class Target:
         self.pool = pool
         self.reason = reason
         self.api_key = ""
+        #: The model entry whose weights must be pushed before serving, when the
+        #: remote has no way to fetch them (`source: "upload"`).
+        self.upload_entry = None
 
     def acquire(self):
         """Return (base_url, release). A pool provisions on demand and bills."""
@@ -313,9 +320,14 @@ def resolve_target(path: str, method: str, query: str, body: bytes,
         return Target(url=detail, reason=f"model {model!r}")
     if where == "pod":
         entry, block = detail
-        from codai.api.runpod_worker import get_model_pod_pool
-        return Target(pool=get_model_pod_pool(model, entry, block),
-                      reason=f"model {model!r} on its own RunPod pod")
+        from codai.api.runpod_worker import (get_model_pod_pool, parse_model_runpod,
+                                             resolve_model_source)
+        target = Target(pool=get_model_pod_pool(model, entry, block),
+                        reason=f"model {model!r} on its own RunPod pod")
+        kind, _ = resolve_model_source(entry, parse_model_runpod(block))
+        if kind == "upload":
+            target.upload_entry = entry
+        return target
     if not cap:
         return None
     url = capability_endpoints().get(cap, "")
@@ -396,6 +408,10 @@ class RemoteGatewayMiddleware:
             return await _error(send, 502,
                                 f"could not reach a remote for this request: {exc}")
         try:
+            # Weights the remote cannot fetch itself, when asked to upload them.
+            if target.upload_entry:
+                await asyncio.to_thread(ensure_model_uploaded, base, target.api_key,
+                                        target.upload_entry)
             # Ship any LoRA the request names, so the remote can actually load it.
             if b'"loras"' in (body or b"") and "json" in headers.get("content-type", ""):
                 body = await asyncio.to_thread(
@@ -560,6 +576,73 @@ def sync_loras(base: str, api_key: str, body: bytes) -> bytes:
     return json.dumps(obj).encode() if changed else body
 
 
+#: (base_url, model name) already pushed — a pod keeps what it was sent, so the
+#: check is worth skipping on the second request. A new pod has a new URL, so it
+#: pays the upload again: pod storage is disposable.
+_UPLOADED: set = set()
+
+
+def ensure_model_uploaded(base: str, api_key: str, entry: dict) -> None:
+    """Push a model's weights to a remote that cannot fetch them itself.
+
+    Only for `source: "upload"` — coderai resolves a HuggingFace repo id or a URL
+    first, because those cost nothing here and run at datacenter speed. Uploading
+    is the deliberate fallback for weights that exist nowhere else: a local merge,
+    a file whose host is gone, something you are not going to publish.
+    """
+    import requests
+
+    path = str((entry or {}).get("path") or "")
+    if not path or not os.path.exists(path):
+        return
+    name = os.path.basename(path.rstrip("/"))
+    if (base, name) in _UPLOADED:
+        return
+    headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
+    is_dir = os.path.isdir(path)
+    size = (os.path.getsize(path) if not is_dir else
+            sum(os.path.getsize(os.path.join(r, f))
+                for r, _, fs in os.walk(path) for f in fs))
+    try:
+        have = requests.get(f"{base}/v1/models/uploaded",
+                            params={"name": name, "bytes": 0 if is_dir else size},
+                            headers=headers, timeout=30)
+        if have.status_code == 200:
+            _UPLOADED.add((base, name))
+            return
+    except Exception as exc:
+        print(f"[remote-gateway] upload check failed for {name}: {exc}", flush=True)
+        return
+
+    section = (entry.get("model_type") or "text_models")
+    params = {"name": name, "model_type": section, "tar": 1 if is_dir else 0}
+    if entry.get("alias"):
+        params["alias"] = entry["alias"]
+    print(f"[remote-gateway] uploading {name} ({size / 1e9:.1f} GB) to {base} — "
+          "this is the cold cost of `source: upload`", flush=True)
+    try:
+        if is_dir:
+            import subprocess
+            proc = subprocess.Popen(
+                ["tar", "-cf", "-", "-C", os.path.dirname(path), name],
+                stdout=subprocess.PIPE)
+            resp = requests.post(f"{base}/v1/models/upload", params=params,
+                                 data=proc.stdout, headers=headers, timeout=None)
+            proc.stdout.close()
+            proc.wait()
+        else:
+            with open(path, "rb") as fh:
+                resp = requests.post(f"{base}/v1/models/upload", params=params,
+                                     data=fh, headers=headers, timeout=None)
+        resp.raise_for_status()
+        _UPLOADED.add((base, name))
+        print(f"[remote-gateway] uploaded {name} to {base}", flush=True)
+    except Exception as exc:
+        # Don't cache a failure: the next request retries rather than serving
+        # from a pod that has half a model.
+        print(f"[remote-gateway] upload of {name} to {base} failed: {exc}", flush=True)
+
+
 async def _error(send, status: int, message: str):
     msg = json.dumps({"error": {"message": message,
                                 "type": "remote_gateway_error"}}).encode()

docs/remote-execution.md

@@ -382,6 +382,49 @@ Triggers, most eager first:
 gated, scaled, reaped, billed against the same caps) or `serverless` (your own
 RunPod endpoint id). `served_model` renames the model for the remote.
 
+### Where a pod gets the weights
+
+A pod cannot see this machine's disk, so a local path means nothing there. One of
+these has to resolve (`source` on the model's runpod block):
+
+| `source` | The pod gets the model by |
+|---|---|
+| `auto` (default) | an explicit `hf_repo`, a path that already IS a repo id, or a repo id **recovered from the HuggingFace cache path** (`models--Owner--Repo`), then `model_url` |
+| `hf` | downloading `hf_repo` from HuggingFace |
+| `url` | downloading `model_url` directly — how most one-off GGUFs are had |
+| `upload` | receiving the weights from here |
+
+`auto` covers most of a normal catalogue without anyone typing anything: weights
+downloaded through coderai live in the HuggingFace cache, whose directory names
+still say which repo they came from.
+
+Not every server can take every source, and the mismatch is refused at
+configuration time rather than after a pod has booted for five minutes:
+
+* **vLLM** is launched `--model <id>` and can only take a HuggingFace repo id.
+* **llama.cpp** takes `-hf user/repo:QUANT` or downloads a URL with `-mu`.
+* **Upload works only on a coderai pod.** vLLM and llama.cpp need the file to
+  exist before their container starts; a coderai pod loads models on demand, so
+  it can be given them afterwards.
+
+Uploading is a deliberate choice, not a fallback: it is the cold cost of every
+NEW pod, since pod storage is disposable. On a fast symmetric link that may be
+perfectly reasonable — a 30 GB model is a few minutes at gigabit — but a
+HuggingFace repo id or a URL costs nothing here and pulls at datacenter speed. A
+RunPod network volume is the better answer for weights used repeatedly.
+
+### Keeping a pod warm
+
+`keep_warm` on a model's runpod block (or a capability's) keeps one pod running
+at all times, so no request waits for a cold boot — image pull plus weight
+download, which is minutes. **Off by default, deliberately**: a warm pod bills
+every hour of every day, including the ones nobody uses it. With it off, a pool
+scales to zero and an idle model costs nothing.
+
+Pods configured warm are started at startup rather than on first use, so the
+first request after a restart does not pay the boot that `keep_warm` exists to
+avoid.
+
 ### Several models on one pod
 
 Two models that would each rent a card can share one, until it saturates:

tests/test_admin_remote_ui.py

@@ -94,5 +94,6 @@ def test_admin_accepts_every_pod_field_the_ui_sends():
     for key in ("engine", "hf_gguf", "health_path", "docker_args",
                 "registry_auth_id", "api_key", "allow_open_pod",
                 "sticky_sessions", "max_inflight_per_pod", "on_busy",
-                "service_url", "served_model"):
+                "service_url", "served_model", "source", "hf_repo",
+                "model_url", "keep_warm", "placement"):
         assert f'"{key}"' in routes, f"admin API drops the {key} field"

tests/test_remote_execution.py

@@ -776,3 +776,77 @@ def test_an_unresolvable_lora_is_left_for_the_remote(monkeypatch):
     monkeypatch.setattr("codai.api.loras.resolve_lora_ref", lambda spec: None)
     body = json.dumps({"loras": [{"model": "org/some-lora", "weight": 1.0}]}).encode()
     assert gw.sync_loras("http://pod:8000", "", body) is body
+
+
+# --------------------------------------------------------------------------- #
+# where a pod gets the weights
+# --------------------------------------------------------------------------- #
+def test_hf_repo_id_is_recovered_from_a_local_cache_path():
+    """Most local models came from HuggingFace: the cache path still says which
+    repo, so a pod can fetch them without anyone typing an id."""
+    from codai.api.runpod_worker import resolve_model_source, parse_model_runpod
+
+    cached = {"path": "/AI/huggingface/hub/models--Qwen--Qwen3.5-9B/"
+                      "snapshots/abc123/model.safetensors"}
+    assert resolve_model_source(cached, parse_model_runpod({})) == ("hf", "Qwen/Qwen3.5-9B")
+    # An entry whose path IS a repo id needs nothing.
+    assert resolve_model_source({"path": "Qwen/Qwen3.5-9B"},
+                                parse_model_runpod({})) == ("hf", "Qwen/Qwen3.5-9B")
+    # A bare local file with nothing else: we cannot invent a source.
+    assert resolve_model_source({"path": "/AI/guffcache/merged.gguf"},
+                                parse_model_runpod({})) == ("", "")
+
+
+def test_explicit_url_and_upload_sources():
+    from codai.api.runpod_worker import resolve_model_source, parse_model_runpod
+
+    entry = {"path": "/AI/guffcache/merged-Q4.gguf"}
+    url = parse_model_runpod({"model_url": "https://host/merged-Q4.gguf"})
+    assert resolve_model_source(entry, url) == ("url", "https://host/merged-Q4.gguf")
+    up = parse_model_runpod({"source": "upload"})
+    assert resolve_model_source(entry, up) == ("upload", "/AI/guffcache/merged-Q4.gguf")
+
+
+def test_each_pod_server_gets_a_source_it_can_actually_use():
+    from codai.api.runpod_worker import (_llamacpp_docker_args, _vllm_docker_args,
+                                         parse_model_runpod)
+
+    gguf = {"path": "/AI/guffcache/merged-Q4.gguf"}
+    # llama.cpp downloads a URL itself (-mu) — the usual way to serve a one-off GGUF.
+    args = _llamacpp_docker_args(parse_model_runpod({"model_url": "https://h/m.gguf"}),
+                                 "m", gguf)
+    assert "-mu https://h/m.gguf" in args
+
+    # vLLM is launched with --model and can only take a repo id; say so rather
+    # than boot a pod for minutes and fail.
+    with pytest.raises(RuntimeError, match="cannot fetch"):
+        _vllm_docker_args(parse_model_runpod({"model_url": "https://h/m.bin"}),
+                          "/AI/local/m", {"path": "/AI/local/m"})
+    # Upload cannot work for a server that needs the file before it starts.
+    with pytest.raises(RuntimeError, match="cannot work here"):
+        _llamacpp_docker_args(parse_model_runpod({"source": "upload"}), "m", gguf)
+    # Nothing at all: a clear instruction, not a mystery.
+    with pytest.raises(RuntimeError, match="where to get the weights"):
+        _llamacpp_docker_args(parse_model_runpod({}), "m", gguf)
+
+
+def test_a_coderai_pod_is_seeded_with_something_it_can_fetch():
+    from codai.api.runpod_worker import pod_plan, parse_model_runpod
+
+    entry = {"path": "/AI/models/my-merge", "model_type": "video_models"}
+    plan = pod_plan(parse_model_runpod({"model_url": "https://host/my-merge.safetensors"}),
+                    "my-merge", "my-merge", entry=entry)
+    seeded = json.loads(plan["env"]["CODERAI_SEED_MODELS"])[0]
+    # The local path is replaced by something the pod can actually resolve.
+    assert seeded["path"] == "https://host/my-merge.safetensors"
+
+
+def test_keep_warm_is_off_by_default_and_means_one_pod():
+    from codai.api.runpod_worker import parse_model_runpod
+
+    assert parse_model_runpod({}).keep_warm is False
+    assert parse_model_runpod({}).min_pods == 0          # scale to zero: costs nothing idle
+    warm = parse_model_runpod({"keep_warm": True})
+    assert warm.keep_warm is True and warm.min_pods == 1
+    # An explicit larger min_pods is not reduced by it.
+    assert parse_model_runpod({"keep_warm": True, "min_pods": 3}).min_pods == 3