shared pools: a name is not a path (v0.3.86)

A shared pool is one pod serving several names over one copy of the weights —
a base model and its adapters. Live, every request naming the adapter got

    404 Client Error: Not Found for url: https://…/v1/chat/completions

and kept getting it: the pool keeps the pod warm, so nothing replaced it. The
pod, asked directly, listed only the base and reported {"loras": []}. Which of
the two names worked depended on which model happened to rent the pod, so it
looked intermittent rather than wrong.

Five places decided "same model?" by comparing `path`, and a LoRA's path IS its
base's path:

* the orchestrator seeded the other pool members with no served name, so they
  arrived with no alias and were registered under their path;
* the pod's boot seeder skipped the second entry as already known;
* register_model_runtime did the same, which made teaching a running pod a
  no-op that answered "already known";
* the engine-side seed filter in main.py did it a third time;
* and vLLM's service key included the served name, so had the adapter got
  through it would have started a SECOND copy of a 27B on the same card.

Identity is now (weights, name) in all five, and one engine serves one set of
weights under every name configured for it: the base under its own,
each adapter through --lora-modules, which --served-model-name must not repeat.

Adoption checks the image, the GPU class and the bearer token — none of which
say anything about the catalogue. So a pod that cannot serve what the pool
promises is now TAUGHT: the adapter bytes are pushed, the entry is registered,
and the same request is retried on the same pod. Renting a second card to serve
a name the running one can hold is the answer this avoids. Once per pod per
model; a pod that still refuses is a real failure.

An engine is also relaunched when it does not serve the name asked of it: vLLM
registers adapters at LAUNCH, and a pod is taught after it boots. Only an
ANSWERED probe that omits the name forces that — restarting a loaded 27B
because one probe timed out would be worse than the doubt.

Finally, raise_for_status() discarded the pod's body, which is the whole
message: it names the models the pod actually has. The caller saw only the
status line, which reads like a routing fault. The pod's own words now travel.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.85"
+__version__ = "0.3.86"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/models_transfer.py

@@ -168,17 +168,29 @@ def register_model_runtime(entry: dict, section: str) -> str:
     and the admin pages see it) and the model manager is told about it (so
     request validation accepts it).
     """
+    from codai.config import seed_entry_name, _same_model
+
     path = entry.get("path")
+    name = seed_entry_name(entry)
     already = False
+    # A LoRA and its base share one path, so the model this instance must be
+    # keyed by is the NAME when the path is already taken by a different one.
+    # Registering the adapter as the base silently returned "already known" and
+    # the pod went on refusing the adapter's name — teaching it was a no-op.
+    key = path
     try:
         from codai.admin.routes import config_manager
         md = getattr(config_manager, "models_data", None)
         if isinstance(md, dict):
             lst = md.setdefault(section, [])
             if isinstance(lst, list):
-                already = any(isinstance(m, dict) and m.get("path") == path
+                already = any(isinstance(m, dict) and _same_model(m, entry)
                               for m in lst)
                 if not already:
+                    if name and any(isinstance(m, dict)
+                                    and str(m.get("path") or "") == str(path or "")
+                                    for m in lst):
+                        key = name
                     lst.append(dict(entry))
     except Exception as exc:
         print(f"[register] catalogue update failed: {exc}", flush=True)
@@ -189,11 +201,15 @@ def register_model_runtime(entry: dict, section: str) -> str:
             from codai.models.manager import multi_model_manager
             cfg = {k: v for k, v in entry.items() if k not in ("path",)}
             cfg.setdefault("load_mode", "on-request")
-            getattr(multi_model_manager, setter)(path, config=cfg)
+            if key != path:
+                # The manager keys on the name it is given; the weights still
+                # have to be findable, so the path travels in the config.
+                cfg["path"] = path
+            getattr(multi_model_manager, setter)(key, config=cfg)
             if entry.get("alias"):
                 # An alias is how a request usually names it (bge-m3 vs BAAI/bge-m3).
                 try:
-                    multi_model_manager.model_aliases[str(entry["alias"])] = path
+                    multi_model_manager.model_aliases[str(entry["alias"])] = key
                 except Exception:
                     pass
             # An engine only lists models the front assigned it, and a model
@@ -203,14 +219,15 @@ def register_model_runtime(entry: dict, section: str) -> str:
                 cur = getattr(multi_model_manager, "_assigned_model_keys", None)
                 if cur is not None:
                     multi_model_manager.set_assigned_models(
-                        set(cur) | {path, entry.get("alias") or path})
+                        set(cur) | {key, entry.get("alias") or key})
             except Exception:
                 pass
         except Exception as exc:
-            print(f"[register] {path} could not be made usable: {exc}", flush=True)
+            print(f"[register] {key} could not be made usable: {exc}", flush=True)
             return "known" if already else "added"
     if not already:
-        print(f"[register] {path} registered in {section}", flush=True)
+        print(f"[register] {key} registered in {section}"
+              + (f" (adapter over {path})" if key != path else ""), flush=True)
     return "known" if already else "added"
 
 

codai/api/remote_gateway.py

@@ -1228,6 +1228,38 @@ def teach_model(base: str, api_key: str, model: str) -> bool:
         return False
 
 
+def teach_pod(base: str, api_key: str, model: str) -> bool:
+    """Make a pod able to serve ``model``: its adapters first, then its entry.
+
+    A pool hands back whichever pod it has, and a pod rented for one name of a
+    SHARED pool has never heard of the others — adoption checks the image, the
+    GPU class and the bearer token, none of which say anything about the
+    catalogue. So the pod that a request lands on may simply not know the model,
+    and says so with a 404 that is permanent: the pod is kept warm, so nothing
+    ever replaces it.
+
+    Renting a second card for a name the running one could serve is the
+    expensive answer. This is the other one: send the adapter bytes, register
+    the entry, and let the caller try again on the same pod.
+
+    ``teach_model`` alone is not enough for an adapter — it registers a content
+    id the pod does not hold yet.
+    """
+    try:
+        from codai.models.manager import _model_entry_for
+        entry = _model_entry_for(model) or {}
+    except Exception:
+        entry = {}
+    if entry:
+        try:
+            ensure_text_loras(base, api_key, entry)
+        except Exception as exc:
+            print(f"[remote-gateway] could not send {model!r}'s adapters to {base}: "
+                  f"{exc}", flush=True)
+            return False
+    return teach_model(base, api_key, model)
+
+
 def ensure_model_uploaded(base: str, api_key: str, entry: dict) -> None:
     """Push a model's weights to a remote that cannot fetch them itself.
 

codai/api/runpod_worker.py

@@ -1264,7 +1264,7 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
         if one:
             seeds.extend(_json.loads(one))
     for extra in (seed_entries or []):
-        one = seed_model_env(extra, "", "")
+        one = seed_model_env(extra, seed_served_name(extra), "")
         if one:
             seeds.extend(_json.loads(one))
     if seeds:
@@ -1602,6 +1602,23 @@ def _account_hint():
         return None
 
 
+def seed_served_name(entry: dict) -> str:
+    """The name a pod must answer to for a seed entry that is not the pool's own.
+
+    The pool's OWN model is seeded with the name the pool serves it under; the
+    others were seeded with nothing, so `seed_model_env` set no alias and the pod
+    registered them under their `path`. For a LoRA that is fatal rather than
+    cosmetic: an adapter's path IS its base, so the second name over one set of
+    weights collapsed into the first and the pod answered 404 under it for the
+    rest of its paid life. Which of the two broke depended on which model
+    happened to rent the pod.
+    """
+    block = entry.get("runpod") if isinstance(entry.get("runpod"), dict) else {}
+    return (str(block.get("served_model") or "").strip()
+            or str(entry.get("id") or "").strip()
+            or str(entry.get("alias") or "").strip())
+
+
 def seed_model_env(entry: dict, served: str = "", source: str = "",
                    volume_path: str = "", engine: str = "") -> str:
     """The model entry to register on a fresh pod, as JSON for CODERAI_SEED_MODELS.

codai/api/vllm_worker.py

@@ -127,17 +127,36 @@ def resolve_service_key(cfg, model_path: Optional[str] = None):
     return mp, key
 
 
+def service_key_for(cfg, model_path=None, served_name=None):
+    """The key for the vLLM instance that serves ``served_name``.
+
+    ONE engine per set of weights, not one per name. vLLM registers adapters at
+    launch and serves each under its own name, so a base and its LoRAs are one
+    instance — keying on the name as well meant a second request for the same
+    weights under a different name started a SECOND copy of the model on the
+    same card. On a 48 GB card holding a 27B that is not a slowdown, it is an
+    out-of-memory crash, and the first engine is the one that dies.
+
+    ``served_name`` is still what the instance must ANSWER to; see
+    :func:`_ensure_service_once`, which relaunches an engine that does not.
+    """
+    _resolved, key = resolve_service_key(cfg, model_path)
+    return key
+
+
 def _launch_cmd(py, cfg, host: str, port: int, model_path: str,
                 served_name: Optional[str] = None, model_config: dict = None,
                 gpu_memory_utilization: Optional[float] = None,
                 max_model_len: Optional[int] = None,
                 max_num_batched_tokens: Optional[int] = None,
                 max_num_seqs: Optional[int] = None) -> list:
-    mid = served_name or (getattr(cfg, "model_id", "vllm") or "vllm")
+    mids = _served_names(model_config,
+                         served_name or (getattr(cfg, "model_id", "") or ""),
+                         model_path)
     cmd = [py, "-m", "vllm.entrypoints.openai.api_server",
            "--host", host, "--port", str(port),
            "--model", model_path,
-           "--served-model-name", mid]
+           "--served-model-name"] + mids
     # A side job serving a small model (the OCR VLM engines) overrides the context the
     # LLM backend is configured with: it inherited ctx=18432, and the activation peak
     # that size profiles for was 9.4 GiB of the 11.1 GiB that made surya-ocr-2 unable to
@@ -238,6 +257,36 @@ def _sibling_lora_specs(model_config: dict) -> list:
     return unique
 
 
+def _served_names(model_config: dict, served_name: str, fallback: str) -> list:
+    """Every name this ONE engine must answer to.
+
+    One engine serves a base and each adapter of it, so the launch cannot name
+    only the model whose request happened to start it: whichever of the two
+    arrived second would be unreachable, which is the failure this whole path
+    exists to stop. Adapter names come from ``--lora-modules`` and must NOT be
+    repeated here — vLLM would then have a model and an adapter under one name.
+    """
+    adapters = {str(s.get("name") or "") for s in _sibling_lora_specs(model_config or {})}
+    names = [str(served_name or "")]
+    base = _base_of(model_config or {})
+    if base:
+        try:
+            from codai.admin.routes import config_manager
+            md = getattr(config_manager, "models_data", None) or {}
+        except Exception:
+            md = {}
+        for entry in (md.get("text_models") or []):
+            if not isinstance(entry, dict) or _base_of(entry) != base:
+                continue
+            names.append(str(entry.get("alias") or entry.get("id") or "").strip())
+    out, seen = [], set()
+    for n in names:
+        if n and n not in seen and n not in adapters:
+            seen.add(n)
+            out.append(n)
+    return out or [fallback]
+
+
 def _lora_args(model_config: dict) -> list:
     """`--enable-lora --lora-modules …` for the adapters this engine must serve."""
     try:
@@ -293,6 +342,27 @@ def _model_config_for(model_name: str) -> dict:
         return {}
 
 
+def _serves_name(url: str, name: str):
+    """True / False / None — the last meaning "could not tell".
+
+    The difference matters: not serving a name costs a relaunch, and failing to
+    ASK looks identical from here. Restarting a loaded 27B because one probe
+    timed out would be far worse than leaving the doubt, so an unanswered probe
+    is never a verdict.
+    """
+    import requests
+    try:
+        r = requests.get(url + "/v1/models", timeout=5)
+        data = r.json().get("data")
+    except Exception:
+        return None
+    if not isinstance(data, list):
+        return None
+    ids = [str(m.get("id", "")) for m in data if isinstance(m, dict)]
+    tail = name.rstrip("/").split("/")[-1]
+    return any(i == name or i.rstrip("/").split("/")[-1] == tail for i in ids)
+
+
 def _remote_serves(url: str, name: str) -> bool:
     """True when a remote vLLM's /v1/models lists ``name`` (or a suffix match)."""
     import requests
@@ -384,13 +454,28 @@ def _ensure_service_once(cfg, model_path: Optional[str] = None,
         return _remote
 
     resolved, svc_key = resolve_service_key(cfg, model_path)
-    if served_name:
-        svc_key = f"{svc_key}|{served_name}"
     with _lock:
         svc = _services.get(svc_key)
         if svc and svc["proc"].poll() is None and _health_ok(svc["url"]):
-            return svc["url"]
-        if svc:
+            # Alive is not the same as useful. An engine serves the names it was
+            # LAUNCHED with, and this instance may have started before the name
+            # now being asked for existed here at all — a pod is taught models
+            # after it boots, and an adapter registered afterwards is not in any
+            # running engine's --lora-modules. Reusing it then answers 404 for
+            # as long as the engine lives.
+            #
+            # Only an answered probe that omits the name forces a relaunch: a
+            # probe that fails says nothing, and restarting a loaded 27B on a
+            # timeout would be far worse than the doubt.
+            if not served_name or _serves_name(svc["url"], served_name) is not False:
+                return svc["url"]
+            print(f"[vllm] the engine on {svc['url']} does not serve "
+                  f"{served_name!r} — it started before this model was known "
+                  f"here; restarting it so one engine serves the base and every "
+                  f"adapter of it", flush=True)
+            _services.pop(svc_key, None)
+            _stop_proc(svc, svc_key)
+        elif svc:
             _services.pop(svc_key, None)
 
         py = ensure_built(cfg)
@@ -628,8 +713,6 @@ def _shrink_candidate(gmu: float, kv_gib: float, target_kv_gb: float,
 
 def _service_meta(cfg, model_path=None, served_name=None) -> dict:
     _resolved, svc_key = resolve_service_key(cfg, model_path)
-    if served_name:
-        svc_key = f"{svc_key}|{served_name}"
     with _lock:
         svc = _services.get(svc_key) or {}
     return svc.get("meta") or {}
@@ -719,9 +802,13 @@ def ensure_service(cfg, model_path: Optional[str] = None,
     raise last if last is not None else RuntimeError("vLLM did not start")
 
 
-def stop_service(svc_key: str) -> None:
-    with _lock:
-        svc = _services.pop(svc_key, None)
+def _stop_proc(svc: dict, svc_key: str = "") -> None:
+    """Tear a service's process (and its ray cluster) down.
+
+    Separate from :func:`stop_service` because it takes no lock: a caller that
+    already holds ``_lock`` — replacing an engine in place — would deadlock on
+    it.
+    """
     if not svc:
         return
     proc = svc["proc"]
@@ -741,7 +828,13 @@ def stop_service(svc_key: str) -> None:
             ray.stop()
         except Exception as exc:
             print(f"[vllm] ray teardown for {svc_key} failed: {exc}", flush=True)
-    print(f"[vllm] service for {svc_key} stopped", flush=True)
+    print(f"[vllm] service for {svc_key or svc.get('url')} stopped", flush=True)
+
+
+def stop_service(svc_key: str) -> None:
+    with _lock:
+        svc = _services.pop(svc_key, None)
+    _stop_proc(svc, svc_key)
 
 
 def stop_all() -> None:
@@ -752,8 +845,6 @@ def stop_all() -> None:
 def is_running(cfg, model_path: Optional[str] = None, served_name: Optional[str] = None) -> bool:
     """True if a vLLM service for this (model_path, served_name) is currently up."""
     _resolved, svc_key = resolve_service_key(cfg, model_path)
-    if served_name:
-        svc_key = f"{svc_key}|{served_name}"
     with _lock:
         svc = _services.get(svc_key)
         return bool(svc and svc["proc"].poll() is None)
@@ -880,8 +971,6 @@ def stop_service_for(cfg, model_path: Optional[str] = None,
     VRAM. Returns an estimate of the GB it was holding (gpu_memory_utilization × total
     VRAM), 0.0 if nothing was running."""
     _resolved, svc_key = resolve_service_key(cfg, model_path)
-    if served_name:
-        svc_key = f"{svc_key}|{served_name}"
     with _lock:
         svc = _services.get(svc_key)
         running = bool(svc and svc["proc"].poll() is None)

codai/backends/runpod.py

@@ -28,6 +28,32 @@ from typing import AsyncGenerator, Dict, List, Optional
 from codai.backends.base import ModelBackend
 
 
+class RemoteModelUnknown(RuntimeError):
+    """The pod answered that it does not know the model we asked for."""
+
+
+def _pod_detail(r) -> str:
+    """What the pod SAID, not what requests says about the status line.
+
+    ``raise_for_status()`` produces "404 Client Error: Not Found for url: …" and
+    throws the body away — and the body is the whole message: it names the models
+    the pod actually has. A pod that had never been told about a model therefore
+    reached the caller as something indistinguishable from a routing fault.
+    """
+    try:
+        body = r.json()
+    except Exception:
+        return (getattr(r, "text", "") or "").strip()[:400]
+    if isinstance(body, dict):
+        err = body.get("error")
+        if isinstance(err, dict) and err.get("message"):
+            return str(err["message"])[:400]
+        for key in ("detail", "message", "error"):
+            if body.get(key):
+                return str(body[key])[:400]
+    return str(body)[:400]
+
+
 class RunpodBackend(ModelBackend):
     """Thin OpenAI proxy to a model hosted on RunPod."""
 
@@ -203,6 +229,32 @@ class RunpodBackend(ModelBackend):
         key = getattr(pool, "api_key", "") or ""
         return {"Authorization": f"Bearer {key}"} if key else {}
 
+    def _check(self, r) -> None:
+        """raise_for_status(), keeping what the pod said and naming the one case
+        we can do something about."""
+        if r.ok:
+            return
+        from codai.api.remote_gateway import _is_unknown_model
+        import requests
+        detail = _pod_detail(r)
+        if _is_unknown_model(r.status_code, (detail or "").encode("utf-8", "replace")):
+            raise RemoteModelUnknown(detail)
+        raise requests.HTTPError(
+            f"the pod answered {r.status_code}: {detail}", response=r)
+
+    def _teach(self, base: str) -> bool:
+        """Tell this pod about our model after it said it has never heard of it."""
+        try:
+            from codai.api.remote_gateway import teach_pod
+        except Exception:
+            return False
+        name = self._model_id or self._served_name
+        pool = getattr(self, "_pool", None)
+        key = getattr(pool, "api_key", "") if pool is not None else ""
+        # The pool's base ends in /v1; teach_pod posts to <root>/v1/models/register.
+        root = base[:-3].rstrip("/") if base.endswith("/v1") else base
+        return bool(teach_pod(root, key, name))
+
     def openai_endpoint(self):
         """``(base_url ending /v1, api_key)`` for a client that will speak to this
         model over HTTP ITSELF, instead of going through us.
@@ -318,7 +370,19 @@ class RunpodBackend(ModelBackend):
                         raise
                     continue
                 try:
-                    r.raise_for_status()
+                    try:
+                        self._check(r)
+                    except RemoteModelUnknown as unknown:
+                        # The pod does not know this model. That is recoverable
+                        # exactly once: teach it and ask the SAME pod again,
+                        # rather than pay for a second card to serve a name the
+                        # running one can hold.
+                        if attempt == 2 or not self._teach(base):
+                            raise
+                        print(f"[runpod] pod did not know "
+                              f"{self._served_name!r} ({unknown}) — taught it, "
+                              f"retrying", flush=True)
+                        continue
                     data = r.json()
                     self._store_usage(data.get("usage", {}))
                     choice = (data.get("choices") or [{}])[0]
@@ -350,9 +414,11 @@ class RunpodBackend(ModelBackend):
                     payload["response_format"] = response_format
                 if tools:
                     payload["tools"] = tools
+                sent = False
                 try:
                     async for chunk in self._stream(base + "/chat/completions", payload,
                                                     delta_key="delta"):
+                        sent = True
                         yield chunk
                 except requests.exceptions.ConnectionError:
                     # Refused before a byte arrived: the pod is gone. Drop it
@@ -361,6 +427,16 @@ class RunpodBackend(ModelBackend):
                     if attempt == 2:
                         raise
                     continue
+                except RemoteModelUnknown as unknown:
+                    # The refusal happens before the first byte, so going again
+                    # replays nothing — but check rather than assume, because a
+                    # retry after output has started would duplicate it.
+                    if sent or attempt == 2 or not await asyncio.to_thread(
+                            self._teach, base):
+                        raise
+                    print(f"[runpod] pod did not know {self._served_name!r} "
+                          f"({unknown}) — taught it, retrying", flush=True)
+                    continue
                 finally:
                     release()
                 return
@@ -419,7 +495,9 @@ class RunpodBackend(ModelBackend):
             try:
                 with pod_http.post(url, json=payload, headers=headers,
                                    stream=True, timeout=3600) as r:
-                    r.raise_for_status()
+                    # Not raise_for_status(): it discards the body, which on a
+                    # 404 is the pod telling us which models it actually has.
+                    self._check(r)
                     buf = b""
                     done = False
                     for bchunk in r.iter_content(chunk_size=8192):

codai/config.py

@@ -1462,6 +1462,33 @@ class Config:
     parser: str = "auto"
 
 
+def seed_entry_name(entry: dict) -> str:
+    """The name a seeded entry is asked for by — not necessarily its path."""
+    if not isinstance(entry, dict):
+        return ""
+    return (str(entry.get("alias") or "").strip()
+            or str(entry.get("id") or "").strip())
+
+
+def _same_model(known: dict, seed: dict) -> bool:
+    """Is ``seed`` the model ``known`` already is?
+
+    Not "same path". A LoRA and the base it adapts share one path BY DEFINITION
+    — the adapter is a few MB of deltas over the base weights — so keying
+    identity on the path alone made the second of the two vanish into the first.
+    A pod seeded with both was told about one, and answered 404 under the other
+    name for the rest of its paid life. Which one survived depended on the order
+    they arrived in, so it looked intermittent rather than wrong.
+
+    Identity is (weights, name): the same path under a DIFFERENT name is another
+    model that happens to load from the same place.
+    """
+    if str(known.get("path") or "") != str(seed.get("path") or ""):
+        return False
+    a, b = seed_entry_name(known), seed_entry_name(seed)
+    return not a or not b or a == b
+
+
 class ConfigManager:
     """Manages configuration loading, saving, and validation."""
     
@@ -2318,10 +2345,11 @@ class ConfigManager:
             if not isinstance(lst, list):
                 continue
             path = str(e["path"])
-            if any(isinstance(m, dict) and m.get("path") == path for m in lst):
+            if any(isinstance(m, dict) and _same_model(m, e) for m in lst):
                 continue            # a real entry already covers it — leave it alone
             lst.append(dict(e))
-            print(f"[seed] registered {path} in {section}", flush=True)
+            print(f"[seed] registered {seed_entry_name(e) or path} in {section}",
+                  flush=True)
         # The catalogue alone is not enough: request validation asks the MODEL
         # MANAGER for its allowed identifiers, and a seeded pod whose manager was
         # never told still answers "not available" with an empty list. The

codai/main.py

@@ -882,9 +882,11 @@ def main():
         for _e in _seeded:
             _sec = _e.get("model_type") or "text_models"
             _lst = models_config.setdefault(_sec, [])
+            # (path, name), not path: a LoRA and its base share one path, so
+            # matching on the path alone dropped the adapter here too.
+            from codai.config import _same_model
             if isinstance(_lst, list) and not any(
-                    isinstance(x, dict) and x.get("path") == _e.get("path")
-                    for x in _lst):
+                    isinstance(x, dict) and _same_model(x, _e) for x in _lst):
                 _lst.append(_e)
             _keys.add(_e.get("alias") or _e.get("path"))
         try:

docs/runpod.md

@@ -390,6 +390,33 @@ an entry recording neither image nor card — written before this check existed
 is refused rather than trusted: one extra pod once, against a silent answer from
 the wrong code. Every refusal says which pod and why.
 
+None of those three says anything about the pod's **catalogue**, and a *shared*
+pool promises several names over one copy of the weights. A pod rented for one
+of them has never heard of the others, so the first request naming a sibling
+got a 404 — and because the pool keeps the pod warm, that 404 was permanent.
+The pod is now taught instead: the adapter bytes are pushed to it, the entry is
+registered, and the SAME request is retried on the SAME pod. Renting a second
+card to serve a name the running one can hold is the answer this avoids.
+
+```
+[runpod] pod did not know 'tuned-alpha' (Model 'tuned-alpha' is not available.
+         Use one of: base-awq) — taught it, retrying
+```
+
+Teaching is tried once per pod per model. A pod that still refuses afterwards
+fails with its OWN message rather than with `404 Client Error: Not Found for
+url: …`, which said nothing about the catalogue and read like a network fault.
+
+#### A name is not a path
+
+A LoRA and the base it adapts load from the same weights, so a model entry's
+`path` cannot identify it — the adapter and its base have the same one. Identity
+is **(weights, name)** everywhere a model is seeded, registered or served, and
+every engine serves one set of weights under every name configured for it: the
+base under its own, each adapter through vLLM's `--lora-modules`. Keying on the
+path alone made the second name vanish into the first; keying the ENGINE on the
+name as well would have started a second copy of the model on the same card.
+
 Every pod's hours reach the ledger, whichever way it ends: a pool retires one
 (`_terminate`), RunPod takes one away (`forget`), the reaper kills a stale one,
 or it disappears entirely — terminated from the console, or straight through the

tests/test_burst_placement.py

@@ -1271,6 +1271,7 @@ def test_a_pod_that_refuses_connections_is_dropped_and_the_request_retried(monke
     calls = []
     class _R:
         status_code = 200
+        ok = True
         def raise_for_status(self): pass
         def json(self): return {"choices": [{"message": {"content": "hi"}}], "usage": {}}
     def _post(url, **kw):

tests/test_ocr_vram_share_and_fallback.py

@@ -167,13 +167,16 @@ def test_stopping_reports_what_was_reserved_not_the_engines_share(monkeypatch):
     stopped = []
     monkeypatch.setattr(vllm_worker, "stop_service", lambda k: stopped.append(k))
     launched = vllm_worker.effective_gmu(0.60)          # 0.60 -> a 14.4 GB reservation
-    monkeypatch.setitem(vllm_worker._services, "m|m",
+    # Keyed by the WEIGHTS, not by the name they are served under: one engine
+    # serves a base and every adapter of it, so a second name over the same
+    # path must find the same instance rather than start another copy.
+    monkeypatch.setitem(vllm_worker._services, "m",
                         {"proc": _Proc(), "port": 1, "url": "u", "ray": None,
                          "gmu": launched})
     cfg = type("C", (), {"model_path": "m", "gpu_memory_utilization": 0.0})()
     freed = vllm_worker.stop_service_for(cfg, model_path="m", served_name="m",
                                          gpu_memory_utilization=0.10)
-    assert stopped == ["m|m"]
+    assert stopped == ["m"]
     assert freed == pytest.approx(launched * 24.0, abs=0.05)     # ~14.4 GB, what it took
     assert freed > 0.10 * 24.0                      # not the (stale) share the caller passed
 

tests/test_pod_teach_on_unknown_model.py

+"""A pod that has never heard of the model is taught, not written off.
+
+A pool hands back whichever pod it holds. Adoption checks the image, the GPU
+class and the bearer token — none of which say anything about the CATALOGUE —
+so a pod rented for one name of a shared pool is handed to a request naming
+another, and answers 404. The pool keeps it warm, so that 404 is permanent.
+
+Two separate failures made it unreadable as well as fatal:
+
+* `raise_for_status()` threw the pod's body away, so the only thing that
+  reached the caller was "404 Client Error: Not Found for url: …" — which reads
+  like a routing or network fault, not a catalogue one. The body says exactly
+  which models the pod has.
+* nothing acted on it. The remote GATEWAY teaches a pod that says "not
+  available" and retries, but /v1/chat/completions is in its _SKIP_PATHS: text
+  generation goes through the in-process RunPod backend, which had no such
+  recovery.
+"""
+
+import pathlib
+import sys
+
+import pytest
+
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT))
+
+from codai.backends import runpod as rb                        # noqa: E402
+
+
+class _Pod:
+    pod_id = "pod-1"
+    url = "https://203.0.113.5:1"
+
+
+class _Pool:
+    api_key = "cra-test"
+
+    def acquire(self, timeout=1200.0, affinity=""):
+        return _Pod(), _Pod.url
+
+    def release(self, pod):
+        pass
+
+    def discard(self, pod, reason=""):
+        pass
+
+
+def _backend():
+    b = rb.RunpodBackend.__new__(rb.RunpodBackend)
+    b._mode = "pods"
+    b._pool = _Pool()
+    b._headers = {}
+    b._url = ""
+    b._model_id = "tuned-alpha"
+    b._served_name = "tuned-alpha"
+    b._last_usage = {}
+    b._last_logprobs = None
+    b._enter_request = lambda: None
+    b._exit_request = lambda: None
+    return b
+
+
+class _Reply:
+    def __init__(self, status, payload):
+        self.status_code = status
+        self.ok = status < 400
+        self._payload = payload
+        self.text = str(payload)
+
+    def json(self):
+        return self._payload
+
+
+UNKNOWN = {"detail": "Model 'tuned-alpha' is not available. Use one of: base-awq"}
+
+
+# --------------------------------------------------------------------------- #
+def test_the_pods_own_words_reach_the_caller(monkeypatch):
+    b = _backend()
+    monkeypatch.setattr(rb.pod_http, "post",
+                        lambda url, **kw: _Reply(400, {"detail": "context too long"}))
+    with pytest.raises(Exception) as err:
+        b.generate_chat([{"role": "user", "content": "x"}])
+    assert "context too long" in str(err.value)
+    assert "Client Error" not in str(err.value)
+
+
+def test_an_unknown_model_is_taught_and_the_request_retried(monkeypatch):
+    b = _backend()
+    replies = iter([_Reply(404, UNKNOWN),
+                    _Reply(200, {"choices": [{"message": {"content": "hi"}}],
+                                 "usage": {}})])
+    monkeypatch.setattr(rb.pod_http, "post", lambda url, **kw: next(replies))
+
+    taught = []
+
+    def _teach(base, key, model):
+        taught.append((base, key, model))
+        return True
+
+    import codai.api.remote_gateway as gw
+    monkeypatch.setattr(gw, "teach_pod", _teach)
+
+    assert b.generate_chat([{"role": "user", "content": "x"}]) == "hi"
+    assert len(taught) == 1
+    base, key, model = taught[0]
+    # The pod ROOT, not the /v1 the pool hands out: register lives at
+    # <root>/v1/models/register and a doubled /v1 is a 404 of its own.
+    assert base == _Pod.url
+    assert key == "cra-test"
+    assert model == "tuned-alpha"
+
+
+def test_a_pod_that_cannot_be_taught_fails_with_its_own_message(monkeypatch):
+    b = _backend()
+    monkeypatch.setattr(rb.pod_http, "post", lambda url, **kw: _Reply(404, UNKNOWN))
+    import codai.api.remote_gateway as gw
+    monkeypatch.setattr(gw, "teach_pod", lambda *a, **k: False)
+
+    with pytest.raises(rb.RemoteModelUnknown) as err:
+        b.generate_chat([{"role": "user", "content": "x"}])
+    assert "is not available" in str(err.value)
+
+
+def test_teaching_is_tried_once_not_forever(monkeypatch):
+    """A pod that still refuses after being taught is a real failure."""
+    b = _backend()
+    posts = []
+
+    def _post(url, **kw):
+        posts.append(url)
+        return _Reply(404, UNKNOWN)
+
+    monkeypatch.setattr(rb.pod_http, "post", _post)
+    import codai.api.remote_gateway as gw
+    monkeypatch.setattr(gw, "teach_pod", lambda *a, **k: True)
+
+    with pytest.raises(rb.RemoteModelUnknown):
+        b.generate_chat([{"role": "user", "content": "x"}])
+    assert len(posts) == 2
+
+
+def test_a_plain_404_that_is_not_about_the_model_is_not_retried(monkeypatch):
+    b = _backend()
+    posts = []
+
+    def _post(url, **kw):
+        posts.append(url)
+        return _Reply(404, {"detail": "no such route"})
+
+    monkeypatch.setattr(rb.pod_http, "post", _post)
+    import codai.api.remote_gateway as gw
+    monkeypatch.setattr(gw, "teach_pod",
+                        lambda *a, **k: pytest.fail("should not teach"))
+
+    with pytest.raises(Exception) as err:
+        b.generate_chat([{"role": "user", "content": "x"}])
+    assert "no such route" in str(err.value)
+    assert len(posts) == 1
+
+
+def test_an_openai_shaped_error_body_is_read_too():
+    r = _Reply(401, {"error": {"message": "Invalid API key.",
+                               "code": "invalid_api_key"}})
+    assert rb._pod_detail(r) == "Invalid API key."
+
+
+def test_teach_pod_sends_the_adapter_before_registering_the_entry(monkeypatch):
+    """Registering alone names a content id the pod does not hold yet."""
+    import codai.api.remote_gateway as gw
+
+    order = []
+    monkeypatch.setattr(gw, "ensure_text_loras",
+                        lambda base, key, entry: order.append("loras"))
+    monkeypatch.setattr(gw, "teach_model",
+                        lambda base, key, model: order.append("register") or True)
+    import codai.models.manager as mgr
+    monkeypatch.setattr(mgr, "_model_entry_for",
+                        lambda name: {"id": name, "loras": [{"name": name}]},
+                        raising=False)
+
+    assert gw.teach_pod("https://pod", "k", "tuned-alpha") is True
+    assert order == ["loras", "register"]
+
+
+def test_teach_pod_does_not_register_when_the_adapter_cannot_be_sent(monkeypatch):
+    """A pod with the entry but not the bytes fails at load, somewhere nobody looks."""
+    import codai.api.remote_gateway as gw
+
+    def _boom(*a, **k):
+        raise RuntimeError("upload refused")
+
+    monkeypatch.setattr(gw, "ensure_text_loras", _boom)
+    monkeypatch.setattr(gw, "teach_model",
+                        lambda *a, **k: pytest.fail("should not register"))
+    import codai.models.manager as mgr
+    monkeypatch.setattr(mgr, "_model_entry_for", lambda name: {"id": name},
+                        raising=False)
+
+    assert gw.teach_pod("https://pod", "k", "tuned-alpha") is False

tests/test_proxy_logprobs.py

@@ -36,6 +36,7 @@ class _Reply:
     #: A real response reports its code, and the backend reads it so that a
     #: server's own explanation of a 4xx survives the trip up.
     status_code = 200
+    ok = True
     reason = "OK"
     url = "http://stub/v1/chat/completions"