a pod could not say why it had nowhere to send a page (v0.4.3)

A rented pod answered, for a model it was serving perfectly well over its
own loopback:

    model 'datalab-to/surya-ocr-2' is not served over HTTP by this install

Not rasterisation: a PNG, which skips rasterising entirely, failed the same
way in 0.26s. The pipeline on the pod needs a LOCAL placement candidate,
where the pipeline on the orchestrator needs a POD one -- so the local path
is the one that had never run.

Two defects, and the second is why the first took all night.

1. `_model_entry_for` reads the catalogue from one place: the ConfigManager
   the admin module holds. A pod has no models.json -- its catalogue is
   seeded from CODERAI_SEED_MODELS into memory -- so any process or import
   order where that global is not that object sees an EMPTY catalogue. The
   seeded model then has no backend pin, resolves to no engine, yields no
   candidate, and the pod disowns a model it is serving. It now falls back
   to the seed list in the environment, which needs no global and no import
   order. A real catalogue entry still wins.

2. `_local_candidate` ended in `except Exception: return None`, and
   `_pod_candidate` too. Every decline now records a reason -- which of the
   catalogue entry, the backend pin or the enabled flag was missing, or the
   exception -- says it once, and clears it when the install starts working.

And the reason travels IN THE ERROR, not only in a log. The machine that
reads a pod's 503 is not the machine that can read its log; a pod's log is
not reachable from the orchestrator at all. Blaming configuration for what
may be a lookup, in the one sentence that does cross the network, is what
made this invisible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.2"
+__version__ = "0.4.3"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/models/manager.py

@@ -260,6 +260,49 @@ def _engine_model_id(engine: str) -> str:
     return (getattr(cfg, "model_id", "") or "").lower() if cfg else ""
 
 
+def _names_of_entry(m: dict) -> set:
+    """Everything a model entry answers to, lowercased."""
+    path = str(m.get("path") or m.get("id") or "")
+    return {path.lower(), os.path.basename(path.rstrip("/")).lower(),
+            str(m.get("alias") or "").lower(), str(m.get("id") or "").lower()}
+
+
+def _env_seed_entries() -> list:
+    """The entries this process was SEEDED with, read from the environment.
+
+    A rented pod has no models.json: its catalogue arrives in
+    CODERAI_SEED_MODELS and is registered in memory, on the ConfigManager the
+    admin module happens to hold. Any process (or import order) where that
+    global is not that object sees an EMPTY catalogue — and then a seeded
+    model has no `backend` pin, resolves to no engine, produces no placement
+    candidate, and the pod answers "not served over HTTP by this install"
+    about a model it is serving perfectly well over loopback.
+
+    Reading the environment needs no global and no import order. It is the
+    same list the seeder used, so the two cannot disagree.
+    """
+    raw = os.environ.get("CODERAI_SEED_MODELS", "").strip()
+    if not raw:
+        return []
+    global _SEED_CACHE
+    cached = globals().get("_SEED_CACHE")
+    if cached is not None and cached[0] == raw:
+        return cached[1]
+    try:
+        import json as _json
+        parsed = _json.loads(raw)
+        if isinstance(parsed, dict):
+            parsed = [parsed]
+        out = [e for e in parsed if isinstance(e, dict) and e.get("path")]
+    except Exception:
+        out = []
+    _SEED_CACHE = (raw, out)
+    return out
+
+
+_SEED_CACHE = None
+
+
 def _model_entry_for(model_name: str):
     """The models.json entry (dict) matching ``model_name`` by path / basename /
     alias / id, or None."""
@@ -276,14 +319,16 @@ def _model_entry_for(model_name: str):
                 for m in lst:
                     if not isinstance(m, dict):
                         continue
-                    path = str(m.get("path") or m.get("id") or "")
-                    base = os.path.basename(path.rstrip("/")).lower()
-                    cands = {path.lower(), base, str(m.get("alias") or "").lower(),
-                             str(m.get("id") or "").lower()}
-                    if name in cands:
+                    if name in _names_of_entry(m):
                         return m
     except Exception:
         pass
+    # Last resort, and only ever non-empty on a seeded pod: see
+    # _env_seed_entries(). Deliberately AFTER the catalogue, so a real entry
+    # (which an operator may have edited) always wins.
+    for m in _env_seed_entries():
+        if name in _names_of_entry(m):
+            return m
     return None
 
 

codai/models/placement.py

@@ -172,21 +172,71 @@ def forget(model_id: str = "") -> None:
 
 
 # ------------------------------------------------------------------ discovery #
+#: model id -> {kind: why that kind of candidate was not built}. Kept so the
+#: ERROR can carry it, not just a log: a rented pod's log is not reachable from
+#: the machine that gets the 503, and "is not served over HTTP by this install"
+#: describes configuration for what may be a lookup, a failed launch or a
+#: missing catalogue entry. One night was spent on the difference.
+_declines: dict = {}
+
+
+def _decline(model_id: str, kind: str, reason: str) -> None:
+    """Record (and say once) why a kind of candidate could not be built."""
+    with _lock:
+        per = _declines.setdefault(model_id, {})
+        said = per.get(kind)
+        per[kind] = reason
+    if said != reason:
+        print(f"[placement] {model_id!r}: no {kind} candidate — {reason}",
+              flush=True)
+
+
+def why_no_candidates(model_id: str) -> str:
+    """The recorded reasons, as one sentence, or ''."""
+    with _lock:
+        per = dict(_declines.get(model_id) or {})
+    if not per:
+        return ""
+    return "; ".join(f"{k}: {v}" for k, v in sorted(per.items()))
+
+
 def _local_candidate(model_id: str) -> Optional[Candidate]:
     """This install's own vLLM, when it is what serves the model."""
     try:
         from codai.models import manager as mm
-        if mm.resolve_engine_backend(model_id) != "vllm":
+        engine = mm.resolve_engine_backend(model_id)
+        if engine != "vllm":
+            # The three things that decide this, named separately: a missing
+            # catalogue entry, a pin to something else, and a disabled vLLM all
+            # arrive here as the same None, and they need different fixes. On a
+            # pod the entry is seeded into memory rather than read from
+            # models.json, so "NOT in this process's catalogue" is a real and
+            # non-obvious answer.
+            try:
+                found = ("found" if mm._model_entry_for(model_id) is not None
+                         else "NOT FOUND in this process")
+                pin = mm._pinned_backend(model_id) or "none"
+                on = mm._engine_enabled("vllm")
+            except Exception as exc:
+                found, pin, on = f"could not be looked up ({exc})", "?", "?"
+            _decline(model_id, "local",
+                     "resolved to %s, not vllm (catalogue entry %s, backend "
+                     "pin %r, vllm enabled=%s)"
+                     % (engine or "no managed engine", found, pin, on))
             return None
         from codai.api import runpod_worker, vllm_worker
         from codai.models.manager import get_active_vllm_config
         vcfg = get_active_vllm_config()
         path = (runpod_worker._model_path_for(model_id) or "").strip() or model_id
         base = vllm_worker.ensure_service(vcfg, model_path=path, served_name=model_id)
-    except Exception:
+    except Exception as exc:
+        _decline(model_id, "local", f"{type(exc).__name__}: {exc}")
         return None
     if not base:
+        _decline(model_id, "local", "the vLLM service returned no base URL")
         return None
+    with _lock:
+        _declines.pop(model_id, None)
     return Candidate(key=f"local:{model_id}", kind="local",
                      base=base.rstrip("/") + "/v1", priority=0, primary=True)
 
@@ -287,11 +337,15 @@ def _pod_candidate(model_id: str) -> Optional[Candidate]:
     try:
         from codai.models import manager as mm
         if not mm.runpod_should_handle(model_id):
+            _decline(model_id, "pod",
+                     "this install does not route it to RunPod (no "
+                     "`backend: runpod` on its entry, or no entry here)")
             return None
         from codai.backends.runpod import RunpodBackend
         backend = RunpodBackend(mm.get_active_runpod_config())
         backend.load_model(model_id)
-    except Exception:
+    except Exception as exc:
+        _decline(model_id, "pod", f"{type(exc).__name__}: {exc}")
         return None
 
     def _acquire():
@@ -429,11 +483,18 @@ def choose(model_id: str, spill_after: int = 0) -> Lease:
     """
     cands = candidates(model_id)
     if not cands:
+        # Carry WHY in the error, not only in a log. The machine that reads
+        # this message is often not the machine that produced it — a rented
+        # pod's log is not reachable from the orchestrator that gets its 503 —
+        # and without the reasons this sentence blames configuration for what
+        # may be a failed launch or a catalogue the process cannot see.
+        why = why_no_candidates(model_id)
         raise OcrError(
             f"model {model_id!r} is not served over HTTP by this install, so an OCR "
             f"engine cannot attach to it. Pin the model to a backend that has an "
             f"endpoint (backend: \"runpod\", a cluster node that holds it, or a "
-            f"service_url), or pick a serve mode that fits this box.",
+            f"service_url), or pick a serve mode that fits this box."
+            + (f" [why: {why}]" if why else ""),
             status=503)
 
     last_exc = None

tests/test_placement_declines.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""A pod must be able to say why it has nowhere to send a page.
+
+A rented pod answered
+
+    model 'datalab-to/surya-ocr-2' is not served over HTTP by this install
+
+for a model it was serving fine over its own loopback. The sentence blames
+configuration; the cause was a lookup. Nothing could tell the two apart,
+because `_local_candidate` ended in `except Exception: return None` and the
+pod's log is not reachable from the machine that receives its 503.
+
+So: every decline is recorded with a reason, and the ERROR carries it.
+"""
+
+import os
+
+import pytest
+
+from codai.models import manager as mm
+from codai.models import placement
+from codai.ocr.base import OcrError
+
+
+SEED = '[{"backend": "vllm", "path": "datalab-to/surya-ocr-2"}]'
+MID = "datalab-to/surya-ocr-2"
+
+
+@pytest.fixture(autouse=True)
+def _clean():
+    placement._declines.clear()
+    placement.forget()
+    mm._SEED_CACHE = None
+    yield
+    placement._declines.clear()
+    placement.forget()
+    mm._SEED_CACHE = None
+
+
+# ------------------------------------------------- the seeded catalogue #
+def test_a_seeded_model_is_found_without_the_admin_global(monkeypatch):
+    """THE pod bug: the entry lives in CODERAI_SEED_MODELS, not models.json."""
+    monkeypatch.setenv("CODERAI_SEED_MODELS", SEED)
+    mm._SEED_CACHE = None
+    assert mm._model_entry_for(MID) is not None
+    assert mm._pinned_backend(MID) == "vllm"
+
+
+def test_the_seed_is_matched_by_basename_too(monkeypatch):
+    monkeypatch.setenv("CODERAI_SEED_MODELS", SEED)
+    mm._SEED_CACHE = None
+    assert mm._model_entry_for("surya-ocr-2") is not None
+
+
+def test_an_unseeded_name_is_still_unknown(monkeypatch):
+    monkeypatch.setenv("CODERAI_SEED_MODELS", SEED)
+    mm._SEED_CACHE = None
+    assert mm._model_entry_for("someone/else") is None
+
+
+def test_no_seed_in_the_environment_changes_nothing(monkeypatch):
+    monkeypatch.delenv("CODERAI_SEED_MODELS", raising=False)
+    mm._SEED_CACHE = None
+    assert mm._env_seed_entries() == []
+
+
+def test_a_malformed_seed_is_not_an_exception(monkeypatch):
+    """A pod must boot and say so, not crash in a lookup."""
+    monkeypatch.setenv("CODERAI_SEED_MODELS", "{not json")
+    mm._SEED_CACHE = None
+    assert mm._env_seed_entries() == []
+    assert mm._model_entry_for(MID) is None
+
+
+def test_a_real_catalogue_entry_wins_over_the_seed(monkeypatch):
+    """An operator's edited entry must not be shadowed by the launch seed."""
+    monkeypatch.setenv("CODERAI_SEED_MODELS", SEED)
+    mm._SEED_CACHE = None
+
+    class _CM:
+        models_data = {"text_models": [{"path": MID, "backend": "ds4",
+                                        "marker": "real"}]}
+
+    import codai.admin.routes as ar
+    monkeypatch.setattr(ar, "config_manager", _CM, raising=False)
+    assert mm._model_entry_for(MID).get("marker") == "real"
+
+
+# ------------------------------------------------------- the declines #
+def test_a_local_decline_names_the_three_things_that_decide_it(monkeypatch):
+    monkeypatch.setattr(mm, "resolve_engine_backend", lambda _m: None)
+    monkeypatch.setattr(mm, "_model_entry_for", lambda _m: None)
+    monkeypatch.setattr(mm, "_pinned_backend", lambda _m: "")
+    monkeypatch.setattr(mm, "_engine_enabled", lambda _e: False)
+    assert placement._local_candidate(MID) is None
+    why = placement.why_no_candidates(MID)
+    assert "NOT FOUND in this process" in why
+    assert "vllm enabled=False" in why
+
+
+def test_a_raised_exception_is_recorded_not_swallowed(monkeypatch):
+    def _boom(_m):
+        raise RuntimeError("vllm venv is missing")
+
+    monkeypatch.setattr(mm, "resolve_engine_backend", _boom)
+    assert placement._local_candidate(MID) is None
+    assert "vllm venv is missing" in placement.why_no_candidates(MID)
+
+
+def test_the_decline_is_said_once_per_reason(monkeypatch, capsys):
+    monkeypatch.setattr(mm, "resolve_engine_backend", lambda _m: None)
+    monkeypatch.setattr(mm, "_model_entry_for", lambda _m: None)
+    monkeypatch.setattr(mm, "_pinned_backend", lambda _m: "")
+    monkeypatch.setattr(mm, "_engine_enabled", lambda _e: False)
+    for _ in range(20):
+        placement._local_candidate(MID)
+    assert capsys.readouterr().out.count("no local candidate") == 1
+
+
+def test_a_changed_reason_is_said_again(monkeypatch, capsys):
+    monkeypatch.setattr(mm, "resolve_engine_backend", lambda _m: None)
+    monkeypatch.setattr(mm, "_model_entry_for", lambda _m: None)
+    monkeypatch.setattr(mm, "_pinned_backend", lambda _m: "")
+    monkeypatch.setattr(mm, "_engine_enabled", lambda _e: False)
+    placement._local_candidate(MID)
+    monkeypatch.setattr(mm, "_engine_enabled", lambda _e: True)
+    placement._local_candidate(MID)
+    assert capsys.readouterr().out.count("no local candidate") == 2
+
+
+def test_the_error_carries_the_reasons(monkeypatch):
+    """The whole point: the 503 travels to a machine the log does not."""
+    monkeypatch.setattr(placement, "candidates", lambda _m: [])
+    placement._decline(MID, "local", "catalogue entry NOT FOUND in this process")
+    with pytest.raises(OcrError) as e:
+        placement.choose(MID)
+    assert "[why: " in str(e.value)
+    assert "NOT FOUND in this process" in str(e.value)
+
+
+def test_the_error_is_unchanged_when_nothing_was_recorded(monkeypatch):
+    monkeypatch.setattr(placement, "candidates", lambda _m: [])
+    with pytest.raises(OcrError) as e:
+        placement.choose(MID)
+    assert "[why: " not in str(e.value)
+
+
+def test_a_working_local_candidate_clears_an_old_decline(monkeypatch):
+    """A fixed install must stop reporting the fault it no longer has."""
+    placement._decline(MID, "local", "something that was true once")
+    monkeypatch.setattr(mm, "resolve_engine_backend", lambda _m: "vllm")
+    from codai.api import runpod_worker, vllm_worker
+    monkeypatch.setattr(mm, "get_active_vllm_config", lambda: object(),
+                        raising=False)
+    monkeypatch.setattr(runpod_worker, "_model_path_for", lambda _m: MID)
+    monkeypatch.setattr(vllm_worker, "ensure_service",
+                        lambda *_a, **_k: "http://127.0.0.1:8000")
+    got = placement._local_candidate(MID)
+    assert got is not None and got.kind == "local"
+    assert placement.why_no_candidates(MID) == ""