let the two vLLM images evolve apart, visibly

The fleet image and the OCR image were coupled in three places, not one: the
profile-parity test, and -- less visibly -- the OCR image's own build check,
which read the FLEET's vLLM pin and failed the build on a mismatch. That one
meant the OCR image could not be built on a different vLLM at all.

They may diverge now. What replaces the coupling is the thing that actually
failed last time: not that the images differed, but that nothing said so, and
someone had to already suspect it. So each profile must PIN vllm and the torch
stack with `==` (no floating version, so two builds of one image cannot differ
either), each build check holds its own venv to its OWN pin -- which still
catches the resolver substituting a version nobody asked for, a failure the
cross-image check could never see -- and a choice the engine would otherwise
make by version is named by the image that depends on it.

That needed a mechanism, so profiles gain `<profile>.env`: KEY=VALUE baked into
the image and exported by boot.sh, which PRINTS each variable it applies and
lets the pod's own environment win. The OCR profile uses it to pin
VLLM_ATTENTION_BACKEND=FLASH_ATTN -- vLLM's own default on Ampere, named rather
than inherited, because FlashInfer is the one backend that image cannot use
(its kernels compile on first use and it ships no CUDA toolkit) and which
backend gets selected is a property of the vLLM VERSION. A future bump can now
change throughput, never whether the pod starts.

Also takes one install's model names out of three shipped files. The
measurement they illustrate is what matters -- a registration overwritten, a
backend fallen through to "cpu" -- not whose models they were.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66

codai/api/remote_gateway.py

@@ -1339,7 +1339,7 @@ def teach_model(base: str, api_key: str, model: str) -> bool:
     # means nothing on the pod, so it is stripped and the pod falls through to
     # its own default. On a pod with no local GPU backend that default is "cpu":
     #
-    #   503 Model 'digesta-alpha' failed to load: Unknown backend: cpu
+    #   503 Model 'lora-alpha' failed to load: Unknown backend: cpu
     #
     # The SEED path has passed the engine since the same bug bit diogene-alpha.
     # Teaching did not, so a model seeded CORRECTLY at boot was then

codai/api/runpod_worker.py

@@ -6619,7 +6619,7 @@ def set_client_pods(model: str, pods: int, teardown: bool = False,
     if not key:
         raise LookupError("no model named")
     # A LIVE POOL, addressed by the key this very function reports back as
-    # `pool`. Several models share one pool, so `pool:digesta-qwen38` is not a
+    # `pool`. Several models share one pool, so `pool:shared-qwen` is not a
     # model and has no runpod block of its own -- and until now the only answer
     # to it was "has no runpod block", from code that already knew the caller
     # could not look the key up. A pool that exists can be addressed; its
@@ -6655,7 +6655,7 @@ def set_client_pods(model: str, pods: int, teardown: bool = False,
         # After the floor is lowered, so the shed measures against the new one.
         out["teardown"] = pool.shed_now()
     # Answer with the name the client asked about, not the internal pool key:
-    # several models may share one pool, and "pool:digesta-qwen38" is not
+    # several models may share one pool, and "pool:shared-qwen" is not
     # something the caller sent or can look up.
     out["pool"] = out["model"]
     out["model"] = key

packaging/runpod/Dockerfile.capability

@@ -129,6 +129,26 @@ EXPOSE 8000
 HEALTHCHECK --interval=30s --timeout=5s --start-period=120s \
     CMD curl -fsS http://127.0.0.1:8000/healthz || curl -fsSk https://127.0.0.1:8000/healthz || exit 1
 
+# Environment the IMAGE fixes, from `<profile>.env` (KEY=VALUE per line).
+#
+# For settings that belong to the image rather than to a model: a choice that
+# must not change under the engine's own version-dependent auto-selection. The
+# OCR image pins its attention backend this way, because FlashInfer is the one
+# backend it cannot use -- its kernels are compiled on first use and no image
+# here ships a CUDA toolkit -- and which backend vLLM picks on a given card is
+# a property of the vLLM version, not of anything configured here.
+#
+# Written to a file boot.sh sources rather than baked as ENV, so one layer
+# serves any profile and an operator can read what the image decided.
+RUN set -eux; \
+    if [ -f "/tmp/profiles/${PROFILE}.env" ]; then \
+        grep -vE '^[[:space:]]*(#|$)' "/tmp/profiles/${PROFILE}.env" \
+            > /opt/coderai/image.env; \
+        echo "image env:"; cat /opt/coderai/image.env; \
+    else \
+        : > /opt/coderai/image.env; \
+    fi
+
 COPY packaging/runpod/boot.sh /opt/coderai/boot.sh
 RUN chmod +x /opt/coderai/boot.sh
 

packaging/runpod/boot.sh

@@ -34,6 +34,23 @@ phase() {
 
 phase "container started"
 phase "profile=${PROFILE:-unknown} image=${CODERAI_IMAGE_TAG:-unknown}"
+
+# What the IMAGE fixes about its own engine (profiles/<profile>.env, baked by
+# Dockerfile.capability). An operator's settings are not here -- these are the
+# choices an image cannot leave to auto-selection, and they are PRINTED so a pod
+# can be asked what it decided instead of having it inferred. A variable already
+# set in the pod's environment wins, so an operator can still override one.
+if [ -s /opt/coderai/image.env ]; then
+    while IFS='=' read -r _k _v; do
+        [ -n "$_k" ] || continue
+        if [ -z "$(eval "echo \${$_k:-}")" ]; then
+            export "$_k=$_v"
+            phase "image env: $_k=$_v"
+        else
+            phase "image env: $_k kept from the pod's environment"
+        fi
+    done < /opt/coderai/image.env
+fi
 phase "user=$(id -un 2>/dev/null || echo unknown) cwd=$(pwd 2>/dev/null || echo unknown)"
 phase "python=$(python -V 2>&1)"
 

packaging/runpod/profiles/vllm-ocr.env

+# Environment this IMAGE fixes. See Dockerfile.capability (`<profile>.env`).
+#
+# FlashInfer is the one attention backend this image cannot use: its kernels are
+# compiled on first use and no image here ships a CUDA toolkit, so an engine that
+# selects it dies during init with
+#   flashinfer/jit/cpp_ext.py: Could not find nvcc and default
+#   cuda_home='/usr/local/cuda' doesn't exist
+# (measured on a pod 2026-10-22, on the fleet image, when an FP8 KV cache was
+# asked for). The fleet image now carries flashinfer-jit-cache so it CAN; this
+# one deliberately does not, because the OCR models never ask for that dtype and
+# the package costs ~2.9 GiB.
+#
+# Which backend vLLM picks for a given card is a property of the vLLM VERSION,
+# not of anything configured here -- so the next vLLM bump could start preferring
+# FlashInfer on Ampere and this image would stop booting, for a reason nothing in
+# its own configuration would explain. Pinning it means a bump can change
+# throughput, never whether the pod starts.
+#
+# FLASH_ATTN is vLLM's own default on Ampere for these models; it is named rather
+# than inherited.
+VLLM_ATTENTION_BACKEND=FLASH_ATTN

packaging/runpod/profiles/vllm-ocr.venv-vllm.check

 # Deep check for the vllm venv: the OpenAI server module must import, and the
-# version must MATCH the plain `vllm` profile rather than being pinned away from
-# it. Two images serving the same weights on two different vLLMs is an invisible
-# difference, and this image used to be exactly that.
+# installed vLLM must be the one THIS profile asked for.
+#
+# It used to require a match with the plain `vllm` profile, because this image
+# had once been pinned away from the fleet and two images serving the same
+# weights on two different vLLMs cost three rounds of misdiagnosis. The two
+# images are allowed to evolve apart now; what must never happen is a version
+# nobody chose. So the pin is read from this profile's OWN requirements and the
+# venv is held to it -- which also catches the resolver quietly substituting a
+# different build, the failure the cross-image check could never see.
 import pathlib
 import vllm.entrypoints.openai.api_server                 # noqa
 import torch, vllm
 print("vllm", vllm.__version__, "torch", torch.__version__, "cuda", torch.version.cuda)
 
-# The pin lives in profiles/vllm.venv-vllm.txt; read it rather than restate it,
-# so bumping the fleet cannot silently leave this image behind.
+# Read this profile's own pin rather than restating it here, so the requirement
+# and the check cannot disagree.
 _want = ""
-for _p in ("/tmp/venv-profiles/vllm.venv-vllm.txt",
-           "packaging/runpod/profiles/vllm.venv-vllm.txt"):
+for _p in ("/tmp/venv-profiles/vllm-ocr.venv-vllm.txt",
+           "packaging/runpod/profiles/vllm-ocr.venv-vllm.txt"):
     _f = pathlib.Path(_p)
     if _f.is_file():
         for _line in _f.read_text().splitlines():
@@ -20,11 +26,11 @@ for _p in ("/tmp/venv-profiles/vllm.venv-vllm.txt",
         break
 if _want:
     assert vllm.__version__.startswith(_want), (
-        "this image must serve the same vLLM as profiles/vllm (%s); got %s"
+        "this venv must serve the vLLM this profile pins (%s); got %s"
         % (_want, vllm.__version__))
-    print("vllm matches the fleet profile:", _want)
+    print("vllm matches this profile's pin:", _want)
 else:
-    print("could not read the fleet's vllm pin; version match not enforced here")
+    print("could not read this profile's vllm pin; version match not enforced here")
 
 # Surya-2 is Qwen3.5-VL, so the architecture has to be in this vLLM's registry or
 # the pod cannot serve the model at all. Reported-not-enforced where the registry

tests/test_ocr_accent_repair.py

@@ -256,65 +256,90 @@ def test_it_defaults_on():
 
 
 # ------------------------------------------- option 2: the pinned-vLLM image #
-def test_the_vllm_ocr_profile_runs_the_SAME_vllm_as_the_fleet():
-    """This image was built to pin vLLM 0.20.1 — the version surya spawns
-    (surya/settings.py:96) — because a page of crisp synthetic Italian came back
-    from 0.27.1 with one U+FFFD per accented letter.
-
-    The pin was the wrong conclusion from a real measurement: surya-ocr-2's
-    tokenizer is BYTE-level, so a U+FFFD per accent does not show the model
-    choosing it, and the Digesta fleet has since been reading accented Italian on
-    0.27.1 (`coderai-vllm`). Two images serving the same weights on two vLLMs is
-    an invisible difference, and that difference cost three rounds of blaming the
-    pod image. So the pins must be IDENTICAL, and 0.3.69 repurposed this image
-    for what actually distinguishes it: the OCR pipeline on the pod."""
+def test_each_vllm_image_says_which_engine_it_runs():
+    """The two vLLM images are free to diverge; neither may do it invisibly.
+
+    This started as "the pins must be IDENTICAL". That rule came from a real
+    incident: the OCR image pinned vLLM 0.20.1 -- the version surya spawns --
+    because a page of crisp synthetic Italian came back from 0.27.1 with one
+    U+FFFD per accented letter. The pin was the wrong conclusion from a real
+    measurement (surya-ocr-2's tokenizer is BYTE-level, so a U+FFFD per accent
+    does not show the model choosing it), and two images serving the same
+    weights on two engines cost three rounds of blaming the pod image.
+
+    What actually did the damage was not that the images differed. It was that
+    nothing said so: the difference had to be DISCOVERED, by someone who already
+    suspected it. So the images may now evolve apart -- the fleet carries
+    FlashInfer's precompiled kernels for an FP8 KV cache and this one does not,
+    because its models never ask for that dtype and the package costs ~2.9 GiB
+    -- and what is enforced instead is that each image states, in its own
+    profile, every version that decides what its engine does:
+
+      * the engine and the torch stack under it are PINNED, never floating, so
+        two builds of one profile cannot quietly differ either;
+      * a choice the engine would otherwise make by version -- which attention
+        backend -- is named by the image that depends on it;
+      * and a pod PRINTS what it was given at boot, so "which engine is this"
+        is answered from its own log instead of inferred.
+    """
     import pathlib
     prof = pathlib.Path(__file__).resolve().parents[1] / "packaging/runpod/profiles"
 
     def _lines(name):
         return [l.strip() for l in (prof / name).read_text().splitlines()
                 if l.strip() and not l.strip().startswith("#")]
-    fleet, ours = _lines("vllm.venv-vllm.txt"), _lines("vllm-ocr.venv-vllm.txt")
-
-    # The invariant is the ENGINE, not the file: every pin that decides which
-    # vLLM (and which torch under it) runs must be identical, because two images
-    # serving the same weights on two engines is the invisible difference this
-    # test exists to prevent.
-    #
-    # One deliberate exception, weighed and accepted 2026-10-22: the fleet image
-    # also carries flashinfer-jit-cache, FlashInfer's precompiled kernels, which
-    # the fleet needs for an FP8 KV cache (without them the engine looks for nvcc
-    # at run time and dies). The OCR models never ask for that dtype, and the
-    # package costs ~2.9 GiB in an image that is already large, so it is not
-    # installed here. Anything else diverging is the old bug coming back.
-    ALLOWED_EXTRA = {"flashinfer-jit-cache", "--extra-index-url https://flashinfer.ai/whl/cu129"}
-
-    def _name(line):
-        return line.split("==")[0].split(">=")[0].strip()
-    extra = [l for l in fleet if l not in ours]
-    assert all(_name(l) in ALLOWED_EXTRA or l in ALLOWED_EXTRA for l in extra), (
-        f"the OCR image drifted from the fleet's vLLM again: {extra}")
-    assert [l for l in ours if l not in fleet] == [], (
-        "the OCR image pins something the fleet does not")
-    for pin in ("vllm==", "torch==", "torchvision==", "torchaudio==", "torchcodec=="):
-        assert [l for l in fleet if l.startswith(pin)] == \
-               [l for l in ours if l.startswith(pin)], f"{pin} differs"
-    # uv, because pip cannot resolve this pin set.
-    assert (prof / "vllm-ocr.venv-vllm.uv").exists()
-    # The light core, like the profile it is derived from.
-    assert (prof / "vllm-ocr.light").exists()
-    # Triton needs a C compiler for its launcher stubs.
-    assert "gcc" in (prof / "vllm-ocr.apt").read_text()
-
-
-def test_the_build_check_refuses_a_silent_drift_from_the_fleet():
-    """The check no longer asserts a hardcoded version — it READS the fleet's pin
-    from disk and requires a match, so bumping profiles/vllm cannot silently
-    leave the OCR image behind on an older vLLM."""
+
+    for profile in ("vllm", "vllm-ocr"):
+        pins = _lines(f"{profile}.venv-vllm.txt")
+        for package in ("vllm", "torch", "torchvision", "torchaudio", "torchcodec"):
+            exact = [l for l in pins if l.split("==")[0].strip() == package]
+            assert exact, f"{profile}: {package} is not pinned at all"
+            assert "==" in exact[0], (
+                f"{profile}: {package} floats ({exact[0]!r}) -- two builds of "
+                f"this one image could then run different engines")
+        # uv, because pip cannot resolve this pin set.
+        assert (prof / f"{profile}.venv-vllm.uv").exists()
+        # The light core, and a C compiler for Triton's launcher stubs.
+        assert (prof / f"{profile}.light").exists()
+        assert "gcc" in (prof / f"{profile}.apt").read_text()
+
+    # The OCR image cannot use FlashInfer: its kernels compile on first use and
+    # no image here ships a CUDA toolkit. Which backend vLLM selects is a
+    # property of its VERSION, so leaving it to auto-selection means a future
+    # bump could stop this image booting for a reason nothing in its own
+    # configuration explains. It is named here instead.
+    env = (prof / "vllm-ocr.env").read_text()
+    assert "VLLM_ATTENTION_BACKEND=" in env
+    assert "FLASHINFER" not in env.split("VLLM_ATTENTION_BACKEND=")[1].split("\n")[0].upper()
+
+
+def test_the_image_env_reaches_the_engine():
+    """A `<profile>.env` is only a pin if the pod actually applies it: the image
+    writes it at build time and boot.sh exports it, announcing each one. An
+    operator's own value wins, so a pod can still be told otherwise."""
+    import pathlib
+    root = pathlib.Path(__file__).resolve().parents[1]
+    dockerfile = (root / "packaging/runpod/Dockerfile.capability").read_text()
+    assert "${PROFILE}.env" in dockerfile
+    assert "/opt/coderai/image.env" in dockerfile
+    boot = (root / "packaging/runpod/boot.sh").read_text()
+    assert "/opt/coderai/image.env" in boot
+    assert "image env:" in boot            # printed, not silently applied
+    assert "kept from the pod's environment" in boot
+
+
+def test_each_build_check_holds_its_venv_to_its_own_pin():
+    """The check reads the pin from the profile it belongs to and requires the
+    installed vLLM to match.
+
+    It used to read the FLEET's pin, which coupled the two images: bumping one
+    would fail the other's build. They are allowed to diverge now. What the
+    check still catches -- and what the cross-image version could not see -- is
+    the resolver quietly installing a vLLM this profile never asked for."""
     import pathlib
     prof = pathlib.Path(__file__).resolve().parents[1] / "packaging/runpod/profiles"
     check = (prof / "vllm-ocr.venv-vllm.check").read_text()
-    assert "vllm.venv-vllm.txt" in check, "the check does not read the fleet pin"
+    assert "vllm-ocr.venv-vllm.txt" in check, "the check reads another profile's pin"
     assert "vllm.__version__.startswith(_want)" in check
     # ...but it must not fail the build on the NAME of a private vLLM symbol.
     assert "except ImportError" in check

tests/test_teach_carries_the_engine.py

 """A taught model must reach the pod with the engine that can serve it.
 
-2026-10-22, digesta-alpha on a fresh pod:
+2026-10-22, lora-alpha on a fresh pod:
 
   [boot +0.8s] seeded models registered: qwen38-awq, diogene-alpha,
-               digesta-alpha, qwen38-vision          <- correct
+               lora-alpha, qwen-vision          <- correct
   ...
-  Resolving model: digesta-alpha
-  [register] digesta-alpha registered in text_models <- teach, overwriting it
-  [lora] added 'digesta-alpha' to the running engine — no reload
-  Loading model on demand: digesta-alpha
+  Resolving model: lora-alpha
+  [register] lora-alpha registered in text_models <- teach, overwriting it
+  [lora] added 'lora-alpha' to the running engine — no reload
+  Loading model on demand: lora-alpha
   Warning: No GPU backend detected.
-  Error loading model digesta-alpha: Unknown backend: cpu
+  Error loading model lora-alpha: Unknown backend: cpu
 
 seed_model_env strips `backend` unless it is told the engine: a RunPod model
 carries `backend: "runpod"`, which is THIS install's placement and means
@@ -29,12 +29,12 @@ from codai.api.runpod_worker import engine_pod_backend, parse_model_runpod
 
 
 ENTRY = {
-    "id": "digesta-alpha",
+    "id": "lora-alpha",
     "backend": "runpod",
-    "loras": [{"name": "digesta-alpha", "path": "sha256:ab", "weight": 1.0}],
+    "loras": [{"name": "lora-alpha", "path": "sha256:ab", "weight": 1.0}],
     "runpod": {"mode": "pods", "engine": "coderai-vllm",
                "hf_repo": "cyankiwi/Qwen3.8-27B-AWQ-INT4",
-               "served_model": "digesta-alpha"},
+               "served_model": "lora-alpha"},
 }
 
 
@@ -58,7 +58,7 @@ def taught(monkeypatch):
         return _R()
     monkeypatch.setattr("codai.api.pod_http.post", _post)
     monkeypatch.setattr("codai.models.manager._model_entry_for",
-                        lambda name: ENTRY if name == "digesta-alpha" else {})
+                        lambda name: ENTRY if name == "lora-alpha" else {})
     rg._REGISTERED.clear()
     return sent
 
@@ -66,17 +66,17 @@ def taught(monkeypatch):
 def test_a_taught_model_carries_its_engine(taught):
     """THE BUG: this arrived with no `backend` at all, and the pod -- which has
     no local GPU backend -- fell through to "cpu"."""
-    assert rg.teach_model("https://pod:1", "k", "digesta-alpha") is True
+    assert rg.teach_model("https://pod:1", "k", "lora-alpha") is True
     rows = taught["body"]
     assert rows, "nothing was registered"
-    row = next(r for r in rows if (r.get("id") or r.get("alias")) == "digesta-alpha")
+    row = next(r for r in rows if (r.get("id") or r.get("alias")) == "lora-alpha")
     assert row["backend"] == "vllm"
 
 
 def test_the_adapter_still_travels(taught):
-    rg.teach_model("https://pod:1", "k", "digesta-alpha")
+    rg.teach_model("https://pod:1", "k", "lora-alpha")
     row = next(r for r in taught["body"]
-               if (r.get("id") or r.get("alias")) == "digesta-alpha")
+               if (r.get("id") or r.get("alias")) == "lora-alpha")
     assert row["loras"], "the LoRA must still be registered with it"
     assert row["path"] == "cyankiwi/Qwen3.8-27B-AWQ-INT4"
 
@@ -95,5 +95,5 @@ def test_a_model_with_no_pod_engine_is_unchanged(taught, monkeypatch):
 
 
 def test_it_posts_to_the_register_endpoint(taught):
-    rg.teach_model("https://pod:1", "k", "digesta-alpha")
+    rg.teach_model("https://pod:1", "k", "lora-alpha")
     assert taught["url"].endswith("/v1/models/register")