ocr: the accents were never lost by the model, so take them back from its bytes (v0.3.28)

MEASURED VERDICT. The byte trace on a page of crisp synthetic Italian, served
through the Aruba install on vLLM 0.27.1:

    unreadable: count=29  blame='the serving path'
    page 0: calls=1  with_logprobs=1  conclusive=True  kinds=None
       why: the byte stream decoded cleanly, so the characters were lost AFTER
            the model: the fault is in the serving path, not the model

surya-ocr-2's own byte stream was valid UTF-8 and every line came back at
conf=0.9605. The model read the page correctly; something after it destroyed 29
accented characters.

What that does NOT implicate: surya-ocr-2 and chandra-ocr-2 are the SAME
architecture (Qwen3_5ForConditionalGeneration) on the same vLLM, and chandra's
accents survive. So this is not vLLM mishandling qwen3_5 -- it is vLLM
mishandling SURYA'S CUSTOM 65,425-entry BYTE-LEVEL vocabulary, in which one
accented letter is two tokens and either byte alone decodes to a marker. It was
also a single model call, which kills the concurrent-batching hypothesis I had
been carrying.

OPTION 1, shipped and on by default: ocr.surya_repair_accents rebuilds the
reply from the per-token bytes in the same response. Applied in the backend
wrapper BEFORE surya parses the HTML, so blocks, boxes, text and confidences are
all built from correct characters; repairing further down would mean fixing the
same text in several places and missing one.

Rewriting a model's output is serious, so it refuses unless all three hold:
strict UTF-8 decode, no marker in the result, and an ASCII skeleton identical to
the original's. The third is what makes it a repair and not a substitution -- it
can only ever change non-ASCII characters, so a truncation, a different block or
a shifted offset all leave the original alone, markers and all. Clean text is
untouched, not even reassembled, and a repaired page reports
`accents_repaired: {blocks, recovered, how}` because the text is no longer
byte-for-byte what the server sent.

Default ON because the alternative is shipping text measured to be corrupt. Set
false to see exactly what the server sent.

OPTION 2, prepared: a `coderai-vllm-ocr` pod image pinned to vLLM 0.20.1 -- the
version surya-ocr itself spawns (surya/settings.py:96) -- so the model can be
served on the stack its authors validate, the repair can be checked against a
reference rather than trusted, and the upstream bug report has a known-good
comparison. Selected per model with `runpod.engine: "coderai-vllm-ocr"`, built
with `build_capability_image.sh vllm-ocr`.

The pin set was verified to exist before the profile was written, because a
profile that cannot build is worse than none: wheels.vllm.ai/0.20.1/cu129 is
live, and torch 2.11.0 / torchvision 0.26.0 / torchaudio 2.11.0 (vLLM 0.20.1's
own metadata pins) all have +cu129 cp313 wheels. Qwen3_5ForConditionalGeneration
is in 0.20.1's registry, so it can serve the model. No torchcodec pin: 0.20.1
does not require one, and pinning a package nothing needs is its own hazard --
the file says what to add if a build ever surprises us. NOT a drop-in for
`coderai-vllm`: everything else on it would be pinned back seven minor releases.

tests: test_ocr_accent_repair.py, 29 tests -- the repair itself, every guard, and
the image profile. 833 passed across the ocr/runpod/config/admin/front selection
with the same 8 pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.27"
+__version__ = "0.3.28"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/routes.py

@@ -4140,6 +4140,7 @@ def build_settings_dict(c, gpu_cards):
             "surya_instances": c.ocr.surya_instances,
             "surya_langs": c.ocr.surya_langs,
             "surya_token_trace": c.ocr.surya_token_trace,
+            "surya_repair_accents": c.ocr.surya_repair_accents,
             "surya_venv": c.ocr.surya_venv,
             "surya_auto_build": c.ocr.surya_auto_build,
             "surya_serve": c.ocr.surya_serve,
@@ -4958,6 +4959,7 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
         if "surya_accept_license" in d: o.surya_accept_license = bool(d["surya_accept_license"])
         if "surya_instances" in d: o.surya_instances = _i("surya_instances", o.surya_instances, 1)
         if "surya_token_trace" in d: o.surya_token_trace = bool(d["surya_token_trace"])
+        if "surya_repair_accents" in d: o.surya_repair_accents = bool(d["surya_repair_accents"])
         if "surya_langs" in d: o.surya_langs = (d.get("surya_langs") or "it").strip()
         if "surya_venv" in d: o.surya_venv = (d.get("surya_venv") or "").strip()
         if "surya_auto_build" in d: o.surya_auto_build = bool(d["surya_auto_build"])

codai/admin/templates/settings.html

@@ -1207,6 +1207,7 @@
       <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer" title="Local mode only. On, Surya runs on this machine's GPU or refuses to load — it will not quietly fall back to the CPU, where a page takes minutes. Off, the CPU is used deliberately."><input type="checkbox" id="s-ocr-surya-use-gpu" checked><span>Require GPU (Local)</span></label>
       <div class="form-group" style="flex:1;min-width:120px"><label class="form-label">Langs</label><input type="text" id="s-ocr-surya-langs" class="form-input" placeholder="it"></div>
       <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer" title="Return the model's own per-token bytes wherever a page came back unreadable. A diagnostic — leave off unless you are chasing lost characters."><input type="checkbox" id="s-ocr-surya-token-trace"><span>Trace unreadable tokens</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer" title="Rebuild the text from the model's own per-token bytes when the reply we are handed has lost characters. Measured fault: surya-ocr-2 emits valid UTF-8 and message.content comes back with one U+FFFD per accented letter."><input type="checkbox" id="s-ocr-surya-repair-accents"><span>Repair lost characters from bytes</span></label>
     </div>
     <p class="form-hint" style="margin:0 0 .3rem">Runs in an <b>isolated venv</b> (Surya caps pillow&lt;11, incompatible with the main venv).</p>
     <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
@@ -2204,6 +2205,7 @@ async function loadSettings(){
     document.getElementById('s-ocr-surya-use-gpu').checked = ocr.surya_use_gpu !== false;
     document.getElementById('s-ocr-surya-langs').value = ocr.surya_langs ?? 'it';
     document.getElementById('s-ocr-surya-token-trace').checked = !!ocr.surya_token_trace;
+    document.getElementById('s-ocr-surya-repair-accents').checked = ocr.surya_repair_accents !== false;
     document.getElementById('s-ocr-surya-venv').value = ocr.surya_venv ?? '';
     document.getElementById('s-ocr-surya-auto-build').checked = !!ocr.surya_auto_build;
     document.getElementById('s-ocr-surya-serve').value = ocr.surya_serve || 'model';
@@ -2487,6 +2489,7 @@ async function saveSettings(silent){
       surya_instances: parseInt(document.getElementById('s-ocr-surya-instances').value) || 1,
       surya_langs: document.getElementById('s-ocr-surya-langs').value.trim() || 'it',
       surya_token_trace: document.getElementById('s-ocr-surya-token-trace').checked,
+      surya_repair_accents: document.getElementById('s-ocr-surya-repair-accents').checked,
       surya_venv: document.getElementById('s-ocr-surya-venv').value.trim(),
       surya_auto_build: document.getElementById('s-ocr-surya-auto-build').checked,
       surya_serve: document.getElementById('s-ocr-surya-serve').value,

codai/api/runpod_worker.py

@@ -324,6 +324,12 @@ PUBLISHED_CAPABILITY_IMAGES = (
     # core, driven by coderai's vLLM proxy backend.
     "llama",
     "vllm",
+    # The same vLLM pod pinned to 0.20.1 — the version `surya-ocr` spawns and
+    # validates against — because serving surya-ocr-2 on 0.27.1 loses every
+    # accented character while the model's own bytes decode cleanly. Rent it for
+    # the OCR model only; everything else on it is pinned back seven minor
+    # releases. See packaging/runpod/profiles/vllm-ocr.txt for the measurement.
+    "vllm-ocr",
 )
 
 #: Model backends served by an engine image, and which image. ds4, colibri
@@ -334,7 +340,15 @@ _ENGINE_POD_BACKENDS = ("ds4", "colibri", "k3", "kt", "vllm")
 _ENGINE_POD_IMAGE = {"kt": "engines-kt", "vllm": "vllm"}
 #: runpod-block `engine` values that mean "a coderai pod running THIS server",
 #: as opposed to `vllm`/`llamacpp`, which mean the upstream images.
-_CODERAI_SERVER_ENGINES = {"coderai-vllm": "vllm", "coderai-llama": "llama"}
+#:
+#: `coderai-vllm-ocr` is the same server pinned to vLLM **0.20.1**, the version
+#: `surya-ocr` itself spawns and validates against (surya/settings.py:96). It
+#: exists because serving surya-ocr-2 on 0.27.1 loses every accented character
+#: while the model's own bytes decode cleanly — see
+#: packaging/runpod/profiles/vllm-ocr.txt for the measurement. Rent it for the
+#: OCR model only: everything else on that image would be pinned back with it.
+_CODERAI_SERVER_ENGINES = {"coderai-vllm": "vllm", "coderai-llama": "llama",
+                           "coderai-vllm-ocr": "vllm-ocr"}
 
 #: Capabilities served by a published image other than their own name.
 _CAPABILITY_IMAGE_ALIASES = {
@@ -555,7 +569,9 @@ def engine_pod_backend(entry: dict, mcfg: "RunpodModelConfig" = None) -> str:
     want = str(getattr(mcfg, "engine", "") or "").strip().lower() if mcfg else ""
     if want in _ENGINE_POD_BACKENDS:
         return want
-    if want == "coderai-vllm":
+    if want in ("coderai-vllm", "coderai-vllm-ocr"):
+        # Both are coderai's vLLM backend; they differ only in which vLLM the
+        # image carries, which is the image's business and not the engine's.
         return "vllm"
     b = str((entry or {}).get("backend") or "").strip().lower()
     return b if b in _ENGINE_POD_BACKENDS else ""

codai/config.py

@@ -1007,6 +1007,18 @@ class OcrConfig:
     # surya-ocr-2's vocabulary is byte-level and decoding either byte of an
     # accented letter alone yields exactly one replacement character.
     surya_token_trace: bool = False
+    # Rebuild a surya reply's text from the model's OWN per-token bytes when the
+    # text we are handed has lost characters. On by default, because the
+    # alternative is shipping text measured to be corrupt: on 2026-10-12 a page of
+    # crisp synthetic Italian came back from surya-ocr-2 with one U+FFFD per
+    # accented letter while the model's byte stream decoded cleanly and it
+    # reported ~96% confidence, and chandra read the identical pixels perfectly.
+    # The correct bytes arrive in the same response's logprobs, so the reply is
+    # simply rebuilt from them. Guarded three ways (strict decode, no marker in
+    # the result, and an identical ASCII skeleton) so it can only ever change
+    # non-ASCII characters, and it does nothing at all when the text is clean.
+    # Set false to see exactly what the server sent.
+    surya_repair_accents: bool = True
     surya_venv: str = ""               # isolated venv dir; blank = ~/.coderai/surya_venv
     surya_auto_build: bool = False     # create the venv + pip install requirements-surya.txt on first use
     # Surya serving mode: "local" = classic det+recognition on torch in the isolated venv
@@ -1723,6 +1735,7 @@ class ConfigManager:
                 "surya_instances": self.config.ocr.surya_instances,
                 "surya_langs": self.config.ocr.surya_langs,
                 "surya_token_trace": self.config.ocr.surya_token_trace,
+                "surya_repair_accents": self.config.ocr.surya_repair_accents,
                 "surya_venv": self.config.ocr.surya_venv,
                 "surya_auto_build": self.config.ocr.surya_auto_build,
                 "surya_use_gpu": self.config.ocr.surya_use_gpu,

codai/ocr/manager.py

@@ -542,6 +542,7 @@ class OcrManager:
             # The token trace patches surya's backend at load, so toggling it has
             # to rebuild the pool rather than quietly taking effect never.
             getattr(cfg, "surya_token_trace", False),
+            getattr(cfg, "surya_repair_accents", True),
             getattr(cfg, "olmocr_enabled", False), getattr(cfg, "olmocr_instances", 1),
             getattr(cfg, "olmocr_serve", "model"), getattr(cfg, "olmocr_model", ""),
             getattr(cfg, "olmocr_model_id", ""), getattr(cfg, "olmocr_server_url", ""),

codai/ocr/surya.py

@@ -70,7 +70,8 @@ class SuryaEngine(SubprocessOcrEngine):
                 # Sent at LOAD, like everything else here: the hook it switches on
                 # patches surya's backend, which has to happen before the first page
                 # rather than per request.
-                "token_trace": bool(getattr(self.cfg, "surya_token_trace", False))}
+                "token_trace": bool(getattr(self.cfg, "surya_token_trace", False)),
+                "repair_accents": bool(getattr(self.cfg, "surya_repair_accents", True))}
 
     #: Modes where the recognition VLM lives in a model server we only talk to.
     SERVED_MODES = ("model", "vllm", "llamacpp", "server")

codai/ocr/workers/ocr_worker.py

@@ -307,6 +307,71 @@ def _tok_entry(entry) -> dict:
     return out
 
 
+def _ascii_skeleton(text: str) -> str:
+    """``text`` with every non-ASCII character dropped.
+
+    The guard that makes the repair provably safe. If two strings have the same
+    ASCII skeleton they are the same text differing ONLY in their non-ASCII
+    characters -- which is exactly the difference between a reply whose accents
+    were destroyed and the same reply with them intact. Any other difference
+    (a truncation, a different block, a shifted offset) changes the skeleton and
+    the repair is refused.
+    """
+    return "".join(c for c in (text or "") if ord(c) < 128)
+
+
+def _reassemble(tokens) -> str:
+    """The model's own bytes, decoded strictly, or None if they are not clean.
+
+    ``None`` on anything doubtful: missing bytes, a stream that does not decode,
+    or a decode that still contains the marker. The repair must never make a
+    reply worse, so every uncertain case leaves the original alone.
+    """
+    if not tokens:
+        return None
+    buf = bytearray()
+    for t in tokens:
+        e = _tok_entry(t)
+        raw = e.get("bytes")
+        if not isinstance(raw, list):
+            return None                 # a server that reports no bytes: no repair
+        for b in raw:
+            buf.append(int(b) & 0xFF)
+    if not buf:
+        return None
+    try:
+        text = bytes(buf).decode("utf-8")        # strict: errors are a refusal
+    except UnicodeDecodeError:
+        return None
+    return None if UNREADABLE in text else text
+
+
+def repair_content(raw: str, tokens):
+    """``(repaired_text, characters_recovered)`` for a reply whose text lost
+    characters, or ``(None, 0)`` to leave it alone.
+
+    The measured fault this exists for: surya-ocr-2 emits perfectly good UTF-8 --
+    the byte trace confirms the stream decodes cleanly and the model reports ~96%
+    confidence -- and the text we read back from ``message.content`` has one
+    U+FFFD per accented letter. The correct bytes are right there in the same
+    response's logprobs, so the reply can simply be rebuilt from them.
+
+    Guarded three ways, because silently rewriting a model's output is a serious
+    thing to do: the reassembled stream must decode strictly, it must contain no
+    marker of its own, and its ASCII skeleton must match the original's exactly.
+    The last one is what makes this a repair rather than a substitution -- it can
+    only ever change non-ASCII characters.
+    """
+    if not raw or UNREADABLE not in raw:
+        return None, 0
+    fixed = _reassemble(tokens)
+    if fixed is None:
+        return None, 0
+    if _ascii_skeleton(fixed) != _ascii_skeleton(raw):
+        return None, 0
+    return fixed, raw.count(UNREADABLE)
+
+
 def _trace_report(calls) -> dict:
     """``_trace_findings`` plus the one fact that makes it safe to read.
 
@@ -445,6 +510,12 @@ class SuryaWorker:
         #: Per-call token logprobs, collected by the hook installed in load().
         self._calls = []
         self._trace = bool(opts.get("token_trace", False))
+        #: Rebuild a reply's text from the model's own bytes when the text we are
+        #: handed has lost characters. On by default: the alternative is shipping
+        #: text we have MEASURED to be corrupt (see repair_content).
+        self._repair = bool(opts.get("repair_accents", True))
+        #: [{"block": i, "recovered": n}] for this page.
+        self._repairs = []
 
     def load(self):
         """Load surya, matching the API of whichever release is installed.
@@ -491,7 +562,7 @@ class SuryaWorker:
         wrapper only reads, never alters, and swallows everything -- a diagnostic
         that can break OCR would be far worse than no diagnostic.
         """
-        if not self._trace:
+        if not (self._trace or self._repair):
             return
         try:
             from surya.inference.backends import openai_client as _oc
@@ -499,13 +570,18 @@ class SuryaWorker:
             print(f"[surya] token trace unavailable: {e}", file=sys.stderr)
             return
         if getattr(_oc, "_coderai_traced", False):
-            _oc._coderai_sink = self._calls          # re-point at this worker
+            # The module is patched once per process; re-point it at THIS worker.
+            _oc._coderai_sink = self._calls
+            _oc._coderai_repairs = self._repairs
+            _oc._coderai_repair = self._repair
             return
         inner = getattr(_oc, "_generate_one", None)
         if inner is None:
             print("[surya] token trace: no _generate_one to hook", file=sys.stderr)
             return
         _oc._coderai_sink = self._calls
+        _oc._coderai_repairs = self._repairs
+        _oc._coderai_repair = self._repair
 
         def _wrapped(*args, **kwargs):
             result = inner(*args, **kwargs)
@@ -513,7 +589,21 @@ class SuryaWorker:
                 sink = getattr(_oc, "_coderai_sink", None)
                 if sink is not None:
                     sink.append(getattr(result, "logprobs", None))
+                # Repaired HERE, before surya parses the reply, so the blocks, the
+                # HTML, the plain text and the confidences downstream are all built
+                # from correct characters. Repairing further down would mean fixing
+                # the same text in several places and missing one.
+                if getattr(_oc, "_coderai_repair", False):
+                    fixed, n = repair_content(getattr(result, "raw", ""),
+                                              getattr(result, "logprobs", None))
+                    if fixed is not None:
+                        result.raw = fixed
+                        reps = getattr(_oc, "_coderai_repairs", None)
+                        if reps is not None:
+                            reps.append({"recovered": n})
             except Exception:
+                # A repair that can break a page would be worse than the corruption
+                # it fixes; the untouched reply is always an acceptable answer.
                 pass
             return result
 
@@ -558,6 +648,7 @@ class SuryaWorker:
         # a long document would otherwise report the first page's anomalies on the
         # hundredth.
         del self._calls[:]
+        del self._repairs[:]
         if self._mode == "predictor":
             preds = self._predict(img)
         else:
@@ -604,6 +695,16 @@ class SuryaWorker:
                 "text": "\n".join(l["text"] for l in lines)}
         # Only when something actually went unread: a clean page stays byte-identical
         # to what it has always returned.
+        if self._repairs:
+            # Said out loud rather than silently: the text the caller gets is not
+            # byte-for-byte what the server sent, and anyone comparing the two
+            # needs to know why.
+            page["accents_repaired"] = {
+                "blocks": len(self._repairs),
+                "recovered": sum(int(r.get("recovered", 0)) for r in self._repairs),
+                "how": ("rebuilt from the model's own per-token bytes, which "
+                        "decoded cleanly where message.content did not"),
+            }
         if self._trace and UNREADABLE in page["text"]:
             try:
                 page["unreadable_trace"] = _trace_report(self._calls)

docs/ocr.md

@@ -570,6 +570,63 @@ A third, independent discriminator needs no code at all: serve the **same weight
 llama.cpp** (`datalab-to/surya-ocr-2-gguf`, with `surya_serve: "llamacpp"`). A different
 detokenizer on identical weights — accents surviving there would convict vLLM outright.
 
+### The verdict, and the repair (`surya_repair_accents`)
+
+The trace was run. On a page of crisp synthetic Italian, served through the Aruba
+install on vLLM 0.27.1:
+
+```
+unreadable: count=29  blame='the serving path'
+page 0: calls=1  with_logprobs=1  conclusive=True  kinds=None
+   why: the byte stream decoded cleanly, so the characters were lost AFTER the
+        model: the fault is in the serving path, not the model
+```
+
+**surya-ocr-2's own byte stream was valid UTF-8**, with every line reported at
+`conf: 0.9605`. The model read the page correctly and something after it destroyed
+the accents. Note what that does *not* implicate: surya-ocr-2 and chandra-ocr-2 are
+the **same architecture** (`Qwen3_5ForConditionalGeneration`) on the same vLLM, and
+chandra's accents survive — so this is not vLLM mishandling `qwen3_5`, it is vLLM
+mishandling **surya's custom 65,425-entry byte-level vocabulary**, where one accented
+letter is two tokens and either byte alone decodes to a marker. It was also a single
+model call, so concurrency is not involved.
+
+Since the correct bytes arrive in the same response's logprobs,
+`ocr.surya_repair_accents` (**on by default**) rebuilds the reply from them. It is
+applied inside the backend wrapper *before* surya parses the HTML, so the blocks, the
+boxes, the plain text and the confidences downstream are all built from correct
+characters — repairing further down would mean fixing the same text in several places
+and missing one.
+
+Rewriting a model's output is a serious thing to do, so it is guarded three ways and
+refuses unless all three hold:
+
+1. the reassembled byte stream decodes **strictly** as UTF-8;
+2. the result contains **no marker of its own**;
+3. its **ASCII skeleton is identical** to the original's.
+
+The third is what makes this a repair rather than a substitution: it can only ever
+change non-ASCII characters. A truncation, a different block or a shifted offset all
+change the skeleton and the original is kept, markers and all. Nothing happens when
+the text is clean — not even a reassembly attempt — and a page that was repaired says
+so:
+
+```json
+{ "accents_repaired": { "blocks": 1, "recovered": 29,
+                        "how": "rebuilt from the model's own per-token bytes, …" } }
+```
+
+Set it false to see exactly what the server sent.
+
+**The other half: `coderai-vllm-ocr`.** A pod image pinned to **vLLM 0.20.1** — the
+version `surya-ocr` itself spawns (`surya/settings.py:96`) — so the model can be served
+on the stack its authors validate, the repair can be checked against a reference rather
+than trusted, and an upstream bug report has a known-good comparison. Select it per
+model with `runpod.engine: "coderai-vllm-ocr"`; build it with
+`./packaging/runpod/build_capability_image.sh vllm-ocr`. It is **not** a drop-in for
+`coderai-vllm`: 0.20.1 is seven minor releases behind, and anything else on that image
+is pinned back with it, so rent it for the OCR model only.
+
 ### Where `conf` comes from on a served engine
 
 A VLM pipeline has no detector score to report, so surya asks the model server for

packaging/runpod/profiles/vllm-ocr.apt

+# Triton JIT-compiles vLLM's kernels at run time and needs a C compiler for its
+# launcher stubs; without one the engine core dies before it registers.
+gcc
+libc6-dev

packaging/runpod/profiles/vllm-ocr.txt

+# vLLM pinned to the version Surya validates against, for Surya-2 OCR.
+#
+# Same shape as the `vllm` profile — vLLM in its own venv on the LIGHT core,
+# driven by coderai's vLLM proxy backend — but pinned to **v0.20.1** instead of
+# 0.27.1, because that is the version `surya-ocr` itself spawns and tests:
+#
+#     surya/settings.py:96   VLLM_DOCKER_IMAGE = "vllm/vllm-openai:v0.20.1"
+#     surya/inference/backends/vllm.py:1  "...docker image with MTP=2"
+#
+# WHY A SEPARATE IMAGE EXISTS AT ALL. Measured on the Digesta orchestrator,
+# 2026-10-12, serving `datalab-to/surya-ocr-2` on vLLM 0.27.1: a page of crisp
+# synthetic Italian came back with one U+FFFD per accented letter — 29 of them,
+# zero accents surviving — while
+#
+#   * the model's OWN byte stream decoded cleanly as UTF-8 (ocr.surya_token_trace
+#     reported conclusive: true, no findings, blame: "the serving path"),
+#   * the model reported ~96% confidence on every line,
+#   * chandra-ocr-2 read the identical pixels with all 12 accents intact,
+#   * and it was a single model call, so batching was not involved.
+#
+# The characters are therefore lost between vLLM's own per-token bytes and the
+# `message.content` surya reads. Note what that does and does not implicate:
+# surya-ocr-2 and chandra-ocr-2 are the SAME architecture
+# (`Qwen3_5ForConditionalGeneration`) on the same vLLM, so this is not vLLM
+# mishandling qwen3_5 — it is vLLM mishandling SURYA'S CUSTOM 65,425-entry
+# BYTE-LEVEL vocabulary, in which one accented letter is two tokens and either
+# byte alone decodes to a replacement character.
+#
+# `ocr.surya_repair_accents` (on by default) already repairs this from the
+# logprob bytes, which is cheaper and needs no new image. This profile is the
+# other half of the answer: it serves the model on the stack its authors test,
+# so the repair can be checked against a reference rather than trusted, and so
+# the upstream bug report has a known-good comparison.
+#
+# NOT a drop-in replacement for `vllm`: 0.20.1 is seven minor releases behind,
+# and anything else that image serves would be pinned back with it. Rent this
+# one for the OCR model only.

packaging/runpod/profiles/vllm-ocr.venv-vllm.check

+# Deep check for the pinned-0.20.1 vllm venv: the OpenAI server module must
+# import, and the version must be the one this image exists to pin — a silent
+# resolve to something newer would defeat the whole point of the image.
+import vllm.entrypoints.openai.api_server                 # noqa
+import torch, vllm
+print("vllm", vllm.__version__, "torch", torch.__version__, "cuda", torch.version.cuda)
+assert vllm.__version__.startswith("0.20."), (
+    "this image exists to pin vLLM 0.20.x (what surya-ocr validates against); "
+    "got %s" % vllm.__version__)
+
+# Surya-2 is Qwen3.5-VL, so the architecture has to be in this vLLM's registry or
+# the pod cannot serve the model at all. Checked through the registry's own
+# lookup where one is exposed and reported-not-enforced otherwise: the private
+# table has been renamed before, and failing a BUILD on the name of a private
+# symbol would be failing for the wrong reason.
+_ARCH = "Qwen3_5ForConditionalGeneration"
+try:
+    from vllm.model_executor.models import registry as _reg
+    _names = set()
+    for _attr in ("_MULTIMODAL_MODELS", "_TEXT_GENERATION_MODELS", "_VLLM_MODELS"):
+        _names |= set(getattr(_reg, _attr, {}) or {})
+    if _names:
+        assert _ARCH in _names, "this vLLM cannot serve datalab-to/surya-ocr-2"
+        print("registry:", _ARCH, "present")
+    else:
+        print("registry: could not enumerate architectures on this vLLM; skipped")
+except ImportError as exc:
+    print("registry: not inspectable (%s); skipped" % exc)

packaging/runpod/profiles/vllm-ocr.venv-vllm.txt

+# vLLM 0.20.1 — the version `surya-ocr` 0.22.1 spawns and validates against
+# (surya/settings.py:96). See vllm-ocr.txt for why this image exists.
+#
+# The CUDA trap is the same one the 0.27.1 profile documents: the PyPI wheel is
+# built against a CUDA the pod hosts do not have, so the +cu129 variant from
+# vLLM's own index is used, with torch's matching cu129 builds. CUDA 12.x
+# minor-version compatibility runs a 12.9 runtime on a 12.8 driver. Resolved
+# with uv (the .uv marker); pip cannot cope with the pin set.
+#
+# The torch triple is NOT a guess: it is what vLLM 0.20.1's own metadata pins
+# (torch==2.11.0, torchvision==0.26.0, torchaudio==2.11.0), and every one of
+# those exists as a +cu129 cp313 wheel on torch's index — checked before this
+# file was written, because a profile that cannot build is worse than none.
+--extra-index-url https://wheels.vllm.ai/0.20.1/cu129
+--extra-index-url https://download.pytorch.org/whl/cu129
+vllm==0.20.1+cu129
+torch==2.11.0+cu129
+torchvision==0.26.0+cu129
+torchaudio==2.11.0+cu129
+# NO torchcodec pin here, deliberately. 0.27.1 needs one (>=0.14) and pinning it
+# to a cu129 build is what stops `import vllm.entrypoints.openai.api_server`
+# dying on libnvrtc.so.13 — but 0.20.1's PyPI metadata does not list torchcodec
+# at all (checked), and pinning a package nothing requires is its own hazard. If
+# a build ever resolves one from PyPI and the server import fails on an nvrtc
+# soname, add `torchcodec==0.11.1+cu129` (it exists for cp313 and matches torch
+# 2.11).
+#
+# transformers: 0.20.1 excludes 5.0-5.4 and 5.5.0, so uv lands on 5.5.1+. That
+# is required anyway — surya-ocr-2 is `qwen3_5`, which only transformers 5.x
+# knows.