chunk: POST /v1/chunk, and the OCR pipeline for all three VLM engines (v0.3.76)

RETRIEVAL CHUNKING. A new endpoint that splits a document into the pieces an
index stores, with the document's structure decided per request by a named or
inline template. NOT embedding windowing: `embed_long_input` makes ONE input fit
a model window and pools it into ONE vector; this produces MANY chunks, each
indexable, each with its section, parent and references. It never pools and
never drops text.

Pure CPU -- token counts are characters / chars_per_token, so no tokenizer is
loaded and no model, GPU or pod is involved. Every image serves it, light-core
ones included, and a test asserts no torch/transformers import creeps in.

The packing loop is reproduced verbatim, including the parts that look
improvable: paragraphs joined with "\n\n" (hence the `+ 2`), and an overlap that
is a RAW CHARACTER TAIL which may begin mid-word. Changing any of it moves every
existing chunk boundary, and then a re-chunked index cannot be compared with the
one it replaces.

`conclusioni` is its own label and is never folded into `dispositivo`: in a
ruling the dispositivo is what the judge decided, in a filing the conclusioni
are what a party ASKED FOR, and merging them would let a demand surface as a
holding.

TWO DEVIATIONS FROM THE SPEC, both recorded in the source where someone
comparing against the original will look:

  * The ceiling is target + overlap + 2 (3682), not the 3680 the prose states --
    which agrees with the spec's own "observed max is 3682" two lines later.
    Asserting 3680 would fail a correct port.
  * The verbatim patterns cannot match `**FATTO**`, yet the fidelity list asks
    for that case. The line-anchored alternatives are widened at BOTH ends
    (`^[\s*#>_]*fatto[\s*#>_]*$`), which is how the patterns that already coped
    spell it. Purely additive: every line that matched still matches, so no
    existing match moves -- but a document whose headers were emphasised used to
    fall back to flat chunking and now gets sectioned, so ITS chunks change and
    it needs re-chunking.

FORWARDABLE, with the hazard that comes with it. /v1/chunk rides the EMBEDDINGS
capability: chunks are on their way to be embedded, that image is full-core so
it carries the chunker, and a capability of its own would be one nothing
advertises -- or would rent a GPU to run a regex split. It was first classified
as deliberately local on the grounds that it is cheap; that was the wrong lens,
since the orchestrator is the box that was OOM-killed today, so WHERE the CPU is
matters more than how much there is.

But a template registered HERE does not exist on the remote: forwarded by name
it returns 404, or worse matches a DIFFERENT template with the same name and
returns different boundaries silently. So a named non-builtin template is
resolved and INLINED before the request leaves. The hook never raises -- a body
it cannot parse is forwarded untouched -- because a rewrite that can throw would
take down every forwarded request, not just a chunk one.

THE OCR PIPELINE NOW COVERS ALL THREE VLM ENGINES, not just surya. Leaving
chandra and olmOCR on the orchestrator was an asymmetry rather than a design:
surya needed a 1.6 GB client venv baked in, while those two speak only HTTP in
served mode (PIL + requests, already in the main venv), so for them the pod side
costs nothing beyond the rasteriser. Two bugs fixed on the way, both of which
would have produced plausible WRONG text rather than an error: the dispatch
hardcoded `engine: "surya"` in the form it posted, and the pod env was
surya-shaped throughout -- a chandra pod was told to run surya against chandra's
model id. Per-engine tuning and per-engine licence acceptance now travel.

AND THE LOG THAT CAUSED A MISDIAGNOSIS. One pod request walks a candidate list
until a (card, region) pair has capacity, and each attempt printed
"provisioning pod for ...". 312 attempts for 10 pods therefore read as runaway
provisioning and was reported as such. It now says "trying candidate N/M for
ONE pod", so the shape of the list is visible instead of inferred.

What these tests CANNOT do is prove byte-identity with the implementation this
ports -- that needs fixtures from a live corpus. They prove the documented
behaviour on each path; identity is what decides whether a re-chunk is safe and
it is still outstanding.

Suite at the 25 pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.75"
+__version__ = "0.3.76"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/app.py

@@ -760,6 +760,10 @@ app.include_router(loras_router, tags=["LoRAs"])
 app.include_router(environments_router, tags=["Environments"])
 app.include_router(spatial_router, tags=["Spatial / 3D"])
 app.include_router(ocr_router, tags=["OCR"])
+# Retrieval chunking. A pure CPU text operation (no tokenizer, no model,
+# no GPU), so every image serves it -- including the light-core ones.
+from codai.api.chunking import router as chunking_router   # noqa: E402
+app.include_router(chunking_router, tags=["Chunking"])
 # The address an attached OCR engine drives: one stable URL of ours, forwarded
 # per request to the least-busy server that holds the model. Not in the schema —
 # it is machine-to-machine, authenticated by a lease rather than an API token.

codai/api/remote_gateway.py

@@ -72,6 +72,17 @@ _CAPABILITY_PREFIXES = (
     ("/v1/video", "video"),
     ("/v1/3d", "spatial"),
     ("/v1/embeddings", "embeddings"),
+    # Retrieval chunking rides the EMBEDDINGS capability rather than inventing
+    # one of its own. Two reasons: chunks are on their way to be embedded, so
+    # the machine that will embed them is the natural place to cut them; and the
+    # embeddings image is full-core, so it carries the chunker. Inventing a
+    # `chunk` capability would mean a capability nothing advertises, and renting
+    # a GPU pod to run a regex split.
+    #
+    # It is pure CPU either way -- token counts are characters /
+    # chars_per_token, no tokenizer, no model -- so forwarding it is about WHERE
+    # the CPU is, which matters when the orchestrator is the busy box.
+    ("/v1/chunk", "embeddings"),
     ("/v1/rerank", "rerank"),
     ("/v1/ocr", "ocr"),
     ("/v1/loras", "loras"),
@@ -144,6 +155,51 @@ _MULTIPART_MODEL = re.compile(
     rb'name="model"\r?\n(?:[^\r\n]*\r?\n)*?\r?\n([^\r\n]*)', re.IGNORECASE)
 
 
+def _inline_chunk_template(path: str, headers: dict, body: bytes) -> bytes:
+    """Resolve a NAMED /v1/chunk structure template into its object before the
+    request leaves this machine.
+
+    A template registered in THIS install's config does not exist on the
+    remote. Forwarded by name it would come back 404 `unknown_template` -- or,
+    far worse, match a DIFFERENT template that happens to share the name and
+    return different chunk boundaries with no indication that anything
+    happened. Chunk boundaries that silently differ by which machine answered
+    make a retrieval index unmeasurable, which is the whole thing
+    docs/chunking.md warns about.
+
+    Built-in names are left alone: every coderai has those, and sending the
+    object would only make the request bigger.
+
+    Never raises: a body it cannot parse is forwarded untouched, and the remote
+    then answers for itself.
+    """
+    if (path or "").rstrip("/") != "/v1/chunk":
+        return body
+    if "json" not in (headers.get("content-type") or "").lower():
+        return body            # multipart goes to /v1/chunk/file; same rule there
+    try:
+        import json as _json
+        obj = _json.loads(body or b"{}")
+        if not isinstance(obj, dict):
+            return body
+        name = obj.get("structure")
+        if not isinstance(name, str) or not name.strip():
+            return body        # already inline, or defaulted
+        from codai.api.chunking import BUILTIN_TEMPLATES, resolve_template
+        if name.strip() in BUILTIN_TEMPLATES:
+            return body
+        from codai.api.chunking import _registered_templates
+        tpl = resolve_template(name, _registered_templates())
+        if not isinstance(tpl, dict):
+            return body
+        obj["structure"] = tpl
+        print(f"[remote] /v1/chunk: inlining the {name!r} template so the remote "
+              f"cuts on the same boundaries this install would", flush=True)
+        return _json.dumps(obj).encode("utf-8")
+    except Exception:
+        return body
+
+
 def capability_for(path: str) -> str:
     """The capability a /v1 path belongs to, or '' when it maps to none."""
     p = (path or "").rstrip("/") or "/"
@@ -673,6 +729,7 @@ class RemoteGatewayMiddleware:
             return await _error(send, 502, f"remote routing failed: {exc}")
         if not target:
             return await self._app(scope, replay, send)
+        body = _inline_chunk_template(scope.get("path") or "", headers, body)
         await self._forward(scope, headers, body, target, send)
 
     async def _forward(self, scope, headers, body, target, send):

codai/api/runpod_worker.py

@@ -1254,46 +1254,72 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
     # capability pod is not what makes its pod able to read a page.
     if pod_runs_ocr_pipeline(mcfg, image):
         _o = _ocr_cfg()
+        # WHICH engine this pod's pipeline is for. A pod serves one model, and
+        # that model is what decides -- surya-ocr-2, chandra-ocr-2 or olmOCR.
+        # Hardcoding surya told a chandra pod to run surya against chandra's
+        # model id, which would have produced plausible-looking text from the
+        # wrong engine.
+        _served_l = str(served or "").lower()
+        _eng = "surya"
+        for _cand in ("chandra", "olmocr", "surya"):
+            if _cand in _served_l:
+                _eng = _cand
+                break
         env["CODERAI_OCR_ENABLED"] = "1"
-        env["CODERAI_OCR_SURYA_ENABLED"] = "1"
-        env["CODERAI_OCR_DEFAULT_ENGINE"] = "surya"
+        env[f"CODERAI_OCR_{_eng.upper()}_ENABLED"] = "1"
+        env["CODERAI_OCR_DEFAULT_ENGINE"] = _eng
         # The baked venv, so the pod does not try to BUILD one on a rented
-        # machine (minutes, and it would land on the container disk).
-        env["CODERAI_OCR_SURYA_VENV"] = "/opt/coderai/venvs/surya"
+        # machine (minutes, and it would land on the container disk). Only
+        # surya has one: chandra and olmOCR in served mode speak HTTP from the
+        # main venv (PIL + requests), which is why the pod side is nearly free
+        # for them and needed a 1.6 GB venv for surya.
+        if _eng == "surya":
+            env["CODERAI_OCR_SURYA_VENV"] = "/opt/coderai/venvs/surya"
         # Surya drives the model through the pod's OWN coderai, in the same
         # "model" mode the orchestrator uses -- which is what makes the gateway
         # mint it an attach token instead of needing a bearer hand-wired here.
         # Over loopback, so a page resumed four times costs four local round
         # trips rather than four across the internet.
-        env["CODERAI_OCR_SURYA_SERVE"] = "model"
-        env["CODERAI_OCR_SURYA_MODEL_ID"] = str(served or "").strip()
-        # Surya is GPL and gated on an explicit acceptance. The pod inherits the
-        # decision made here; it cannot make it for itself.
-        env["CODERAI_OCR_SURYA_ACCEPT_LICENSE"] = "1"
+        env[f"CODERAI_OCR_{_eng.upper()}_SERVE"] = "model"
+        env[f"CODERAI_OCR_{_eng.upper()}_MODEL_ID"] = str(served or "").strip()
+        # Licences are inherited, never decided by the pod: surya is GPL and
+        # chandra's weights are a modified OpenRAIL-M. Asking for the engine by
+        # name HERE is the decision.
+        if _eng == "surya":
+            env["CODERAI_OCR_SURYA_ACCEPT_LICENSE"] = "1"
+        elif _eng == "chandra":
+            env["CODERAI_OCR_CHANDRA_ACCEPT_LICENSE"] = "1"
         # This image carries NO docTR -- `auto` would route a failed page to an
-        # engine that is not there. Surya or nothing, said out loud.
+        # engine that is not there. This engine or nothing, said out loud.
         env["CODERAI_OCR_FALLBACK_ENGINES"] = "off"
         # And the TUNING. Every one of these was set in response to a measured
         # failure, and a pod reading pages at surya's defaults instead would be
         # indistinguishable from the "pre-fix pod image" that got blamed three
         # times for orchestrator bugs.
-        for _var, _attr in (("CODERAI_OCR_DPI", "dpi"),
-                            ("CODERAI_OCR_SURYA_DPI", "surya_dpi"),
-                            ("CODERAI_OCR_SURYA_MAX_TOKENS", "surya_max_tokens"),
-                            ("CODERAI_OCR_SURYA_MAX_CONTINUATIONS",
-                             "surya_max_continuations"),
-                            ("CODERAI_OCR_SURYA_CONTEXT_TOKENS",
-                             "surya_context_tokens"),
-                            ("CODERAI_OCR_SURYA_MAX_RETRIES", "surya_max_retries"),
-                            ("CODERAI_OCR_SURYA_REPEAT_MAX_REPEATS",
-                             "surya_repeat_max_repeats"),
-                            ("CODERAI_OCR_SURYA_REPEAT_WINDOW",
-                             "surya_repeat_window"),
-                            ("CODERAI_OCR_SURYA_REPEAT_SCALING",
-                             "surya_repeat_scaling"),
-                            ("CODERAI_OCR_SURYA_LANES", "surya_lanes"),
-                            ("CODERAI_OCR_SURYA_INSTANCES", "surya_instances"),
-                            ("CODERAI_OCR_SURYA_LANGS", "surya_langs"),
+        # Per-engine tuning, by name, so a chandra pod gets chandra's budgets
+        # rather than surya's. Every one of these was set in response to a
+        # measured failure, and a pod running the engine's own defaults instead
+        # is indistinguishable from the "pre-fix pod image" that got blamed
+        # three times for orchestrator bugs.
+        _E = _eng.upper()
+        _per_engine = [(f"CODERAI_OCR_{_E}_DPI", f"{_eng}_dpi"),
+                       (f"CODERAI_OCR_{_E}_MAX_TOKENS", f"{_eng}_max_tokens"),
+                       (f"CODERAI_OCR_{_E}_MAX_CONTINUATIONS",
+                        f"{_eng}_max_continuations"),
+                       (f"CODERAI_OCR_{_E}_CONTEXT_TOKENS",
+                        f"{_eng}_context_tokens"),
+                       (f"CODERAI_OCR_{_E}_MAX_RETRIES", f"{_eng}_max_retries"),
+                       (f"CODERAI_OCR_{_E}_INSTANCES", f"{_eng}_instances"),
+                       (f"CODERAI_OCR_{_E}_LANES", f"{_eng}_lanes"),
+                       (f"CODERAI_OCR_{_E}_LANGS", f"{_eng}_langs")]
+        if _eng == "surya":
+            _per_engine += [
+                ("CODERAI_OCR_SURYA_REPEAT_MAX_REPEATS",
+                 "surya_repeat_max_repeats"),
+                ("CODERAI_OCR_SURYA_REPEAT_WINDOW", "surya_repeat_window"),
+                ("CODERAI_OCR_SURYA_REPEAT_SCALING", "surya_repeat_scaling")]
+        for _var, _attr in _per_engine + [
+                            ("CODERAI_OCR_DPI", "dpi"),
                             ("CODERAI_OCR_MAX_CONCURRENCY", "max_concurrency"),
                             ("CODERAI_OCR_PAGE_WINDOW", "page_window"),
                             ("CODERAI_OCR_RASTER_WORKERS", "raster_workers"),
@@ -1301,7 +1327,7 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
                              "blank_page_max_chars"),
                             ("CODERAI_OCR_DETECT_MODE", "detect_mode"),
                             ("CODERAI_OCR_DETECT_MODEL_PATH", "detect_model_path"),
-                            ("CODERAI_OCR_DETECT_CONF", "detect_conf")):
+                            ("CODERAI_OCR_DETECT_CONF", "detect_conf")]:
             _val = getattr(_o, _attr, None) if _o is not None else None
             if _val not in (None, "", 0, 0.0):
                 env[_var] = str(_val)
@@ -1321,9 +1347,9 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
                   "it was never given, and will report no stamps rather than "
                   "fail. Set the path (staged on the pod) or use detect_mode="
                   "layout.", flush=True)
-        print(f"[runpod] the OCR PIPELINE runs on this pod ({image}): it is sent "
-              f"one page as a PDF and reads it on the card, with its own vLLM "
-              f"over loopback", flush=True)
+        print(f"[runpod] the {_eng} OCR PIPELINE runs on this pod ({image}): it "
+              f"is sent one page as a PDF and reads it on the card, with its own "
+              f"vLLM over loopback", flush=True)
 
     if cap == "ocr":
         env["CODERAI_OCR_ENABLED"] = "1"
@@ -2719,7 +2745,15 @@ class RunpodPodPool:
                 print(f"[runpod] {_dc or '(no region)'}: attaching volume "
                       f"{_v or '(none — this region has no volume configured)'}",
                       flush=True)
-            print(f"[runpod] provisioning pod for {self.model_key!r}: {sel['display_name']} "
+            # ONE pod request walks this whole list until a (card, region) pair
+            # has capacity, so this line is printed PER CANDIDATE -- and it used
+            # to say "provisioning pod", which reads as a pod being launched.
+            # 312 attempts for 10 pods in one afternoon therefore looked like
+            # runaway provisioning to an operator reading the log, and was
+            # reported as such. Say which attempt this is, out of how many, so
+            # the shape of the list is visible instead of inferred.
+            print(f"[runpod] {self.model_key!r}: trying candidate "
+                  f"{i + 1}/{len(ranked)} for ONE pod — {sel['display_name']} "
                   f"({sel['cloud_type']}{'/spot' if sel['is_spot'] else ''}) ${sel['price']}/hr"
                   + (f" in {_dc}" if _dc else " (region: RunPod's choice)") + _alt,
                   flush=True)

codai/config.py

@@ -380,6 +380,41 @@ class EnhanceConfig:
     allow_rife_ncnn: bool = False     # allow the external rife-ncnn-vulkan binary instead of a torch model
 
 
+@dataclass
+class ChunkingConfig:
+    """Retrieval chunking (`POST /v1/chunk`).
+
+    NOT embedding windowing. `ocr`/`embeddings` settings make one input fit a
+    model window and pool it into ONE vector; this splits a document into MANY
+    separately-indexable chunks, each with its section, parent and references.
+
+    Token counts are approximated as characters / `chars_per_token`, so the
+    endpoint loads no tokenizer and needs no model, no GPU and no pod -- which
+    is what lets every image serve it, light-core ones included.
+    """
+    # The defaults are the ones a production index was built with: changing them
+    # changes every chunk boundary, which makes a re-chunked index incomparable
+    # with the one it replaces and retrieval quality unmeasurable across the
+    # change. Treat them as a format, not a preference.
+    chunk_tokens: int = 800              # target size, approximated tokens
+    overlap: int = 120                   # carried tail, approximated tokens
+    chars_per_token: int = 4             # the approximation divisor
+    structural: bool = True              # False = flat length chunking
+    structure: str = "it-legal"          # default template name
+    require_text: bool = True            # refuse container bytes (see below)
+    # A chunker that accepts a PDF or an OOXML container embeds garbage that the
+    # index cannot afterwards distinguish from content. Measured on a production
+    # corpus: 0.59% of chunks held control characters, 0.04% raw OOXML --
+    # documents whose container was chunked instead of its extracted text.
+    # `require_text: false` opts out deliberately.
+    max_chars: int = 8_000_000           # above this -> 413 too_large
+    # Operator-supplied structure templates, same JSON shape as the built-in
+    # `it-legal`, selected by name. An unknown name is an ERROR, never a silent
+    # fallback: a caller who asked for their own template and silently got
+    # Italian legal sectioning would index the wrong boundaries and never know.
+    templates: dict = field(default_factory=dict)
+
+
 @dataclass
 class CompactionConfig:
     """Global defaults for auto-compaction of an over-long chat history.
@@ -1379,6 +1414,7 @@ class Config:
     runpod: RunpodConfig = field(default_factory=RunpodConfig)
     ocr: OcrConfig = field(default_factory=OcrConfig)
     compaction: CompactionConfig = field(default_factory=CompactionConfig)
+    chunking: ChunkingConfig = field(default_factory=ChunkingConfig)
     broker: BrokerConfig = field(default_factory=BrokerConfig)
     system_prompt: Optional[str] = None
     tools_closer_prompt: bool = False
@@ -1576,6 +1612,7 @@ class ConfigManager:
                 runpod=_dc(RunpodConfig, config_data.get("runpod", {})),
                 ocr=_dc(OcrConfig, config_data.get("ocr", {})),
                 compaction=_dc(CompactionConfig, config_data.get("compaction", {})),
+                chunking=_dc(ChunkingConfig, config_data.get("chunking", {})),
                 broker=_dc(BrokerConfig, config_data.get("broker", {})),
                 system_prompt=config_data.get("system_prompt"),
                 tools_closer_prompt=config_data.get("tools_closer_prompt", False),
@@ -2017,6 +2054,16 @@ class ConfigManager:
                 "extract_schema": self.config.ocr.extract_schema,
                 "extract_validate": self.config.ocr.extract_validate,
             },
+            "chunking": {
+                "chunk_tokens": self.config.chunking.chunk_tokens,
+                "overlap": self.config.chunking.overlap,
+                "chars_per_token": self.config.chunking.chars_per_token,
+                "structural": self.config.chunking.structural,
+                "structure": self.config.chunking.structure,
+                "require_text": self.config.chunking.require_text,
+                "max_chars": self.config.chunking.max_chars,
+                "templates": self.config.chunking.templates,
+            },
             "compaction": {
                 "enabled": self.config.compaction.enabled,
                 "pct": self.config.compaction.pct,

codai/ocr/manager.py

@@ -349,7 +349,7 @@ def _warn_if_oversubscribed(name: str, pages: int) -> None:
 
 
 def _post_page_to_pod(base: str, api_key: str, blob: bytes, name: str,
-                      dpi: int, opts: dict, detect) -> dict:
+                      dpi: int, opts: dict, detect, engine: str = "surya") -> dict:
     """POST one page to a pod's own /v1/ocr and return its response dict.
 
     Through `pod_http`, not `requests`: a pod on the direct path serves HTTPS
@@ -363,7 +363,10 @@ def _post_page_to_pod(base: str, api_key: str, blob: bytes, name: str,
         url = url + "/ocr"
     else:
         url = url + "/v1/ocr"
-    fields = {"engine": "surya"}
+    # The engine the POD must use. Hardcoded "surya" sent a chandra page to
+    # surya, which the pod would have served with the wrong model -- and the
+    # text would have come back looking plausible.
+    fields = {"engine": str(engine or "surya")}
     if dpi:
         fields["dpi"] = str(int(dpi))
     if detect:
@@ -420,6 +423,9 @@ def _page_from_pod(d: dict, index: int) -> OcrPage:
 def pod_pipeline_model(cfg, engine: str = "") -> str:
     """The model id whose pods run the OCR pipeline themselves, or "".
 
+    Answers for whichever VLM engine is being used -- surya, chandra or
+    olmOCR -- each read from its own `<engine>_serve` / `<engine>_model_id`.
+
     THE IMAGE IS THE SWITCH. A model served by `coderai-vllm-ocr` carries surya
     and the rasteriser, so the page goes to it as a one-page PDF; one served by
     `coderai-vllm` serves only the model and the pipeline stays here. There is
@@ -429,12 +435,19 @@ def pod_pipeline_model(cfg, engine: str = "") -> str:
     Returns "" for every other case -- no model mode, no entry, a different
     image -- so the caller simply does what it has always done.
     """
-    name = (engine or "").strip().lower()
-    if name and name != "surya":
+    name = (engine or "").strip().lower() or str(
+        getattr(cfg, "default_engine", "") or "").strip().lower()
+    # All three VLM engines can run on the pod, not just surya. The image is
+    # what differs: surya needs its client venv baked in, while chandra and
+    # olmOCR in served mode speak only HTTP (PIL + requests, already present),
+    # so for them the pod side costs nothing beyond the rasteriser. Leaving
+    # them on the orchestrator while surya moved was an asymmetry, not a
+    # design.
+    if name not in ("surya", "chandra", "olmocr"):
         return ""
-    if str(getattr(cfg, "surya_serve", "") or "").strip().lower() != "model":
+    if str(getattr(cfg, f"{name}_serve", "") or "").strip().lower() != "model":
         return ""
-    mid = str(getattr(cfg, "surya_model_id", "") or "").strip()
+    mid = str(getattr(cfg, f"{name}_model_id", "") or "").strip()
     if not mid:
         return ""
     try:
@@ -1217,7 +1230,9 @@ class OcrManager:
             try:
                 _out = await self._read_on_pods(
                     data, filename, content_type, _remote_model,
-                    dpi=use_dpi, page_opts=page_opts, detect=detect, cfg=cfg)
+                    dpi=use_dpi, page_opts=page_opts, detect=detect, cfg=cfg,
+                    engine_name=(engine or "").strip().lower()
+                    or str(getattr(cfg, "default_engine", "") or "").lower())
             except OcrError:
                 raise
             except Exception as exc:
@@ -1275,7 +1290,8 @@ class OcrManager:
             return 0
 
     async def _read_on_pods(self, data: bytes, filename: str, content_type: str,
-                            model_id: str, *, dpi: int, page_opts, detect, cfg):
+                            model_id: str, *, dpi: int, page_opts, detect, cfg,
+                            engine_name: str = "surya"):
         """Send ONE PAGE PER REQUEST to pods that run the pipeline themselves.
 
         The page travels as a small PDF and the pod renders and reads it on the
@@ -1318,7 +1334,7 @@ class OcrManager:
                     return await asyncio.to_thread(
                         _post_page_to_pod, lease.base, lease.api_key, blob,
                         f"{idx}.pdf" if is_pdf else (filename or "page"),
-                        dpi, opts, detect)
+                        dpi, opts, detect, engine_name)
                 finally:
                     lease.release()
 

tests/test_ocr_dual_mode.py

@@ -344,9 +344,9 @@ def test_pages_come_back_in_document_order(monkeypatch):
 
     seen = []
 
-    def _post(base, key, blob, name, dpi, opts, detect):
+    def _post(base, key, blob, name, dpi, opts, detect, engine="surya"):
         seen.append(name)
-        return {"engine": "surya", "pages": [{"text": name, "width": 1, "height": 2}]}
+        return {"engine": engine, "pages": [{"text": name, "width": 1, "height": 2}]}
     monkeypatch.setattr(om, "_post_page_to_pod", _post)
     name, pages, blank, stamps, sigs = asyncio.run(mgr._read_on_pods(
         _pdf(5), "d.pdf", "application/pdf", "m",
@@ -468,3 +468,109 @@ def test_the_fallback_message_names_the_cause_not_just_the_effect():
     fn = src.split("async def _read_on_pods")[1].split("\n    def ")[0]
     assert "placement answered 'local'" in fn
     assert "nothing rented" in fn
+
+
+# ------------------------------- all three VLM engines, not just surya #
+#
+# surya needed a 1.6 GB client venv baked into the image. chandra and olmOCR in
+# served mode speak only HTTP (PIL + requests, already in the main venv), so for
+# them the pod side costs nothing beyond the rasteriser. Leaving them on the
+# orchestrator while surya moved was an asymmetry, not a design.
+class _MultiCfg:
+    surya_serve = "model"
+    surya_model_id = "datalab-to/surya-ocr-2"
+    chandra_serve = "model"
+    chandra_model_id = "datalab-to/chandra-ocr-2"
+    olmocr_serve = "model"
+    olmocr_model_id = "allenai/olmOCR-2-7B-1025-FP8"
+    default_engine = "surya"
+    max_concurrency = 8
+    page_window = 4
+    blank_page_max_chars = 0
+    fallback_engines = "off"
+
+
+@pytest.mark.parametrize("engine,expect", [
+    ("surya", "datalab-to/surya-ocr-2"),
+    ("chandra", "datalab-to/chandra-ocr-2"),
+    ("olmocr", "allenai/olmOCR-2-7B-1025-FP8"),
+])
+def test_every_vlm_engine_can_take_the_pod_path(monkeypatch, engine, expect):
+    monkeypatch.setattr(rw, "_model_entry", lambda mid: {
+        "runpod": {"image": "ghcr.io/nextime/coderai-vllm-ocr:latest"}})
+    assert om.pod_pipeline_model(_MultiCfg(), engine) == expect
+
+
+@pytest.mark.parametrize("engine", ["surya", "chandra", "olmocr"])
+def test_a_plain_vllm_image_keeps_every_engine_local(monkeypatch, engine):
+    monkeypatch.setattr(rw, "_model_entry", lambda mid: {
+        "runpod": {"image": "ghcr.io/nextime/coderai-vllm:latest"}})
+    assert om.pod_pipeline_model(_MultiCfg(), engine) == ""
+
+
+def test_a_non_vlm_engine_never_takes_the_pod_path(monkeypatch):
+    """paddle and docTR run in-process; there is no model server to drive."""
+    monkeypatch.setattr(rw, "_model_entry", lambda mid: {
+        "runpod": {"image": "ghcr.io/nextime/coderai-vllm-ocr:latest"}})
+    for eng in ("paddle", "doctr", "nonsense"):
+        assert om.pod_pipeline_model(_MultiCfg(), eng) == ""
+
+
+def test_each_engine_reads_its_own_serve_mode(monkeypatch):
+    """chandra on `server` mode must not be dragged onto the pod path by
+    surya's `model` mode."""
+    class _C(_MultiCfg):
+        chandra_serve = "server"
+    monkeypatch.setattr(rw, "_model_entry", lambda mid: {
+        "runpod": {"image": "ghcr.io/nextime/coderai-vllm-ocr:latest"}})
+    assert om.pod_pipeline_model(_C(), "chandra") == ""
+    assert om.pod_pipeline_model(_C(), "surya") == "datalab-to/surya-ocr-2"
+
+
+def test_the_dispatch_tells_the_pod_WHICH_engine(monkeypatch):
+    """Hardcoded "surya" sent a chandra page to surya, which the pod would have
+    served with the wrong model -- and the text would have come back looking
+    plausible."""
+    from codai.api import pod_http
+    seen = {}
+
+    class _R:
+        status_code = 200
+
+        def json(self):
+            return {"engine": "chandra", "pages": []}
+    monkeypatch.setattr(pod_http, "post",
+                        lambda url, **kw: seen.update(kw) or _R())
+    om._post_page_to_pod("https://1.2.3.4:8000/v1", "t", b"%PDF-x", "0.pdf",
+                         250, {}, None, "chandra")
+    assert seen["data"]["engine"] == "chandra"
+
+
+def test_the_pod_env_is_shaped_by_the_served_model_not_hardcoded():
+    src = (ROOT / "codai/api/runpod_worker.py").read_text()
+    blk = src.split("if pod_runs_ocr_pipeline(mcfg, image):")[1] \
+             .split('if cap == "ocr":')[0]
+    assert 'env[f"CODERAI_OCR_{_eng.upper()}_ENABLED"]' in blk
+    assert 'env[f"CODERAI_OCR_{_eng.upper()}_SERVE"]' in blk
+    assert '"CODERAI_OCR_SURYA_ENABLED"' not in blk, "still hardcoded to surya"
+
+
+def test_only_surya_is_pointed_at_a_baked_venv():
+    """chandra and olmOCR have none -- they speak HTTP from the main venv, which
+    is why the pod side is nearly free for them."""
+    src = (ROOT / "codai/api/runpod_worker.py").read_text()
+    blk = src.split("if pod_runs_ocr_pipeline(mcfg, image):")[1] \
+             .split('if cap == "ocr":')[0]
+    assert 'if _eng == "surya":' in blk
+    venv = blk.split('CODERAI_OCR_SURYA_VENV')[0]
+    assert venv.rstrip().endswith('if _eng == "surya":') or 'if _eng == "surya"' in venv
+
+
+def test_each_engine_inherits_its_own_licence_decision():
+    """surya is GPL, chandra's weights are a modified OpenRAIL-M. A pod cannot
+    accept either for itself."""
+    src = (ROOT / "codai/api/runpod_worker.py").read_text()
+    blk = src.split("if pod_runs_ocr_pipeline(mcfg, image):")[1] \
+             .split('if cap == "ocr":')[0]
+    assert "CODERAI_OCR_SURYA_ACCEPT_LICENSE" in blk
+    assert "CODERAI_OCR_CHANDRA_ACCEPT_LICENSE" in blk