ocr: surya should ask who serves the model, not where the GPU is (v0.3.7)

OCR on the Digesta install answered 503: "vLLM isolated venv not found at
/cache/vllm_venv". That host has no GPU at all — renting A40 pods is its whole
job — so the venv it was told to build could never have run, while the Surya-2
checkpoint it needed was already a configured model (backend: runpod,
engine: coderai-vllm) nothing in the OCR path could ask for.

The model layer was never the problem; `ocr.surya_serve` was asking the wrong
question. It enumerated server PROCESSES — torch in a local venv, our own vLLM,
a URL typed in by hand — when the answer the operator had already written down
was a MODEL. olmOCR has had that bridge since it shipped ("model" mode, through
our own chat path); Surya never got one, because Surya2 drives its server
directly with guided-JSON decoding and logprobs that a proxy hop would drop.

So it gets the same bridge, in the only shape it can take: resolve the model
through whichever backend owns it and hand the worker the address. Three values,
because Surya refuses a server whose /v1/models disagrees with the checkpoint it
expects, and a pod URL is useless without its token — which only Surya's vLLM
client reads, so a bearer-locked endpoint is always attached to as "vllm".

The address is a lease, not a location: a min_pods=0 pod is reaped when OCR goes
quiet, and the page that finds it gone re-resolves once — renting a fresh one —
instead of every later page failing against a dead URL. A failure against a
server that still answers /health is a real error and is reported as one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.6"
+__version__ = "0.3.7"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/routes.py

@@ -4139,6 +4139,7 @@ def build_settings_dict(c, gpu_cards):
             "surya_auto_build": c.ocr.surya_auto_build,
             "surya_serve": c.ocr.surya_serve,
             "surya_model": c.ocr.surya_model,
+            "surya_model_id": c.ocr.surya_model_id,
             "surya_server_url": c.ocr.surya_server_url,
             "build_retry_cooldown_s": c.ocr.build_retry_cooldown_s,
             "evict_drain_timeout_s": c.ocr.evict_drain_timeout_s,
@@ -4936,6 +4937,7 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
         if "surya_auto_build" in d: o.surya_auto_build = bool(d["surya_auto_build"])
         if "surya_serve" in d: o.surya_serve = (d.get("surya_serve") or "local").strip()
         if "surya_model" in d: o.surya_model = (d.get("surya_model") or "").strip()
+        if "surya_model_id" in d: o.surya_model_id = (d.get("surya_model_id") or "").strip()
         if "surya_server_url" in d: o.surya_server_url = (d.get("surya_server_url") or "").strip()
         if "build_retry_cooldown_s" in d: o.build_retry_cooldown_s = _f("build_retry_cooldown_s", o.build_retry_cooldown_s, 0.0)
         if "vlm_gpu_memory_utilization" in d: o.vlm_gpu_memory_utilization = _f("vlm_gpu_memory_utilization", o.vlm_gpu_memory_utilization, 0.0)

codai/admin/templates/settings.html

@@ -1189,6 +1189,20 @@
       <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-surya-auto-build"><span>Auto-build on first use</span></label>
     </div>
     <div style="display:flex;gap:.5rem;align-items:center;flex-wrap:wrap"><button type="button" class="btn btn-sm" onclick="ocrBuildVenv('surya')">Build Surya venv now</button><span id="s-ocr-surya-build-status" class="form-hint"></span></div>
+    <p class="form-hint" style="margin:.5rem 0 .3rem"><b>Serving.</b> <i>Local</i> runs detection+recognition on torch in that venv (surya-ocr &le;0.17). The others serve the <b>Surya2</b> VLM from an OpenAI server Surya attaches to: <i>Model</i> uses a model from this install's catalogue — a RunPod pod rented on demand, a <code>service_url</code> elsewhere, or the local vLLM backend — and is the only one that works with <b>no GPU on this box</b>; <i>vLLM</i> serves the checkpoint from coderai's own isolated vLLM venv (needs CUDA here); <i>External server</i> attaches to a URL you already run.</p>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <div class="form-group" style="flex:1;min-width:150px"><label class="form-label">Serve</label>
+        <select id="s-ocr-surya-serve" class="form-input">
+          <option value="local">Local (torch, this box)</option>
+          <option value="model">Model (served by this install)</option>
+          <option value="vllm">vLLM (own venv, CUDA)</option>
+          <option value="llamacpp">External server (URL)</option>
+        </select>
+      </div>
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">Surya2 checkpoint</label><input type="text" id="s-ocr-surya-model" class="form-input" placeholder="datalab-to/surya-ocr-2"></div>
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">Model id (Model mode)</label><input type="text" id="s-ocr-surya-model-id" class="form-input" placeholder="datalab-to/surya-ocr-2"></div>
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">Server URL (External)</label><input type="text" id="s-ocr-surya-server-url" class="form-input" placeholder="http://127.0.0.1:8080/v1"></div>
+    </div>
 
     <div class="card-title" style="font-size:13px;margin-top:.4rem">olmOCR-2 <span class="muted" style="font-weight:400">(document VLM)</span></div>
     <p class="form-hint" style="margin:0 0 .4rem"><a href="https://huggingface.co/allenai/olmOCR-2-7B-1025" target="_blank" rel="noopener">allenai/olmOCR-2</a> (Apache-2.0) reads a whole page at once — equations as LaTeX, tables as HTML, reading order sorted — and is the one to reach for on old scans, maths and multi-column pages. It is a <b>vision model</b>, so it returns <b>no bounding boxes</b> and no layout regions: use paddle or Surya when you need those.</p>
@@ -2119,6 +2133,10 @@ async function loadSettings(){
     document.getElementById('s-ocr-surya-langs').value = ocr.surya_langs ?? 'it';
     document.getElementById('s-ocr-surya-venv').value = ocr.surya_venv ?? '';
     document.getElementById('s-ocr-surya-auto-build').checked = !!ocr.surya_auto_build;
+    document.getElementById('s-ocr-surya-serve').value = ocr.surya_serve || 'local';
+    document.getElementById('s-ocr-surya-model').value = ocr.surya_model ?? '';
+    document.getElementById('s-ocr-surya-model-id').value = ocr.surya_model_id ?? '';
+    document.getElementById('s-ocr-surya-server-url').value = ocr.surya_server_url ?? '';
     document.getElementById('s-ocr-olmocr-enabled').checked = !!ocr.olmocr_enabled;
     document.getElementById('s-ocr-olmocr-serve').value = ocr.olmocr_serve ?? 'model';
     document.getElementById('s-ocr-olmocr-instances').value = ocr.olmocr_instances ?? 1;
@@ -2367,6 +2385,10 @@ async function saveSettings(silent){
       surya_langs: document.getElementById('s-ocr-surya-langs').value.trim() || 'it',
       surya_venv: document.getElementById('s-ocr-surya-venv').value.trim(),
       surya_auto_build: document.getElementById('s-ocr-surya-auto-build').checked,
+      surya_serve: document.getElementById('s-ocr-surya-serve').value,
+      surya_model: document.getElementById('s-ocr-surya-model').value.trim(),
+      surya_model_id: document.getElementById('s-ocr-surya-model-id').value.trim(),
+      surya_server_url: document.getElementById('s-ocr-surya-server-url').value.trim(),
       olmocr_enabled: document.getElementById('s-ocr-olmocr-enabled').checked,
       olmocr_serve: document.getElementById('s-ocr-olmocr-serve').value || 'model',
       olmocr_instances: parseInt(document.getElementById('s-ocr-olmocr-instances').value) || 1,

codai/backends/runpod.py

@@ -157,6 +157,27 @@ class RunpodBackend(ModelBackend):
         self._last_pod = pod
         return url.rstrip("/") + "/v1", (lambda: self._pool.release(pod))
 
+    def openai_endpoint(self):
+        """``(base_url ending /v1, api_key)`` for a client that will speak to this
+        model over HTTP ITSELF, instead of going through us.
+
+        The OCR VLM engines work that way: Surya and olmOCR attach to an OpenAI server
+        and drive it with their own prompts, guided-decoding schemas and logprobs — all
+        things a proxy hop would quietly drop. So they get the address and the key.
+
+        Provisioning happens here (a cold pool rents and boots a pod before returning),
+        but the in-flight slot is released at once: a long-lived attachment must not
+        hold a scaling slot open, and the pod's own idle reaper decides when it goes.
+        The address is therefore a lease — the caller re-resolves when it stops
+        answering rather than treating it as permanent."""
+        base, release = self._acquire_endpoint()
+        try:
+            auth = (self._headers or {}).get("Authorization", "")
+            key = auth[7:].strip() if auth[:7].lower() == "bearer " else ""
+            return base, key
+        finally:
+            release()
+
     def _unreachable(self) -> None:
         """The pod refused the connection: drop it and let the caller retry.
 

codai/config.py

@@ -966,12 +966,19 @@ class OcrConfig:
     surya_venv: str = ""               # isolated venv dir; blank = ~/.coderai/surya_venv
     surya_auto_build: bool = False     # create the venv + pip install requirements-surya.txt on first use
     # Surya serving mode: "local" = classic det+recognition on torch in the isolated venv
-    # (surya-ocr <=0.17); "vllm"/"llamacpp" = the latest "Surya2" VLM served by an external
-    # OpenAI server that Surya attaches to (SURYA_INFERENCE_URL). "vllm" reuses coderai's
-    # vLLM backend to serve `surya_model` (continuous batching); "llamacpp" attaches to a
-    # llama.cpp server at `surya_server_url`.
-    surya_serve: str = "local"         # local | vllm | llamacpp
+    # (surya-ocr <=0.17); the rest serve the latest "Surya2" VLM from an OpenAI server that
+    # Surya attaches to (SURYA_INFERENCE_URL), which is where the work then happens:
+    #   "model"    — the VLM is a MODEL in models.json (`surya_model_id`), served however
+    #                that entry says: a RunPod pod this install rents on demand, another
+    #                server via service_url, or the local vLLM backend. The only served
+    #                mode that works with no GPU on this box — which is precisely the box
+    #                whose job is to rent one.
+    #   "vllm"     — coderai's OWN isolated vLLM venv serves `surya_model` here (CUDA).
+    #   "llamacpp" — attach to a llama.cpp/OpenAI server already running, at
+    #                `surya_server_url`. ("server" is accepted as a synonym.)
+    surya_serve: str = "local"         # local | model | vllm | llamacpp
     surya_model: str = "datalab-to/surya-ocr-2"   # HF checkpoint for the served (vllm) backend
+    surya_model_id: str = ""           # ("model" mode) the models.json id serving Surya2
     surya_server_url: str = ""         # external OpenAI server URL (llamacpp/manual); blank = auto
 
     # --- olmOCR-2 (AllenAI, Apache-2.0) --- a document VLM: whole-page transcription in
@@ -1606,6 +1613,7 @@ class ConfigManager:
                 "surya_auto_build": self.config.ocr.surya_auto_build,
                 "surya_serve": self.config.ocr.surya_serve,
                 "surya_model": self.config.ocr.surya_model,
+                "surya_model_id": self.config.ocr.surya_model_id,
                 "surya_server_url": self.config.ocr.surya_server_url,
                 "build_retry_cooldown_s": self.config.ocr.build_retry_cooldown_s,
                 "evict_drain_timeout_s": self.config.ocr.evict_drain_timeout_s,
@@ -1726,6 +1734,8 @@ class ConfigManager:
                                 ("CODERAI_OCR_OLMOCR_MODEL_ID", "olmocr_model_id"),
                                 ("CODERAI_OCR_OLMOCR_SERVER_URL", "olmocr_server_url"),
                                 ("CODERAI_OCR_SURYA_SERVE", "surya_serve"),
+                                ("CODERAI_OCR_SURYA_MODEL", "surya_model"),
+                                ("CODERAI_OCR_SURYA_MODEL_ID", "surya_model_id"),
                                 ("CODERAI_OCR_SURYA_SERVER_URL", "surya_server_url"),
                                 ("CODERAI_OCR_FALLBACK_ENGINES", "fallback_engines")):
                 _val = (_os.environ.get(_env) or "").strip()

codai/frontproxy/app.py

@@ -1180,8 +1180,8 @@ class FrontProxy:
             local.append(True)
         if getattr(o, "surya_enabled", False) and getattr(o, "surya_accept_license", False):
             mode = (getattr(o, "surya_serve", "local") or "local").strip().lower()
-            external = mode == "llamacpp" and bool(
-                (getattr(o, "surya_server_url", "") or "").strip())
+            external = mode == "model" or (mode in ("llamacpp", "server") and bool(
+                (getattr(o, "surya_server_url", "") or "").strip()))
             local.append(not external)
         if getattr(o, "olmocr_enabled", False):
             mode = (getattr(o, "olmocr_serve", "model") or "model").strip().lower()

codai/ocr/served_model.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+
+"""An OpenAI endpoint for a model coderai already serves, wherever it is serving it.
+
+Surya-2 and olmOCR are VLMs: the OCR engine is a thin client and the actual work
+happens in a model server. Until now an install could point an OCR engine at coderai's
+OWN vLLM (``ocr.*_serve = "vllm"``, an isolated venv on this box) or at a URL typed in
+by hand (``"llamacpp"``/``"server"``). Both assume the model server is somewhere the
+operator can name.
+
+On a GPU-less orchestrator neither holds. The Digesta install is the case that found
+it: ``ru002848`` has no GPU at all — coderai's whole job there is to rent A40 pods on
+demand — and the OCR VLM is already a first-class model in ``models.json`` with
+``backend: runpod``. Asking for OCR built a local vLLM venv that could never run,
+failed with ``NO_VLLM_VENV``, and the document went unread, while the pod that would
+have served it in sixty seconds was never asked for.
+
+So this module answers one question — *where is model X being served, and with what
+key* — and lets whichever backend owns the model answer it: a RunPod pod pool
+provisions one and reports its proxy URL, a model pinned to another coderai reports
+its ``service_url``, a local vLLM model reports its own instance. The OCR engine then
+attaches over HTTP exactly as it would to a hand-typed URL, and the renting, the
+scaling, the budget ledger and the idle reaper stay where they belong.
+
+The endpoint is a lease, not an address: a scale-to-zero pod is reaped when OCR goes
+quiet, and the caller is expected to re-resolve rather than to trust it forever.
+"""
+
+import threading
+
+from codai.ocr.base import OcrError
+
+_lock = threading.Lock()
+_backends: dict = {}            # model id -> a loaded backend that knows its endpoint
+
+
+def _entry_backend(model_id: str):
+    """A loaded backend for ``model_id``, or an OcrError saying why there is none.
+
+    Only the backends that END UP somewhere addressable can answer: a model loaded
+    into this process has no URL, and inventing a loopback to our own ``/v1`` would
+    hide a misconfiguration behind an extra hop.
+    """
+    from codai.models import manager as mm
+
+    try:
+        from codai.backends import remote_openai
+        if remote_openai.remote_should_handle(model_id):
+            backend = remote_openai.RemoteOpenAIBackend()
+            backend.load_model(model_id)
+            return backend
+    except OcrError:
+        raise
+    except Exception as exc:
+        raise OcrError(f"model {model_id!r}: remote endpoint unusable: {exc}", status=503)
+
+    try:
+        if mm.runpod_should_handle(model_id):
+            from codai.backends.runpod import RunpodBackend
+            backend = RunpodBackend(mm.get_active_runpod_config())
+            backend.load_model(model_id)
+            return backend
+    except Exception as exc:
+        raise OcrError(f"model {model_id!r}: RunPod endpoint unusable: {exc}", status=503)
+
+    if mm.resolve_engine_backend(model_id) == "vllm":
+        try:
+            from codai.api import runpod_worker, vllm_worker
+            from codai.models.manager import get_active_vllm_config
+            vcfg = get_active_vllm_config()
+            path = (runpod_worker._model_path_for(model_id) or "").strip() or model_id
+            base = vllm_worker.ensure_service(vcfg, model_path=path, served_name=model_id)
+            return _Fixed(base.rstrip("/") + "/v1", "", model_id)
+        except Exception as exc:
+            raise OcrError(f"model {model_id!r}: local vLLM unusable: {exc}", status=503)
+
+    raise OcrError(
+        f"model {model_id!r} is not served over HTTP by this install, so an OCR engine "
+        f"cannot attach to it. Pin the model to a backend that has an endpoint "
+        f"(backend: \"runpod\", or a service_url pointing at a server that holds it), "
+        f"or pick a serve mode that fits this box.",
+        status=503,
+    )
+
+
+class _Fixed:
+    """A backend-shaped answer for the cases that resolve to one fixed URL."""
+
+    def __init__(self, url: str, key: str, served: str):
+        self._url, self._key, self._served = url, key, served
+
+    def openai_endpoint(self):
+        return self._url, self._key
+
+    @property
+    def served_name(self) -> str:
+        return self._served
+
+
+def _served_name_of(backend, model_id: str) -> str:
+    for attr in ("served_name", "_served_name"):
+        name = getattr(backend, attr, "")
+        if isinstance(name, str) and name.strip():
+            return name.strip()
+    return model_id
+
+
+def endpoint_for(model_id: str):
+    """``(base_url ending /v1, api_key, served_name)`` for ``model_id``.
+
+    Blocks while a cold pool rents and boots a pod — a first OCR request after an idle
+    night legitimately waits for a GPU to exist. Raises :class:`OcrError` (503) with a
+    message that says what to configure, because that error is what reaches the client.
+    """
+    model_id = (model_id or "").strip()
+    if not model_id:
+        raise OcrError("no model id configured for the served OCR engine", status=400)
+
+    with _lock:
+        backend = _backends.get(model_id)
+    if backend is None:
+        backend = _entry_backend(model_id)
+        with _lock:
+            _backends.setdefault(model_id, backend)
+            backend = _backends[model_id]
+
+    try:
+        base, key = backend.openai_endpoint()
+    except OcrError:
+        raise
+    except Exception as exc:
+        raise OcrError(f"model {model_id!r}: no endpoint available ({exc})", status=503)
+    if not base:
+        raise OcrError(f"model {model_id!r}: backend reported no endpoint", status=503)
+    return base, key, _served_name_of(backend, model_id)
+
+
+def forget(model_id: str = "") -> None:
+    """Drop cached backends so the next resolve starts over (config changed, or a
+    pool that no longer answers). A pool itself is global to the process, so this
+    costs nothing but a re-``load_model``."""
+    with _lock:
+        if model_id:
+            _backends.pop(model_id.strip(), None)
+        else:
+            _backends.clear()
+
+
+def health_url(base: str) -> str:
+    """The ``/health`` base an attached client probes: the OpenAI base without ``/v1``.
+
+    Surya's attach path refuses a URL whose ``/health`` does not answer 200, which is
+    also the cheapest way for us to find out that a leased pod has gone away.
+    """
+    base = (base or "").rstrip("/")
+    return base[:-len("/v1")].rstrip("/") if base.endswith("/v1") else base

codai/ocr/surya.py

@@ -24,6 +24,12 @@ class SuryaEngine(SubprocessOcrEngine):
     name = "surya"
     _worker_engine = "surya"
 
+    # Where the recognition VLM is, what the server calls it, and the token that
+    # opens it. Empty in the local mode, and resolved by load() in the served ones.
+    _server_url = ""
+    _server_key = ""
+    _served_name = ""
+
     def _venv_dir(self) -> str:
         return _resolve_venv_dir(self.cfg.surya_venv, "surya_venv", ("surya",))
 
@@ -36,19 +42,39 @@ class SuryaEngine(SubprocessOcrEngine):
     def _worker_opts(self) -> dict:
         return {"langs": self.cfg.surya_langs or "it"}
 
+    #: Modes where the recognition VLM lives in a model server we only talk to.
+    SERVED_MODES = ("model", "vllm", "llamacpp", "server")
+
     def _serve_mode(self) -> str:
-        return (getattr(self.cfg, "surya_serve", "local") or "local").strip().lower()
+        mode = (getattr(self.cfg, "surya_serve", "local") or "local").strip().lower()
+        return "llamacpp" if mode == "server" else mode
 
     def _worker_env(self) -> dict:
         """For the served ("Surya2") modes, tell the worker to attach to an external
-        OpenAI server instead of running detection+recognition locally."""
+        OpenAI server instead of running detection+recognition locally.
+
+        Surya reads these from its own settings, so the handshake is three values and
+        no patching: which client to use, where it is, and what the server calls the
+        model — it refuses to attach to a server whose /v1/models disagrees with the
+        checkpoint it expects. The key matters as much: a pod URL is reachable by
+        anyone who learns it, so ours are launched bearer-locked, and Surya's vLLM
+        client takes that token from VLLM_API_KEY (its llama.cpp client hardcodes
+        "EMPTY", which is why a locked endpoint is always attached to as "vllm")."""
         mode = self._serve_mode()
-        if mode not in ("vllm", "llamacpp"):
+        if mode not in self.SERVED_MODES:
             return {}
         url = self._server_url
         if not url:
             return {}
-        return {"SURYA_INFERENCE_BACKEND": mode, "SURYA_INFERENCE_URL": url}
+        env = {
+            "SURYA_INFERENCE_BACKEND": "llamacpp" if mode == "llamacpp" else "vllm",
+            "SURYA_INFERENCE_URL": url,
+        }
+        if self._served_name:
+            env["SURYA_MODEL_CHECKPOINT"] = self._served_name
+        if self._server_key:
+            env["VLLM_API_KEY"] = self._server_key
+        return env
 
     def prelaunch_vram_gb(self) -> float:
         """In the served modes Surya rides a vLLM instance that grabs
@@ -83,7 +109,7 @@ class SuryaEngine(SubprocessOcrEngine):
         nil, and the instance's real cost is reported once through
         :meth:`prelaunch_vram_gb` / the vLLM stop estimate. Counting a whole GB per
         instance here asked to free 24 GB on a 24-instance pool."""
-        return 0.05 if self._serve_mode() in ("vllm", "llamacpp") else 1.0
+        return 0.05 if self._serve_mode() in self.SERVED_MODES else 1.0
 
     def load(self) -> None:
         if not self.cfg.surya_accept_license:
@@ -92,8 +118,28 @@ class SuryaEngine(SubprocessOcrEngine):
                 status=400,
             )
         self._server_url = ""
+        self._server_key = ""
+        self._served_name = ""
         mode = self._serve_mode()
-        if mode == "vllm":
+        if mode == "model":
+            # The recognition VLM is a MODEL this install already serves — on the
+            # Digesta orchestrator a RunPod A40 pod it rents on demand. Ask whichever
+            # backend owns it where it ended up, and attach there. This is the only
+            # served mode that works on a box with no GPU of its own, which is also
+            # the box whose entire job is to rent one.
+            from codai.ocr.served_model import endpoint_for
+            model_id = ((getattr(self.cfg, "surya_model_id", "") or "").strip()
+                        or (getattr(self.cfg, "surya_model", "") or "").strip())
+            if not model_id:
+                raise OcrError(
+                    "Surya 'model' mode needs ocr.surya_model_id — the id of the "
+                    "Surya-2 checkpoint in models.json (it must be served over HTTP: "
+                    "backend \"runpod\", a service_url, or the local vLLM backend).",
+                    status=400)
+            self._server_url, self._server_key, self._served_name = endpoint_for(model_id)
+            print(f"[ocr] surya attaches to {self._server_url} "
+                  f"(served as {self._served_name!r})", flush=True)
+        elif mode == "vllm":
             # Serve the Surya2 VLM checkpoint via coderai's vLLM backend (continuous
             # batching) and attach Surya to it. Requires vllm enabled/auto-buildable.
             from codai.api import vllm_worker
@@ -106,6 +152,7 @@ class SuryaEngine(SubprocessOcrEngine):
                                               gpu_memory_utilization=self._vlm_gmu(),
                                               **vlm_serve_limits(self.cfg))
             self._server_url = base.rstrip("/") + "/v1"
+            self._served_name = model
         elif mode == "llamacpp":
             url = (getattr(self.cfg, "surya_server_url", "") or "").strip()
             if not url:
@@ -114,3 +161,38 @@ class SuryaEngine(SubprocessOcrEngine):
                     "OpenAI endpoint).", status=400)
             self._server_url = url
         super().load()
+
+    # ------------------------------------------------------------------ leases
+
+    def _endpoint_alive(self) -> bool:
+        """Whether the server this worker was pinned to still answers /health."""
+        import requests
+        from codai.ocr.served_model import health_url
+        try:
+            r = requests.get(health_url(self._server_url) + "/health", timeout=5)
+            return r.status_code == 200
+        except Exception:
+            return False
+
+    def recognize_image(self, image):
+        """One page, re-resolving the endpoint if the lease expired underneath us.
+
+        In 'model' mode the address is a rented pod, and a pod that has been idle for
+        its timeout is reaped — correctly, it costs money. The worker is then pinned to
+        a URL nothing answers, and would fail every page from then on. So a failure is
+        checked against the endpoint itself: if it is still healthy the error is real
+        and is reported, and if it is gone the worker is restarted against a freshly
+        provisioned one. Exactly once — a second failure is an outage, not a lease."""
+        try:
+            return super().recognize_image(image)
+        except OcrError as first:
+            if self._serve_mode() != "model" or self._endpoint_alive():
+                raise
+            print(f"[ocr] surya endpoint {self._server_url} is gone "
+                  f"({first}) — re-resolving", flush=True)
+            from codai.ocr import served_model
+            served_model.forget()
+            self.cleanup()
+            self.load()
+            self._loaded = True
+            return super().recognize_image(image)

docs/ocr.md

@@ -63,15 +63,46 @@ matches the chosen serving mode:
 
 | `surya_serve` | What it is | surya‑ocr version | How it runs |
 |---|---|---|---|
-| `local` (default) | classic detection + recognition on torch | **≤ 0.17.1** (`requirements-surya.txt` pins 0.17.1) | in the isolated venv, GPU |
-| `vllm` | the latest **"Surya2"** VLM (Qwen3.5‑VL, `datalab-to/surya-ocr-2`) | **≥ 0.20** (e.g. 0.22.1) | served by coderai's **vLLM backend**; Surya attaches via `SURYA_INFERENCE_URL` |
-| `llamacpp` | same Surya2 VLM | ≥ 0.20 | attaches to a `llama-server` at `ocr.surya_server_url` |
-
-Surya2 (≥0.20) is a VLM that CANNOT run standalone in the venv — it requires an external
-OpenAI server (vLLM/llama‑server). `vllm` mode is recommended: coderai serves
-`ocr.surya_model` through the vLLM backend (continuous batching) and points Surya at it.
-So: install **surya‑ocr 0.17.1** for `local`, or **surya‑ocr>=0.20** for `vllm`/`llamacpp` —
-one per venv.
+| `local` (default) | classic detection + recognition on torch | **≤ 0.17.1** | in the isolated venv, on a GPU of this box |
+| `model` | the **"Surya2"** VLM as a MODEL this install serves (`ocr.surya_model_id` → `models.json`) | **≥ 0.20** (pinned 0.22.1) | wherever that entry says: a **RunPod pod rented on demand**, another host via `service_url`, or the local vLLM backend. **The only served mode that needs no GPU here.** |
+| `vllm` | the same Surya2 VLM, served from coderai's own isolated vLLM venv | ≥ 0.20 | this box, CUDA required |
+| `llamacpp` (alias `server`) | the same Surya2 VLM | ≥ 0.20 | attaches to a server you already run, at `ocr.surya_server_url` |
+
+Surya2 (≥0.20) is a VLM that CANNOT run standalone in the venv — it needs an OpenAI
+server to attach to. In the served modes the venv is therefore light: it renders the page
+and speaks HTTP, and every GPU second is spent on the other side.
+
+### `model` mode: Surya on whatever serves the model
+
+Surya is a *client* — layout, reading order, guided-JSON decoding and logprobs are its
+own, which is why it drives a model server directly instead of going back through
+coderai's `/v1` (one proxy hop would drop the guided decoding and the logprobs with it).
+What it needs from coderai is therefore an address, and `model` mode is where it asks for
+one in the vocabulary the rest of the server already speaks:
+
+```jsonc
+// models.json — the VLM is an ordinary model entry, served like any other
+"datalab-to/surya-ocr-2": { "backend": "runpod", "engine": "coderai-vllm", … }
+```
+```jsonc
+// config.json
+"ocr": { "surya_serve": "model", "surya_model_id": "datalab-to/surya-ocr-2" }
+```
+
+coderai resolves that id through the backend that owns it — provisioning a pod if the
+pool is cold — and hands the worker `SURYA_INFERENCE_URL`, `SURYA_MODEL_CHECKPOINT` (the
+served name; Surya refuses a server whose `/v1/models` disagrees) and `VLLM_API_KEY` (pods
+are bearer-locked). Renting, autoscaling, the spend ledger and the idle reaper stay with
+the model layer; the OCR side holds only a lease. When a scale-to-zero pod is reaped
+between documents, the next page finds the endpoint unhealthy, re-resolves once — which
+rents a fresh pod — and is read rather than lost.
+
+This is the mode for a **GPU-less orchestrator**: an install whose job is to rent cards
+has no card of its own, so `vllm` mode there can only ever fail (`NO_VLLM_VENV`), and a
+URL pasted into `llamacpp` mode stops existing fifteen minutes after the last page.
+
+So: install **surya‑ocr 0.17.1** for `local`, or **surya‑ocr>=0.20** for
+`model`/`vllm`/`llamacpp` — one per venv.
 Or set `paddle_auto_build` / `surya_auto_build` to have coderai create the venv and install
 on first use. In the OCI image these venvs are **baked in** at `/opt/coderai/paddle_venv` /
 `/opt/coderai/surya_venv` (same as the `lipsync_venv` / `parler-venv`), so the image is

requirements-surya.txt

@@ -21,7 +21,13 @@
 # det+recognition locally on torch (GPU), so it's the classic pin. See docs/ocr.md.
 surya-ocr==0.22.1
 pypdfium2>=4.20.0
-# NOTE: 0.17.1 is for ocr.surya_serve="local" (classic det+recognition on torch, runs in
-# THIS venv). For ocr.surya_serve="vllm"/"llamacpp" (the latest "Surya2" VLM, Qwen3.5-VL
-# `datalab-to/surya-ocr-2`), install surya-ocr>=0.20 instead (e.g. 0.22.1) — it needs an
-# external vLLM/llama-server backend, not local torch. One version per venv; see docs/ocr.md.
+# NOTE: ONE VERSION PER VENV, and the mode decides which.
+#   ocr.surya_serve="local"  -> surya-ocr==0.17.1, the last release that runs
+#                               det+recognition locally on torch (needs a GPU here).
+#   ocr.surya_serve="model" / "vllm" / "llamacpp"  -> surya-ocr>=0.20 (pinned 0.22.1
+#                               above): the "Surya2" VLM (`datalab-to/surya-ocr-2`),
+#                               which this venv only TALKS to over HTTP. That makes the
+#                               venv light — PIL, httpx and an OpenAI client do the work
+#                               — and is what lets a GPU-less install serve Surya at all
+#                               ("model" mode: the VLM is a models.json entry, e.g. a
+#                               RunPod pod rented on demand). See docs/ocr.md.