a control plane that rented pods to deliver a teardown (v0.3.87)

Seven items from a production field report. Three of them were one cause.

/v1/runpod is NOT in the remote gateway's local list. It carries a `model`
field like an inference request, so the gateway read it as one, resolved the
model to its pod, and FORWARDED the control-plane call there. Asking for six
pods back rented a seventh to ask on; the pod has no runpod configuration of
its own, so it answered

    404 {"detail": "'datalab-to/surya-ocr-2' has no runpod block"}

which reads exactly like the orchestrator refusing a name it should know. The
same call blocked for minutes while a pod booted to receive it. An operator
could not turn off $3.54/hr from the API. Reproduced here while testing the
fix: two teardown requests rented two A40s, which is the clearest possible
statement of the bug.

surya answered 200 with empty pages for a whole document. Three failures in a
row, each of which had to be fixed:

  * its client catches the HTTP error itself and returns a generation with no
    tokens, so the lease check in recognize_image — which only ran in `except`
    — never ran, and the pod address was never re-resolved;
  * the repetition detector was handed that empty string, found no progress in
    it and re-ran the page at a rising temperature, four full paid generations
    per page, `tokens: [0,0,0,0]`;
  * and the page came back as read. We had already measured the ink and said
    there IS something on it; the judgement never reached the response, so a
    client treating an empty read as an unreadable scan consumed roughly a
    thousand documents from its queue and wrote them off.

A page with ink and no text is now `unread`: it goes to the fallback engine
like any other degraded page, it is listed in `unread_pages`, and a document
whose every inked page came back empty is no longer a 200 (`ocr.unread_status`,
502 by default; a partially read document stays 200 with the list).

`bge-m3` and `BAAI/bge-m3` each rented their own A40 — the pool key was
whichever name the request carried. Keyed on the model ENTRY now, so every name
for one set of weights lands on one pool.

A client read timeout cannot bound an OCR call: it times each READ, so a
response that dribbles never trips it and one stayed open nineteen minutes
against timeout=420. `ocr.request_timeout_s` is the server-side stopwatch.

Chunking: `it-legal` accepts a decorated heading (`**DISPOSITIVO**`) where the
implementation it ports does not, which splits one more section and leaves a
short trailing chunk. That was true and undocumented — the table printed the
strict pattern. Documented now, with `it-legal-strict` named as the
byte-identical one for re-chunking an existing index. `min_chunk_chars` folds a
runt chunk into its predecessor, off by default because merging moves a
boundary, and never silently: the absorbed section label is reported, since
`conclusioni` is what a party asked for and `dispositivo` is what the court
decided.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.86"
+__version__ = "0.3.87"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/chunking.py

@@ -467,12 +467,73 @@ def extract_refs(text: str) -> List[str]:
 # ---------------------------------------------------------------------------
 # The whole operation
 # ---------------------------------------------------------------------------
+def _absorb_runts(chunks, minimum: int, want_refs: bool) -> list:
+    """Fold a chunk shorter than ``minimum`` into the one before it.
+
+    A `P.Q.M.` is often two lines. As its own chunk it is a 162-character entry
+    in a retrieval index — too short to carry meaning on its own and long enough
+    to match, which is the worst combination. Asked for by an operator whose
+    index was filling with them.
+
+    OFF by default, and it must stay off by default: merging moves a boundary,
+    and a re-chunked corpus whose boundaries moved cannot be compared with the
+    index it replaces. Neither this implementation nor the one it ports does
+    this unless asked.
+
+    What it does NOT do is hide the join. The absorbed chunk's label is recorded
+    on the survivor (`absorbed_sections`) and in the response, because section
+    labels carry meaning that must not be quietly rewritten: `conclusioni` is
+    what a party ASKED FOR and `dispositivo` is what the court DECIDED, and an
+    index that confuses them surfaces a lawyer's demand as a holding. A reader
+    of a merged chunk can always see which labels went into it.
+
+    The FIRST chunk is never absorbed — there is nothing before it.
+    """
+    try:
+        minimum = int(minimum or 0)
+    except (TypeError, ValueError):
+        minimum = 0
+    if minimum <= 0 or len(chunks) < 2:
+        return []
+    out = []
+    absorbed = []
+    for obj in chunks:
+        if out and len(obj["text"]) < minimum:
+            prev = out[-1]
+            absorbed.append({"into": prev["ord"],
+                             "section": obj.get("section"),
+                             "n_chars": obj["n_chars"]})
+            prev["text"] = prev["text"] + "\n\n" + obj["text"]
+            prev["n_chars"] = len(prev["text"])
+            if obj.get("section") and obj.get("section") != prev.get("section"):
+                labels = prev.setdefault("absorbed_sections", [])
+                if obj["section"] not in labels:
+                    labels.append(obj["section"])
+            if want_refs:
+                merged = list(prev.get("norm_refs") or [])
+                for ref in obj.get("norm_refs") or []:
+                    if ref not in merged:
+                        merged.append(ref)
+                prev["norm_refs"] = merged
+            if "end" in obj and "start" in prev:
+                prev["end"] = obj["end"]
+            continue
+        out.append(obj)
+    if not absorbed:
+        return []
+    for i, obj in enumerate(out):
+        obj["ord"] = i
+    chunks[:] = out
+    return absorbed
+
+
 def chunk_document(text: str, *, chunk_tokens: int = DEFAULT_CHUNK_TOKENS,
                    overlap_tokens: int = DEFAULT_OVERLAP_TOKENS,
                    chars_per_token: int = DEFAULT_CHARS_PER_TOKEN,
                    structural: bool = True, structure: Any = "it-legal",
                    refs: Any = "it-legal", include_offsets: bool = False,
                    require_text: bool = True,
+                   min_chunk_chars: int = 0,
                    registered: Optional[dict] = None) -> dict:
     """Chunk `text` and return the response document (see docs/chunking.md)."""
     if not isinstance(text, str):
@@ -523,6 +584,8 @@ def chunk_document(text: str, *, chunk_tokens: int = DEFAULT_CHUNK_TOKENS,
                     cursor = at
             chunks.append(obj)
 
+    absorbed = _absorb_runts(chunks, min_chunk_chars, want_refs)
+
     return {
         "chunks": chunks,
         "n_chunks": len(chunks),
@@ -531,7 +594,9 @@ def chunk_document(text: str, *, chunk_tokens: int = DEFAULT_CHUNK_TOKENS,
         "structural_applied": bool(applied),
         "params": {"chunk_tokens": int(chunk_tokens),
                    "overlap_tokens": int(overlap_tokens),
-                   "chars_per_token": int(chars_per_token)},
+                   "chars_per_token": int(chars_per_token),
+                   "min_chunk_chars": int(min_chunk_chars or 0)},
+        **({"absorbed": absorbed} if absorbed else {}),
     }
 
 
@@ -608,6 +673,14 @@ class ChunkRequest(BaseModel):
     extract_refs: Optional[Any] = Field(None, description="Reference extractor: a name, an inline object, or false to skip.")
     require_text: Optional[bool] = Field(None, description="Reject binary input instead of chunking container bytes.")
     include_offsets: Optional[bool] = Field(None, description="Add char offsets into the normalised source text.")
+    min_chunk_chars: Optional[int] = Field(
+        None, ge=0,
+        description="Fold a chunk shorter than this into the one before it. 0 "
+                    "(the default) keeps every chunk, which is what the "
+                    "reference implementation does: merging MOVES a boundary, "
+                    "and a corpus re-chunked with moved boundaries cannot be "
+                    "compared with the index it replaces. The labels that were "
+                    "absorbed are reported, never silently dropped.")
     model_config = ConfigDict(extra="allow")
 
 
@@ -631,6 +704,8 @@ def _resolve_params(text: str, body) -> dict:
         include_offsets=bool(body.include_offsets),
         require_text=(body.require_text if body.require_text is not None
                       else _bool_setting("require_text", "CODERAI_CHUNK_REQUIRE_TEXT", True)),
+        min_chunk_chars=(body.min_chunk_chars if body.min_chunk_chars is not None
+                         else g("min_chunk_chars", "CODERAI_CHUNK_MIN_CHARS", 0)),
         registered=_registered_templates(),
     )
 
@@ -687,6 +762,7 @@ async def create_chunks_from_file(
     extract_refs: Optional[str] = Form(None),
     require_text: Optional[bool] = Form(None),
     include_offsets: Optional[bool] = Form(None),
+    min_chunk_chars: Optional[int] = Form(None),
 ):
     """The multipart form of `/v1/chunk`, with the same field names as values --
     matching how `/v1/ocr` takes its options.
@@ -704,7 +780,7 @@ async def create_chunks_from_file(
         chunk_tokens=chunk_tokens, overlap_tokens=overlap_tokens,
         chars_per_token=chars_per_token, structural=structural,
         structure=structure, require_text=require_text,
-        include_offsets=include_offsets,
+        include_offsets=include_offsets, min_chunk_chars=min_chunk_chars,
         extract_refs=(False if (extract_refs or "").strip().lower()
                       in ("0", "false", "no", "off") else (extract_refs or None)))
     return _run(_resolve_params(raw.decode("utf-8", "ignore"), body))

codai/api/ocr.py

@@ -29,6 +29,7 @@ import asyncio
 from typing import List, Optional
 
 from fastapi import APIRouter, HTTPException, UploadFile, File, Form
+from fastapi.responses import JSONResponse
 from pydantic import BaseModel, ConfigDict, Field
 
 from codai.ocr.base import OcrError
@@ -76,6 +77,64 @@ def _page_opts(max_tokens, max_continuations) -> dict:
     return out
 
 
+async def _bounded(cfg, coro):
+    """Run an OCR read under the install's wall-clock ceiling, if it set one.
+
+    A client's read timeout cannot bound this. httpx (and requests) time each
+    READ, so a response that dribbles a byte at a time never trips one: a
+    chandra read stayed open for nineteen minutes against a `timeout=420`.
+    Bounding it needs a stopwatch over the whole call, and the server is the
+    side that knows what it is doing.
+    """
+    try:
+        limit = float(getattr(cfg.ocr, "request_timeout_s", 0) or 0)
+    except Exception:
+        limit = 0.0
+    if limit <= 0:
+        return await coro
+    try:
+        return await asyncio.wait_for(coro, timeout=limit)
+    except asyncio.TimeoutError:
+        raise OcrError(f"the read passed the {limit:g}s ceiling for one request "
+                       f"(ocr.request_timeout_s) and was stopped", status=504)
+
+
+def _answer(result: dict, cfg):
+    """The OCR response, with a status code that matches what we found.
+
+    A document whose every readable page came back empty must NOT be a 200. The
+    orchestrator has already measured ink on those pages and said there is
+    something on them, so "no text" is this run failing, not the document being
+    blank — but the two are identical in a 200 whose `text` is empty, and a
+    client that treats an empty read as an unreadable scan then consumes the
+    document from its queue and writes it off. That happened to roughly a
+    thousand documents in one night behind a lost engine lease.
+
+    Only the all-or-nothing case changes status: a document that produced SOME
+    text is still a 200, with `unread_pages` naming the ones it did not. Set
+    `ocr.unread_status` to 0 to keep the old behaviour.
+    """
+    try:
+        status = int(getattr(cfg.ocr, "unread_status", 502) or 0)
+    except Exception:
+        status = 502
+    unread = result.get("unread_pages") or []
+    if not status or not unread:
+        return result
+    if (result.get("text") or "").strip():
+        return result                   # partially read: 200 and a named list
+    return JSONResponse(status_code=status, content={
+        "error": {
+            "message": (f"every page with ink on it came back empty "
+                        f"({len(unread)} of {result.get('num_pages', 0)}): the "
+                        f"document was not read. This is not a blank document "
+                        f"— see unread_pages."),
+            "type": "ocr_unread",
+        },
+        **result,
+    })
+
+
 @router.post("/v1/ocr", summary="OCR an image or PDF with a dedicated OCR engine")
 async def create_ocr(
     file: UploadFile = File(...),
@@ -100,7 +159,7 @@ async def create_ocr(
         raise HTTPException(status_code=413, detail="file too large (max 100 MB)")
 
     try:
-        return await ocr_manager.ocr_document(
+        result = await _bounded(cfg, ocr_manager.ocr_document(
             data,
             filename=file.filename or "",
             content_type=file.content_type or "",
@@ -111,7 +170,8 @@ async def create_ocr(
             schema=schema_,
             lang=lang,
             page_opts=_page_opts(max_tokens, max_continuations),
-        )
+        ))
+        return _answer(result, cfg)
     except OcrError as e:
         raise HTTPException(status_code=e.status, detail=str(e))
     except Exception as e:

codai/api/remote_gateway.py

@@ -135,7 +135,20 @@ _SKIP_PATHS = ("/v1/chat/completions", "/v1/completions", "/v1/models",
 
 #: Paths that are this install's own, matched by prefix at run time (the literal
 #: route templates above are what the coverage test reads).
-_LOCAL_PREFIXES = ("/v1/files/upload", "/v1/files/blob")
+#:
+#: ``/v1/runpod`` is THIS install's control plane: it rents, reports and
+#: releases the pods. It carries a ``model`` field like an inference request, so
+#: the gateway read it as one, resolved the model to its pod — and FORWARDED the
+#: scale request to a pod, renting one in order to deliver it. The pod has no
+#: runpod configuration of its own, so it answered
+#:
+#:     404 {"detail": "'datalab-to/surya-ocr-2' has no runpod block"}
+#:
+#: which reads exactly like the orchestrator refusing a name it should know.
+#: Teardown was the worst case: asking to give six pods back rented a seventh to
+#: ask it on, blocked for minutes hunting capacity, and then reported that the
+#: model does not exist. An operator could not turn off $3.54/hr from the API.
+_LOCAL_PREFIXES = ("/v1/files/upload", "/v1/files/blob", "/v1/runpod")
 
 #: Orchestration endpoints: a sequence of calls to other endpoints, not a model.
 #: They are NEVER forwarded as a whole — the chain stays here and each step goes

codai/api/runpod_worker.py

@@ -4297,6 +4297,32 @@ _scaler_started = False
 _SINGLE_MODEL_ENGINES = ("vllm", "llamacpp")
 
 
+def canonical_model_key(model_key: str, entry: dict = None) -> str:
+    """ONE pool key per set of weights, whatever name the caller used.
+
+    A model answers to several names — its id, its HuggingFace path, any alias —
+    and the pool key was simply whichever one the request happened to carry. So
+    `bge-m3` and `BAAI/bge-m3` built two pools over the same weights and rented
+    two A40s, one of them pure waste at $0.59/hr, with /v1/models listing both
+    names and nothing anywhere saying they are the same thing.
+
+    Keyed on the ENTRY, so every name for one model lands on one pool. A model
+    with no entry keeps its name: there is nothing else to key on, and refusing
+    to pool it would be worse than pooling it under the name given.
+    """
+    try:
+        if entry is None:
+            from codai.models.manager import _model_entry_for
+            entry = _model_entry_for(model_key) or {}
+    except Exception:
+        entry = {}
+    if not isinstance(entry, dict):
+        entry = {}
+    return (str(entry.get("id") or "").strip()
+            or str(entry.get("path") or "").strip()
+            or str(model_key))
+
+
 def shared_pool_key(mcfg: "RunpodModelConfig", model_key: str,
                     model_path: str = "") -> str:
     """The pool key for this model: its own, or a shared one it opted into.
@@ -4307,7 +4333,7 @@ def shared_pool_key(mcfg: "RunpodModelConfig", model_key: str,
     """
     shared = (getattr(mcfg, "pool", "") or "").strip().lower()
     if not shared:
-        return str(model_key)
+        return canonical_model_key(model_key)
     engine = resolve_pod_engine(mcfg, model_key, model_path)
     if engine in _SINGLE_MODEL_ENGINES:
         raise RuntimeError(
@@ -4726,7 +4752,15 @@ def get_live_pool(key: str):
     if not k:
         return None
     with _pools_lock:
-        return _pools.get(k)
+        got = _pools.get(k)
+        if got is not None:
+            return got
+        # An ALIAS of a pooled model. `status` reports the canonical key, but a
+        # client holds whichever name it has been using all along, and being
+        # told a running pool does not exist is how a pod ends up with no way
+        # to turn it off.
+        alt = canonical_model_key(k)
+        return _pools.get(alt) if alt != k else None
 
 
 def _scale_live_pool(pool, key: str, pods: int, teardown: bool = False,

codai/api/vllm_worker.py

@@ -269,7 +269,15 @@ def _served_names(model_config: dict, served_name: str, fallback: str) -> list:
     adapters = {str(s.get("name") or "") for s in _sibling_lora_specs(model_config or {})}
     names = [str(served_name or "")]
     base = _base_of(model_config or {})
-    if base:
+    if adapters and base:
+        # Several names over ONE engine. Which of them asked for it must not
+        # decide what the engine answers to — order-dependence is the bug this
+        # is being changed for — so the full set is collected every time: the
+        # weights' own name, and every name the catalogue gives this base.
+        #
+        # Only when there ARE adapters. A model served alone is named exactly as
+        # it was before, because nothing about it is ambiguous.
+        names.append(str(fallback or ""))
         try:
             from codai.admin.routes import config_manager
             md = getattr(config_manager, "models_data", None) or {}
@@ -278,7 +286,8 @@ def _served_names(model_config: dict, served_name: str, fallback: str) -> list:
         for entry in (md.get("text_models") or []):
             if not isinstance(entry, dict) or _base_of(entry) != base:
                 continue
-            names.append(str(entry.get("alias") or entry.get("id") or "").strip())
+            names.append(str(entry.get("alias") or "").strip())
+            names.append(str(entry.get("id") or "").strip())
     out, seen = [], set()
     for n in names:
         if n and n not in seen and n not in adapters:

codai/config.py

@@ -1108,6 +1108,18 @@ class OcrConfig:
     # ...and only in a document with at least this many pages, since a median
     # taken over two pages is not a median.
     retry_short_pages_min_doc: int = 4
+    # The status a document gets when EVERY page with ink on it came back
+    # empty. 200 with no text is indistinguishable from a blank document, and a
+    # client that treats an empty read as an unreadable scan then writes the
+    # document off; this says plainly that the read did not happen. 0 keeps the
+    # old 200. A partially read document is always 200, with `unread_pages`.
+    unread_status: int = 502
+    # Wall-clock ceiling for ONE /v1/ocr request, 0 = no limit. A read that
+    # dribbles bytes never trips a client's READ timeout -- that timeout is
+    # per-read, so a response trickling for nineteen minutes looks healthy to
+    # httpx the whole way. A client cannot bound the call without a stopwatch
+    # of its own, so the bound belongs here as well.
+    request_timeout_s: float = 0.0
 
     # --- PaddleOCR / PP-Structure ---
     paddle_enabled: bool = True
@@ -2064,6 +2076,8 @@ class ConfigManager:
                 "fallback_engines": self.config.ocr.fallback_engines,
                 "retry_degraded": self.config.ocr.retry_degraded,
                 "retry_degraded_max_pages": self.config.ocr.retry_degraded_max_pages,
+                "unread_status": self.config.ocr.unread_status,
+                "request_timeout_s": self.config.ocr.request_timeout_s,
                 "retry_short_pages_below": self.config.ocr.retry_short_pages_below,
                 "retry_short_pages_min_doc": self.config.ocr.retry_short_pages_min_doc,
                 "olmocr_enabled": self.config.ocr.olmocr_enabled,

codai/ocr/manager.py

@@ -474,6 +474,30 @@ def _drain(stream, n: int) -> list:
     return out
 
 
+def _unread_pages(pages) -> list:
+    """Pages with ink on them and no text: read attempts that did not happen.
+
+    Deliberately NOT the same list as `blank_pages`. A blank page is a fact
+    about the document; an unread page is a fact about this run, and only one
+    of the two is worth retrying.
+    """
+    out = []
+    for page in pages or []:
+        meta = getattr(page, "meta", None) or {}
+        if (getattr(page, "text", "") or "").strip():
+            continue
+        why = ""
+        if meta.get("engine_empty"):
+            why = "the engine returned no tokens for this page"
+        else:
+            ink = meta.get("page_ink")
+            if isinstance(ink, dict) and ink.get("blank") is False:
+                why = str(ink.get("why") or "there is ink on this page")
+        if why:
+            out.append({"page": getattr(page, "index", len(out)), "why": why})
+    return out
+
+
 def _mark_blank_pages(cfg, images, pages) -> list:
     """Attach an ink measurement to any page that came back with little text.
 
@@ -1288,6 +1312,16 @@ class OcrManager:
         report = unreadable_report(pages)
         if report:
             out["unreadable"] = report
+        # Pages this install itself judged readable and then produced no text
+        # for. A caller cannot tell that from a blank page by looking at the
+        # text, and the two lead to opposite places: one is a document to skip,
+        # the other is a read to repeat. We already computed the judgement --
+        # `page_ink` says there is ink on the page -- it simply never reached
+        # the response, so ~1,000 documents were consumed from a work queue and
+        # written off as unreadable scans after a lost engine lease.
+        unread = _unread_pages(pages)
+        if unread:
+            out["unread_pages"] = unread
         if recovered:
             # Top level, because a caller deciding whether to trust this
             # document should not have to walk the pages to find out that part
@@ -1326,6 +1360,19 @@ class OcrManager:
         meta = getattr(page, "meta", None) or {}
         if meta.get("repeat_detected"):
             return "looping"
+        # Nothing came back from the model at all. Not the same as a page the
+        # model read and found empty: this page was never read, so another
+        # engine is exactly the right thing to try. A lost engine lease answers
+        # like this, and so does a pod taken away mid-batch.
+        if meta.get("engine_empty"):
+            return "unread"
+        ink = meta.get("page_ink")
+        if (isinstance(ink, dict) and ink.get("blank") is False
+                and not (getattr(page, "text", "") or "").strip()):
+            # The orchestrator already measured ink on this page and said there
+            # IS something on it. Answering 200 with no text for a page we
+            # ourselves judged readable is the silent-wrong-answer case.
+            return "unread"
         info = meta.get("model_attempts") or {}
         if isinstance(info, dict) and info.get("truncated"):
             return "truncated"

codai/ocr/surya.py

@@ -317,22 +317,40 @@ class SuryaEngine(SubprocessOcrEngine):
         a URL nothing answers, and would fail every page from then on. So a failure is
         checked against the endpoint itself: if it is still healthy the error is real
         and is reported, and if it is gone the worker is restarted against a freshly
-        provisioned one. Exactly once — a second failure is an outage, not a lease."""
+        provisioned one. Exactly once — a second failure is an outage, not a lease.
+
+        A failure is not always an exception. surya's client catches the HTTP
+        error itself ("Inference error: Error code: 404 …") and returns a
+        generation with no tokens, so a page can come back EMPTY with nothing
+        raised — and the check below, which used to run only in `except`, never
+        ran. Observed live: a front restart voided the lease, every page
+        answered 404, and the document came back 200 with no text and no error
+        anywhere in it. An empty page is therefore checked the same way a raised
+        one is; the probe only costs anything on a page that already produced
+        nothing.
+        """
+        page = None
         try:
-            return super().recognize_image(image)
+            page = super().recognize_image(image)
         except OcrError as first:
             if self._serve_mode() != "model" or self._endpoint_alive():
                 raise
-            print(f"[ocr] surya endpoint {self._server_url} is gone "
-                  f"({first}) — re-resolving", flush=True)
-            from codai.models import placement
-            from codai.ocr import served_model
-            served_model.forget()
-            # The candidate set is discovered, so a vanished pod or node must be
-            # re-discovered too; otherwise the re-resolve hands back the same
-            # dead address it just rejected.
-            placement.forget()
-            self.cleanup()
-            self.load()
-            self._loaded = True
-            return super().recognize_image(image)
+            why = str(first)
+        else:
+            if self._serve_mode() != "model" or (page.text or "").strip() \
+                    or self._endpoint_alive():
+                return page
+            why = "the engine returned no text and its lease is gone"
+        print(f"[ocr] surya endpoint {self._server_url} is gone "
+              f"({why}) — re-resolving", flush=True)
+        from codai.models import placement
+        from codai.ocr import served_model
+        served_model.forget()
+        # The candidate set is discovered, so a vanished pod or node must be
+        # re-discovered too; otherwise the re-resolve hands back the same
+        # dead address it just rejected.
+        placement.forget()
+        self.cleanup()
+        self.load()
+        self._loaded = True
+        return super().recognize_image(image)

codai/ocr/workers/ocr_worker.py

@@ -347,6 +347,19 @@ def _note_loop(raw: str) -> None:
     st["loop_chars"] = len(raw or "")
 
 
+def _note_empty() -> None:
+    """Record that a call to the model came back with nothing at all.
+
+    Not the same as a page the model could not read: NOTHING came back, so the
+    model was never asked successfully. Seen live when a front restart voided an
+    OCR lease — every call answered 404 "unknown OCR engine lease", surya's
+    client swallowed it and returned an empty generation, and the page was
+    reported as read.
+    """
+    st = _state()
+    st["empty_replies"] = int(st.get("empty_replies") or 0) + 1
+
+
 def _page_override(name: str, default):
     """A per-request value for the page this THREAD is reading, else the default.
 
@@ -1400,6 +1413,21 @@ class SuryaWorker:
                 budget = _RETRY_BUDGET[0]
                 if budget is not None:
                     max_retries = min(int(max_retries or 0), int(budget))
+                # A reply with NO tokens did not repeat itself — it never
+                # happened. The detector is handed an empty string, finds no
+                # progress in it and asks for another go at a higher
+                # temperature; the temperature cannot help a call that failed.
+                # Live, with a dead lease behind it, that was four full paid
+                # generations per page, `tokens: [0,0,0,0]`, and a document
+                # returned as successfully read with no text in it.
+                if not int(getattr(result, "token_count", 0) or 0) \
+                        and not (getattr(result, "raw", "") or "").strip():
+                    _note_empty()
+                    print("[surya] the model returned nothing for this page — "
+                          "not a repetition, so it is NOT re-run at a higher "
+                          "temperature; the page is reported unread",
+                          file=sys.stderr, flush=True)
+                    return False
                 again = inner_retry(result, retries, max_retries, *a, **kw)
                 if again:
                     print(f"[surya] page repeated itself — retry "
@@ -1551,6 +1579,18 @@ class SuryaWorker:
         if _state().get("looping"):
             page["repeat_detected"] = True
             page["repeat_chars"] = int(_state().get("loop_chars") or 0)
+        # Every call for this page came back empty. The page was never read, and
+        # that is a different thing from a page that was read and found blank —
+        # one is ours to fix, the other is the document's.
+        _empty = int(_state().get("empty_replies") or 0)
+        if _empty:
+            page["engine_empty"] = {
+                "calls": _empty,
+                "why": ("the model returned no tokens at all; the page was not "
+                        "read. A transport failure answers like this — a lost "
+                        "engine lease, a pod taken away mid-batch — and so does "
+                        "a refused request."),
+            }
         tries = _attempts()
         toks = _attempt_token_counts()
         cap = int(_page_override("max_tokens", _MAX_TOKENS[0])

docs/chunking.md

@@ -167,14 +167,30 @@ beats registered; an unknown name is an error, never a silent fallback.
 ### Built-in `it-legal`
 
 Carries both families, because party filings outnumber rulings in the merito
-corpus. Patterns, verbatim from the production implementation:
+corpus.
+
+**`it-legal` is deliberately NOT byte-identical to the original.** Its
+single-word anchors accept a DECORATED heading — `**P.Q.M.**`, `_DISPOSITIVO_`,
+`## FATTO` — because OCR engines that emit markdown (chandra, olmOCR) produce
+exactly those, and the original's `^\s*dispositivo\s*$` does not match one. The
+difference is `[\s*#>_]*` where the original has `\s*`, on the three anchored
+patterns (`premess[ae]`, `fatto`, `dispositivo`).
+
+That difference is VISIBLE in the output: a decorated `**P.Q.M.**` becomes its
+own `dispositivo` section here and stays inside `motivazione` there — one extra
+chunk, usually a short one, and the preceding chunk shorter by the same text.
+If you are re-chunking a corpus indexed with the original, use
+[`it-legal-strict`](#built-in-it-legal-strict), which IS byte-identical; a
+boundary that moves makes the new index incomparable with the old.
+
+Patterns:
 
 | Label | Pattern |
 |---|---|
-| `premessa` | `tanto\s+premesso\|ci[oò]\s+premesso\|^\s*[*#>_ ]*premesso\s+che\b\|^\s*premess[ae]\s*$` |
-| `fatto` | `svolgimento\s+del\s+processo\|ragioni\s+di\s+fatto\|in\s+fatto\b\|fatto\s+e\s+diritto\|^\s*fatto\s*$\|esposizione\s+del\s+fatto` |
+| `premessa` | `tanto\s+premesso\|ci[oò]\s+premesso\|^\s*[*#>_ ]*premesso\s+che\b\|^[\s*#>_]*premess[ae][\s*#>_]*$` |
+| `fatto` | `svolgimento\s+del\s+processo\|ragioni\s+di\s+fatto\|in\s+fatto\b\|fatto\s+e\s+diritto\|^[\s*#>_]*fatto[\s*#>_]*$\|esposizione\s+del\s+fatto` |
 | `motivazione` | `motivi\s+della\s+decisione\|ragioni\s+della\s+decisione\|motivazione\|considerato\s+in\s+diritto\|in\s+diritto\b\|motivi\s+in\s+diritto\|ragioni\s+in\s+diritto` |
-| `dispositivo` | `p\.?\s*q\.?\s*m\.?\|p\.?\s*t\.?\s*m\.?\|per\s+questi\s+motivi\|^\s*dispositivo\s*$\|così\s+deciso\|così\s+provvede` |
+| `dispositivo` | `p\.?\s*q\.?\s*m\.?\|p\.?\s*t\.?\s*m\.?\|per\s+questi\s+motivi\|^[\s*#>_]*dispositivo[\s*#>_]*$\|così\s+deciso\|così\s+provvede` |
 | `conclusioni` | `^\s*[*#>_ ]*conclusion[ei]\b\|\bconclusioni\s*:\|voglia\s+(?:l['’]ill\.?mo\s+)?(?:il\s+)?(?:tribunale\|giudice\|giudicante)\|^\s*[*#>_ ]*chiede\b\|salvis\s+iuribus` |
 
 **`conclusioni` is a label of its own and must never be folded into
@@ -266,7 +282,7 @@ extracts article and act citations, normalises them, and de-duplicates while
   split across two keys depending on how the source wrote it. `del` separates
   as well as `/` (`legge 241 del 1990`), and the bare `l` needs its dot.
 
-### `it-legal-strict`
+### Built-in `it-legal-strict`
 
 `it-legal` widens the line-anchored headers to tolerate markdown emphasis, so
 `**FATTO**` opens a section. The implementation this ports does not: its

tests/test_chunking.py

@@ -529,7 +529,7 @@ def test_the_params_are_echoed_so_a_caller_can_tell_what_applied():
     r = ch.chunk_document(RULING, chunk_tokens=100, overlap_tokens=10,
                           chars_per_token=5)
     assert r["params"] == {"chunk_tokens": 100, "overlap_tokens": 10,
-                           "chars_per_token": 5}
+                           "chars_per_token": 5, "min_chunk_chars": 0}
 
 
 # ------------------------------------------- forwarding it to a remote #

tests/test_shared_pool_adapter_identity.py

@@ -281,3 +281,11 @@ def test_an_unanswered_probe_never_costs_a_relaunch(monkeypatch):
     monkeypatch.setattr(vw, "_serves_name", lambda url, name: None)
     assert vw._ensure_service_once(_Cfg(), BASE_REPO, "tuned-alpha") == "http://127.0.0.1:9"
     assert stopped == []
+
+
+def test_the_engine_answers_the_same_set_whichever_name_started_it(catalogue):
+    """Order-dependence is the bug; the launch must not inherit it."""
+    by_base = vw._served_names(dict(BASE), "base-awq", BASE_REPO)
+    by_adapter = vw._served_names(dict(BASE), "tuned-alpha", BASE_REPO)
+    assert set(by_base) == set(by_adapter)
+    assert BASE_REPO in by_base