ocr: chandra and olmOCR report a measured confidence, and vLLM pods need g++ (v0.3.30)

CONFIDENCE. chandra hard-coded `conf=0.0` and olmOCR reported nothing at all.
chandra's code even argued for it: "the model reports none, and inventing one
would be read as a measurement". The first half was right and the conclusion was
wrong -- the number was obtainable all along, by the route surya already uses.
Both are served over an OpenAI chat endpoint, so the request now asks for
logprobs and the reply's mean exp(logprob) is the confidence. It is the same
computation surya's own client does, so the three VLM figures are comparable
with each other instead of being three different notions of "confidence".

    engine    scope   source
    paddle    line    recogniser score
    doctr     line    recogniser score
    surya     block   mean token probability
    chandra   page    mean token probability
    olmocr    page    mean token probability

The SCOPE is reported too (`/v1/ocr/engines` and `meta.confidence_scope`) and it
is not cosmetic: a per-line score can single out the line to re-read, a
page-level one can only say the page went badly. surya calls per layout block so
its figure is genuinely per-block; chandra and olmOCR generate a whole page in
one call, so chandra puts the one page figure on every line and region (which is
what `unreadable`'s per-spot conf reads) and says the scope is "page" rather than
letting it pass for per-line precision.

A server may decline logprobs, and then conf is 0.0 -- which must NOT read as
"the model was hopeless". `meta.confidence_measured` separates the two. That is
the same distinction the unreadable-glyph trace needed and twice got wrong in
this subsystem: a zero meaning "not measured" and a zero meaning "measured as
bad" are different facts, and conflating them produces confident nonsense.
coderai's in-process chat path surfaces no logprobs, so it reports not-measured
rather than a 0.0 posing as one.

g++. Digesta got an olmOCR pod past the TLS problem and straight into a 502:
Triton JIT-compiles vLLM's kernels at run time and the launcher it builds is
C++, so it invokes g++ -- and `vllm.apt` shipped only gcc. With only gcc the
image looks right, vLLM starts, and then the engine core dies before it
registers. Added to `vllm.apt` AND to `vllm-ocr.apt`, which was copied from it
and had inherited the same gap before ever being used.

BOTH vLLM pod images therefore need a rebuild to carry it: coderai-vllm (which
olmOCR uses, so this is what unblocks Digesta) and coderai-vllm-ocr (pushed an
hour ago and affected identically). One at a time, as ever.

tests: test_ocr_confidence.py, 20 tests -- the arithmetic (exp then mean, not the
geometric mean), None-not-zero for an unreporting server, the page-vs-block
scope, the rotated-retry carrying the retry's figure, and a registry check so a
new engine cannot arrive with no answer to "what does conf mean here". Ten
existing fakes updated for the new (text, confidence) contract. 858 passed
across the ocr/front/config/admin/runpod selection with the same 8 pre-existing
failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.29"
+__version__ = "0.3.30"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/routes.py

@@ -4168,6 +4168,7 @@ def build_settings_dict(c, gpu_cards):
             "olmocr_temperature": c.ocr.olmocr_temperature,
             "olmocr_timeout": c.ocr.olmocr_timeout,
             "olmocr_retry_rotation": c.ocr.olmocr_retry_rotation,
+            "olmocr_logprobs": c.ocr.olmocr_logprobs,
             "chandra_enabled": c.ocr.chandra_enabled,
             "chandra_accept_license": c.ocr.chandra_accept_license,
             "chandra_instances": c.ocr.chandra_instances,
@@ -4184,6 +4185,7 @@ def build_settings_dict(c, gpu_cards):
             "chandra_top_p": c.ocr.chandra_top_p,
             "chandra_max_retries": c.ocr.chandra_max_retries,
             "chandra_timeout": c.ocr.chandra_timeout,
+            "chandra_logprobs": c.ocr.chandra_logprobs,
             "detect_mode": c.ocr.detect_mode,
             "detect_model_path": c.ocr.detect_model_path,
             "detect_conf": c.ocr.detect_conf,
@@ -4988,6 +4990,7 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
         if "olmocr_temperature" in d: o.olmocr_temperature = _f("olmocr_temperature", o.olmocr_temperature, 0.0)
         if "olmocr_timeout" in d: o.olmocr_timeout = _f("olmocr_timeout", o.olmocr_timeout, 1.0)
         if "olmocr_retry_rotation" in d: o.olmocr_retry_rotation = bool(d["olmocr_retry_rotation"])
+        if "olmocr_logprobs" in d: o.olmocr_logprobs = bool(d["olmocr_logprobs"])
         if "chandra_enabled" in d: o.chandra_enabled = bool(d["chandra_enabled"])
         if "chandra_accept_license" in d: o.chandra_accept_license = bool(d["chandra_accept_license"])
         if "chandra_instances" in d: o.chandra_instances = _i("chandra_instances", o.chandra_instances, 1)
@@ -5005,6 +5008,7 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
         # 0 is meaningful: accept the first reply however it looks.
         if "chandra_max_retries" in d: o.chandra_max_retries = _i("chandra_max_retries", o.chandra_max_retries, 0)
         if "chandra_timeout" in d: o.chandra_timeout = _f("chandra_timeout", o.chandra_timeout, 1.0)
+        if "chandra_logprobs" in d: o.chandra_logprobs = bool(d["chandra_logprobs"])
         # detection
         if "detect_mode" in d: o.detect_mode = (d.get("detect_mode") or "off").strip()
         if "detect_model_path" in d: o.detect_model_path = (d.get("detect_model_path") or "").strip()

codai/config.py

@@ -1084,6 +1084,10 @@ class OcrConfig:
     olmocr_temperature: float = 0.1
     olmocr_timeout: float = 300.0        # per-page ceiling on the model call
     olmocr_retry_rotation: bool = True   # re-ask once on a page the model reports rotated
+    # As chandra_logprobs, for olmOCR. It returns no lines, so its figure lives on
+    # the page's meta — there is nowhere per-line to put one and nothing per-line
+    # to measure.
+    olmocr_logprobs: bool = True
 
     # --- Chandra (Datalab; code Apache-2.0, WEIGHTS modified OpenRAIL-M) --- a document
     # VLM like olmOCR, but its `ocr_layout` prompt returns one <div> per layout block with
@@ -1127,6 +1131,13 @@ class OcrConfig:
     # is what chandra's own client does. 0 = accept the first reply whatever it looks like.
     chandra_max_retries: int = 2
     chandra_timeout: float = 300.0       # per-page ceiling on the model call
+    # Ask the server for per-token logprobs, which is what makes the reported
+    # `conf` a MEASUREMENT (the mean of exp(logprob), the same computation surya
+    # does) rather than a placeholder. Cheap: one float per token. Chandra
+    # generates a whole page in one call, so the figure is page-level and every
+    # line carries it, with meta.confidence_scope saying so; a server that
+    # declines the field leaves conf at 0.0 and confidence_measured false.
+    chandra_logprobs: bool = True
 
     # --- stamp / signature detection ---  [O3]
     detect_mode: str = "off"             # off|layout|detector|both
@@ -1763,6 +1774,7 @@ class ConfigManager:
                 "olmocr_temperature": self.config.ocr.olmocr_temperature,
                 "olmocr_timeout": self.config.ocr.olmocr_timeout,
                 "olmocr_retry_rotation": self.config.ocr.olmocr_retry_rotation,
+                "olmocr_logprobs": self.config.ocr.olmocr_logprobs,
                 "chandra_enabled": self.config.ocr.chandra_enabled,
                 "chandra_accept_license": self.config.ocr.chandra_accept_license,
                 "chandra_instances": self.config.ocr.chandra_instances,
@@ -1779,6 +1791,7 @@ class ConfigManager:
                 "chandra_top_p": self.config.ocr.chandra_top_p,
                 "chandra_max_retries": self.config.ocr.chandra_max_retries,
                 "chandra_timeout": self.config.ocr.chandra_timeout,
+                "chandra_logprobs": self.config.ocr.chandra_logprobs,
                 "detect_mode": self.config.ocr.detect_mode,
                 "detect_model_path": self.config.ocr.detect_model_path,
                 "detect_conf": self.config.ocr.detect_conf,

codai/ocr/base.py

@@ -297,6 +297,55 @@ def _rasterize_pdf(data: bytes, dpi: int) -> List["Any"]:
         raise OcrError(f"failed to rasterise PDF (pdf2image): {e}", status=400)
 
 
+def mean_token_prob(logprobs) -> Optional[float]:
+    """Mean ``exp(logprob)`` over a reply's tokens, or None if none were reported.
+
+    The measured confidence a VLM engine can actually offer. It is the same
+    computation surya's own client does (``_mean_token_prob`` in
+    ``surya/inference/backends/openai_client.py``), so a number from here is
+    comparable with the one surya reports rather than a second, differently
+    defined "confidence".
+
+    ``None``, never a default, when the server reported nothing. A 0.0 that means
+    "not measured" is indistinguishable from a 0.0 that means "the model was
+    certain it was unsure", and that confusion already produced one confidently
+    wrong verdict in this subsystem (see ``UNREADABLE``). Callers must be able to
+    tell the two apart, which is why every engine that uses this also reports
+    whether the figure was measured.
+
+    Accepts the OpenAI shape — ``choices[0].logprobs.content`` — as dicts or as
+    model objects, since the client library returns either depending on version.
+    """
+    import math
+    if not logprobs:
+        return None
+    total, n = 0.0, 0
+    for entry in logprobs:
+        lp = (entry.get("logprob") if isinstance(entry, dict)
+              else getattr(entry, "logprob", None))
+        if not isinstance(lp, (int, float)):
+            continue
+        try:
+            total += math.exp(float(lp))
+        except (OverflowError, ValueError):
+            continue
+        n += 1
+    return (total / n) if n else None
+
+
+def logprobs_of(result) -> Optional[list]:
+    """``choices[0].logprobs.content`` out of a chat-completions reply, or None."""
+    try:
+        choices = (result or {}).get("choices") or []
+        if not choices:
+            return None
+        lp = (choices[0] or {}).get("logprobs") or {}
+        content = lp.get("content")
+        return content or None
+    except Exception:
+        return None
+
+
 def vlm_serve_limits(cfg) -> dict:
     """The serving limits a VLM OCR engine imposes on its own vLLM instance, as kwargs
     for ``vllm_worker.ensure_service``.

codai/ocr/chandra.py

@@ -46,7 +46,7 @@ from html.parser import HTMLParser
 from typing import List, Optional
 
 from codai.ocr.base import (OcrEngine, OcrPage, OcrLine, OcrRegion, OcrError,
-                            vlm_serve_limits)
+                            logprobs_of, mean_token_prob, vlm_serve_limits)
 
 
 DEFAULT_MODEL = "datalab-to/chandra-ocr-2"
@@ -376,7 +376,8 @@ def parse_bbox(value, width: int, height: int, scale: int = BBOX_SCALE
 
 
 def parse_page(content: str, width: int, height: int, index: int = 0,
-               include_furniture: bool = True) -> OcrPage:
+               include_furniture: bool = True,
+               conf: Optional[float] = None) -> OcrPage:
     """Chandra's reply → an :class:`OcrPage`.
 
     One top-level div is one layout block, and becomes all three of the things coderai's
@@ -385,6 +386,7 @@ def parse_page(content: str, width: int, height: int, index: int = 0,
     only thing that made it a table.
     """
     page = OcrPage(index=index, width=int(width), height=int(height))
+    score = float(conf) if isinstance(conf, (int, float)) else 0.0
     parser = _Blocks()
     parser.feed(content or "")
     parser.close()
@@ -401,13 +403,19 @@ def parse_page(content: str, width: int, height: int, index: int = 0,
         text = html_to_text(blk["html"])
         labels[label] = labels.get(label, 0) + 1
         if box is not None:
-            page.regions.append(OcrRegion(label=label, bbox=box, conf=0.0))
+            page.regions.append(OcrRegion(label=label, bbox=box, conf=score))
         if not text:
             continue
-        # conf 0.0 throughout: the model reports none, and inventing one would be
-        # read as a measurement. Callers that gate on confidence must not be fed a
-        # number nobody computed.
-        page.lines.append(OcrLine(text=text, bbox=box or [0.0, 0.0, 0.0, 0.0], conf=0.0))
+        # ``score`` is the mean token probability of the whole reply, because
+        # chandra generates the ENTIRE page in one call -- unlike surya, which
+        # calls per block and so has a genuinely per-block figure. The same number
+        # is therefore put on every line, and ``meta.confidence_scope`` says
+        # "page" so nobody reads it as per-line precision it does not have. 0.0
+        # when the server reported no logprobs, and ``confidence_measured`` says
+        # which, because a 0.0 that means "not measured" must not be mistaken for
+        # a 0.0 that means "unsure".
+        page.lines.append(OcrLine(text=text, bbox=box or [0.0, 0.0, 0.0, 0.0],
+                                  conf=score))
         texts.append(text)
         if label == "Table":
             page.tables.append({"bbox": box, "html": blk["html"]})
@@ -417,16 +425,21 @@ def parse_page(content: str, width: int, height: int, index: int = 0,
         loose = html_to_text("".join(parser.loose))
         if loose:
             texts.append(loose)
-            page.lines.append(OcrLine(text=loose, bbox=[0.0, 0.0, 0.0, 0.0], conf=0.0))
+            page.lines.append(OcrLine(text=loose, bbox=[0.0, 0.0, 0.0, 0.0],
+                                      conf=score))
     if not texts and not parser.blocks:
         loose = html_to_text(content or "")
         if loose:
             texts.append(loose)
-            page.lines.append(OcrLine(text=loose, bbox=[0.0, 0.0, 0.0, 0.0], conf=0.0))
+            page.lines.append(OcrLine(text=loose, bbox=[0.0, 0.0, 0.0, 0.0],
+                                      conf=score))
 
     page.text = "\n\n".join(texts)
     if labels:
         page.meta["labels"] = labels
+    page.meta["confidence"] = round(score, 4)
+    page.meta["confidence_measured"] = isinstance(conf, (int, float))
+    page.meta["confidence_scope"] = "page"
     return page
 
 
@@ -566,10 +579,11 @@ class ChandraEngine(OcrEngine):
     def recognize_image(self, image) -> OcrPage:
         src_w, src_h = image.size
         data_uri, (w, h) = prepare_image(image)
-        content, meta = self._ask_with_retries(data_uri)
+        content, meta, conf = self._ask_with_retries(data_uri)
         page = parse_page(
             content, w, h,
-            include_furniture=bool(getattr(self.cfg, "chandra_include_furniture", True)))
+            include_furniture=bool(getattr(self.cfg, "chandra_include_furniture", True)),
+            conf=conf)
         # The boxes are the model's, normalised against the image the model saw; the page
         # dimensions reported are the same ones, so a client scaling boxes back onto its
         # own render has a consistent pair. The source size is reported too, since it is
@@ -579,7 +593,7 @@ class ChandraEngine(OcrEngine):
         page.meta.update(meta)
         return page
 
-    def _ask_with_retries(self, data_uri) -> (str, dict):
+    def _ask_with_retries(self, data_uri) -> (str, dict, "Optional[float]"):
         """The page, re-asked while the reply is degenerate.
 
         A repetition loop is not an error the server reports — it answers 200 with a
@@ -591,26 +605,26 @@ class ChandraEngine(OcrEngine):
         top_p = float(getattr(self.cfg, "chandra_top_p", 0.1) or 0.1)
         max_retries = max(0, int(getattr(self.cfg, "chandra_max_retries", 2) or 0))
         meta = {}
-        content = ""
+        content, conf = "", None
         for attempt in range(max_retries + 1):
             if attempt == 0:
                 temp, p = base_temp, top_p
             else:
                 temp, p = min(base_temp + 0.2 * attempt, 0.8), 0.95
-            content = self._ask(data_uri, temp, p)
+            content, conf = self._ask(data_uri, temp, p)
             if not detect_repeat_token(content):
                 if attempt:
                     meta["retries"] = attempt
-                return content, meta
+                return content, meta, conf
         # Out of retries, and the last reply still loops. Returned anyway, flagged: a
         # looping transcription usually has a correct beginning, and the flag is what lets
         # a caller decide rather than be handed a silent mess.
         meta["retries"] = max_retries
         meta["repeat_detected"] = True
-        return content, meta
+        return content, meta, conf
 
-    def _ask(self, data_uri, temperature: float, top_p: float) -> str:
-        """One page → the model's raw reply text."""
+    def _ask(self, data_uri, temperature: float, top_p: float):
+        """One page → ``(raw reply text, mean token probability or None)``."""
         messages = [{
             "role": "user",
             "content": [
@@ -627,7 +641,15 @@ class ChandraEngine(OcrEngine):
             return self._ask_model(messages, max_tokens, temperature, top_p)
         return self._ask_http(messages, max_tokens, temperature, top_p)
 
-    def _ask_model(self, messages, max_tokens, temperature, top_p) -> str:
+    def _want_logprobs(self) -> bool:
+        """Whether to ask the server to report per-token logprobs.
+
+        They are what makes ``conf`` a measurement rather than a guess (see
+        base.mean_token_prob). Cheap — a float per token — and the only reason to
+        turn it off is a server that refuses the field outright."""
+        return bool(getattr(self.cfg, "chandra_logprobs", True))
+
+    def _ask_model(self, messages, max_tokens, temperature, top_p):
         """Model mode: ask the dispatcher which server should take this page.
 
         The in-process path is right for a model this machine serves and wrong for one
@@ -650,7 +672,7 @@ class ChandraEngine(OcrEngine):
         finally:
             lease.release()
 
-    def _ask_local(self, messages, max_tokens, temperature, top_p) -> str:
+    def _ask_local(self, messages, max_tokens, temperature, top_p):
         """Through coderai's own in-process chat handler (model mode)."""
         import asyncio
 
@@ -674,10 +696,13 @@ class ChandraEngine(OcrEngine):
         content = _content_of(result)
         if not content.strip():
             raise OcrError(f"Chandra model '{model_id}' returned no text", status=500)
-        return content
+        # No confidence from this path: coderai's in-process chat handler does not
+        # surface per-token logprobs. Reported as "not measured" rather than 0.0
+        # pretending to be one.
+        return content, None
 
     def _ask_http(self, messages, max_tokens, temperature, top_p, base: str = "",
-                  api_key: str = "", model_name: str = "") -> str:
+                  api_key: str = "", model_name: str = ""):
         """Against an OpenAI-compatible server (vllm/server modes).
 
         ``base``/``api_key``/``model_name`` override the configured server for one page,
@@ -698,6 +723,8 @@ class ChandraEngine(OcrEngine):
             "top_p": top_p,
             "stream": False,
         }
+        if self._want_logprobs():
+            payload["logprobs"] = True
         headers = {"Content-Type": "application/json"}
         key = api_key or (getattr(self.cfg, "chandra_api_key", "") or "").strip()
         if key:
@@ -713,12 +740,16 @@ class ChandraEngine(OcrEngine):
             raise OcrError(f"Chandra server {url} returned HTTP {r.status_code}: "
                            f"{r.text[:300]}", status=502)
         try:
-            content = _content_of(r.json())
+            body = r.json()
+            content = _content_of(body)
         except Exception as e:
             raise OcrError(f"Chandra server returned unparseable JSON: {e}", status=502)
         if not content.strip():
             raise OcrError("Chandra server returned no text", status=502)
-        return content
+        # None when the server declined logprobs, which the page then reports as
+        # confidence_measured: false. A server that ignores the field is a
+        # configuration, not an error.
+        return content, mean_token_prob(logprobs_of(body))
 
 
 def _content_of(result) -> str:

codai/ocr/manager.py

@@ -98,6 +98,24 @@ _AUTO_ALIASES = {"", "auto", "any", "multi", "multilingual", "detect"}
 #: Override with ``ocr.paddle_auto_lang``.
 AUTO_PADDLE_LANG = "latin"
 
+#: What this engine's ``conf`` actually is. ``scope`` says what one number
+#: covers, which is not cosmetic: a per-line score can single out the line to
+#: re-read, while a page-level one can only say the page went badly. The VLMs
+#: report the mean of ``exp(logprob)`` over one generation's tokens (the same
+#: computation surya's own client does), and surya calls per block where chandra
+#: and olmOCR generate the whole page at once. ``measured`` false would mean the
+#: number is invented; nothing here does that, but the per-page
+#: ``confidence_measured`` flag still reports when a server declined logprobs and
+#: left the figure at 0.0.
+_CONFIDENCE = {
+    "paddle":  {"measured": True, "scope": "line", "source": "recogniser score"},
+    "doctr":   {"measured": True, "scope": "line", "source": "recogniser score"},
+    "surya":   {"measured": True, "scope": "block", "source": "mean token probability"},
+    "chandra": {"measured": True, "scope": "page", "source": "mean token probability"},
+    "olmocr":  {"measured": True, "scope": "page", "source": "mean token probability"},
+}
+
+
 #: Engines whose LOADED instance is built for ONE language, so serving another
 #: needs a separate pool.
 #:
@@ -553,6 +571,8 @@ class OcrManager:
             getattr(cfg, "chandra_serve", "model"), getattr(cfg, "chandra_model", ""),
             getattr(cfg, "chandra_model_id", ""), getattr(cfg, "chandra_server_url", ""),
             getattr(cfg, "chandra_layout", True),
+            getattr(cfg, "chandra_logprobs", True),
+            getattr(cfg, "olmocr_logprobs", True),
             getattr(cfg, "vlm_gpu_memory_utilization", 0.0),
         )
 
@@ -607,6 +627,12 @@ class OcrManager:
                 # client stop guessing which of the two it got.
                 "language_aware": engine_uses_language(name),
                 "lang": resolve_lang(cfg, name, None),
+                # Whether `conf` on this engine is a measurement or a placeholder,
+                # and over what. The detector engines score each line; the VLMs
+                # average exp(logprob) over the tokens of one generation, which for
+                # chandra and olmOCR covers a whole page and for surya a block. A
+                # client that gates on confidence has to know which it is getting.
+                "confidence": _CONFIDENCE[name],
             }
             if not enabled:
                 if name == "surya" and cfg.surya_enabled and not cfg.surya_accept_license:

codai/ocr/olmocr.py

@@ -36,7 +36,8 @@ import json
 import threading
 from typing import Optional
 
-from codai.ocr.base import OcrEngine, OcrPage, OcrError, vlm_serve_limits
+from codai.ocr.base import (OcrEngine, OcrPage, OcrError, logprobs_of,
+                            mean_token_prob, vlm_serve_limits)
 
 
 #: olmocr.prompts.build_no_anchoring_v4_yaml_prompt() — kept verbatim: the model was
@@ -294,7 +295,7 @@ class OlmOcrEngine(OcrEngine):
         w, h = image.size
         page = OcrPage(index=0, width=int(w), height=int(h))
 
-        content = self._ask(image)
+        content, conf = self._ask(image)
         meta, body = parse_front_matter(content)
 
         rotation = meta.get("rotation_correction") or 0
@@ -308,21 +309,33 @@ class OlmOcrEngine(OcrEngine):
                 # PIL rotates counter-clockwise; olmOCR reports the correction to APPLY,
                 # so expand=True and the same sign the toolkit uses.
                 retry_img = image.rotate(-turned, expand=True)
-                retry = self._ask(retry_img)
+                retry, retry_conf = self._ask(retry_img)
                 rmeta, rbody = parse_front_matter(retry)
                 if rbody.strip():
                     meta, body = rmeta, rbody
                     meta["rotation_applied"] = turned
+                    # The retry is the answer that is kept, so its confidence is
+                    # the one that describes it.
+                    conf = retry_conf
                     rw, rh = retry_img.size
                     page.width, page.height = int(rw), int(rh)
 
         page.text = body
         page.meta = {k: v for k, v in meta.items() if v is not None}
-        # No detector, so no lines/regions/tables boxes — the text IS the result.
+        # PAGE-level, because olmOCR has no detector and returns no lines: there is
+        # nowhere per-line to put a number and nothing per-line to measure. The
+        # figure is the mean token probability of the one generation that produced
+        # the whole page (base.mean_token_prob), the same computation surya uses, so
+        # it is comparable with surya's rather than a second notion of
+        # "confidence". ``confidence_measured`` distinguishes a real 0.0 from a
+        # server that reported no logprobs.
+        page.meta["confidence"] = round(float(conf), 4) if conf is not None else 0.0
+        page.meta["confidence_measured"] = conf is not None
+        page.meta["confidence_scope"] = "page"
         return page
 
-    def _ask(self, image) -> str:
-        """One page → the model's raw reply text."""
+    def _ask(self, image):
+        """One page → ``(raw reply text, mean token probability or None)``."""
         data_uri = prepare_image(
             image, int(getattr(self.cfg, "olmocr_longest_side", DEFAULT_LONGEST_SIDE) or
                        DEFAULT_LONGEST_SIDE))
@@ -340,7 +353,7 @@ class OlmOcrEngine(OcrEngine):
             return self._ask_model(messages, max_tokens, temperature)
         return self._ask_http(messages, max_tokens, temperature)
 
-    def _ask_model(self, messages, max_tokens, temperature) -> str:
+    def _ask_model(self, messages, max_tokens, temperature):
         """Model mode: ask the dispatcher which server should take this page.
 
         The in-process path below is the right one for a model this machine
@@ -367,7 +380,7 @@ class OlmOcrEngine(OcrEngine):
         finally:
             lease.release()
 
-    def _ask_local(self, messages, max_tokens, temperature) -> str:
+    def _ask_local(self, messages, max_tokens, temperature):
         """Through coderai's own in-process chat handler (model mode)."""
         import asyncio
 
@@ -390,10 +403,12 @@ class OlmOcrEngine(OcrEngine):
         content = _content_of(result)
         if not content.strip():
             raise OcrError(f"olmOCR model '{model_id}' returned no text", status=500)
-        return content
+        # coderai's in-process chat handler surfaces no per-token logprobs, so this
+        # path reports "not measured" rather than a 0.0 posing as one.
+        return content, None
 
     def _ask_http(self, messages, max_tokens, temperature, base: str = "",
-                  api_key: str = "", model_name: str = "") -> str:
+                  api_key: str = "", model_name: str = ""):
         """Against an OpenAI-compatible server (vllm/server modes).
 
         ``base``/``api_key``/``model_name`` override the configured server for one
@@ -419,6 +434,8 @@ class OlmOcrEngine(OcrEngine):
             "temperature": temperature,
             "stream": False,
         }
+        if bool(getattr(self.cfg, "olmocr_logprobs", True)):
+            payload["logprobs"] = True
         headers = {"Content-Type": "application/json"}
         key = api_key or (getattr(self.cfg, "olmocr_api_key", "") or "").strip()
         if key:
@@ -433,12 +450,13 @@ class OlmOcrEngine(OcrEngine):
             raise OcrError(f"olmOCR server {url} returned HTTP {r.status_code}: "
                            f"{r.text[:300]}", status=502)
         try:
-            content = _content_of(r.json())
+            body = r.json()
+            content = _content_of(body)
         except Exception as e:
             raise OcrError(f"olmOCR server returned unparseable JSON: {e}", status=502)
         if not content.strip():
             raise OcrError("olmOCR server returned no text", status=502)
-        return content
+        return content, mean_token_prob(logprobs_of(body))
 
 
 def _content_of(result) -> str:

docs/ocr.md

@@ -664,6 +664,43 @@ model with `runpod.engine: "coderai-vllm-ocr"`; build it with
 `coderai-vllm`: 0.20.1 is seven minor releases behind, and anything else on that image
 is pinned back with it, so rent it for the OCR model only.
 
+### What `conf` means, per engine
+
+Every engine now reports a **measured** confidence, and `/v1/ocr/engines` says what
+one number covers before you send anything:
+
+| engine | scope | source |
+|---|---|---|
+| `paddle` | line | recogniser score |
+| `doctr` | line | recogniser score |
+| `surya` | **block** | mean token probability |
+| `chandra` | **page** | mean token probability |
+| `olmocr` | **page** | mean token probability |
+
+The scope is not cosmetic. A per-line score can single out the line to re-read; a
+page-level one can only tell you the page went badly. surya calls the model once per
+layout block, so its figure is genuinely per-block; chandra and olmOCR generate a
+whole page in one call, so theirs is page-level — chandra puts that same number on
+every line and region (which is what `unreadable`'s per-spot `conf` reads) and says
+`meta.confidence_scope: "page"` so it is not mistaken for per-line precision.
+
+All three VLM figures are the mean of `exp(logprob)` over the generation's tokens —
+the same computation surya's own client does, so they are comparable with each
+other rather than three different notions of "confidence". It costs one float per
+token; `ocr.chandra_logprobs` and `ocr.olmocr_logprobs` turn the request off.
+
+**A server may decline to report logprobs**, and then `conf` is `0.0` — which must
+not be read as "the model was hopeless". `meta.confidence_measured` distinguishes
+the two. That is the same distinction the unreadable-glyph trace needs and twice
+got wrong (see the correction above): a zero that means *not measured* and a zero
+that means *measured as bad* are different facts, and code that conflates them
+produces confident nonsense.
+
+(chandra used to hard-code `conf=0.0`, with a comment arguing that the model
+reported none and inventing one would be read as a measurement. The first half was
+right; the conclusion was wrong, because the number was obtainable all along by the
+route surya already used.)
+
 ### Where `conf` comes from on a served engine
 
 A VLM pipeline has no detector score to report, so surya asks the model server for

packaging/runpod/profiles/vllm-ocr.apt

-# Triton JIT-compiles vLLM's kernels at run time and needs a C compiler for its
+# Triton JIT-compiles vLLM's kernels at run time and needs a compiler for its
 # launcher stubs; without one the engine core dies before it registers.
+#
+# BOTH compilers, not just gcc. The launcher Triton builds is C++, so it invokes
+# g++ — with only gcc present the image looks right, vLLM starts, and then the
+# engine dies and the pod answers 502. Found the hard way on an olmOCR pod: the
+# TLS problem was fixed, the pod became reachable, and the next layer underneath
+# was a missing g++.
 gcc
+g++
 libc6-dev

packaging/runpod/profiles/vllm.apt

-# Triton JIT-compiles vLLM's kernels at run time and needs a C compiler for its
+# Triton JIT-compiles vLLM's kernels at run time and needs a compiler for its
 # launcher stubs; without one the engine core dies before it registers.
+#
+# BOTH compilers, not just gcc. The launcher Triton builds is C++, so it invokes
+# g++ — with only gcc present the image looks right, vLLM starts, and then the
+# engine dies and the pod answers 502. Found the hard way on an olmOCR pod: the
+# TLS problem was fixed, the pod became reachable, and the next layer underneath
+# was a missing g++.
 gcc
+g++
 libc6-dev

tests/test_ocr_chandra.py

@@ -252,7 +252,8 @@ def test_the_boxes_are_read_against_the_scaled_size_not_the_original():
     seen = {}
 
     def _ask(uri, temp, top_p):
-        return '<div data-bbox="0 0 1000 1000" data-label="Text"><p>x</p></div>'
+        return ('<div data-bbox="0 0 1000 1000" data-label="Text"><p>x</p></div>',
+                0.9)
     eng._ask = _ask
     page = eng.recognize_image(Image.new("RGB", (4960, 7016)))
     assert page.width * page.height <= 3072 * 2048
@@ -280,9 +281,9 @@ def test_a_loop_is_re_asked_with_a_higher_temperature():
 
     def _ask(uri, temp, top_p):
         temps.append(temp)
-        return replies[min(len(temps) - 1, len(replies) - 1)]
+        return replies[min(len(temps) - 1, len(replies) - 1)], 0.9
     eng._ask = _ask
-    content, meta = eng._ask_with_retries("data:,")
+    content, meta, _conf = eng._ask_with_retries("data:,")
     assert temps == [0.0, 0.2]
     assert meta["retries"] == 1
     assert "buono" in content
@@ -293,8 +294,8 @@ def test_a_page_that_loops_to_the_end_is_returned_flagged_not_dropped():
     lets a caller decide instead of being handed a silent mess."""
     eng = ch.ChandraEngine(_cfg(chandra_max_retries=1))
     eng._mode = lambda: "server"
-    eng._ask = lambda uri, temp, top_p: "loop " * 60
-    content, meta = eng._ask_with_retries("data:,")
+    eng._ask = lambda uri, temp, top_p: ("loop " * 60, 0.5)
+    content, meta, _conf = eng._ask_with_retries("data:,")
     assert meta["repeat_detected"] is True
     assert meta["retries"] == 1
     assert content
@@ -307,9 +308,9 @@ def test_retries_off_accepts_the_first_reply():
 
     def _ask(uri, temp, top_p):
         calls.append(temp)
-        return "loop " * 60
+        return "loop " * 60, 0.5
     eng._ask = _ask
-    _, meta = eng._ask_with_retries("data:,")
+    _, meta, _conf = eng._ask_with_retries("data:,")
     assert len(calls) == 1
     assert meta["repeat_detected"] is True
 
@@ -317,8 +318,9 @@ def test_retries_off_accepts_the_first_reply():
 def test_a_clean_first_reply_reports_no_retries():
     eng = ch.ChandraEngine(_cfg())
     eng._mode = lambda: "server"
-    eng._ask = lambda uri, temp, top_p: '<div data-label="Text"><p>ok</p></div>'
-    _, meta = eng._ask_with_retries("data:,")
+    eng._ask = lambda uri, temp, top_p: ('<div data-label="Text"><p>ok</p></div>',
+                                         0.9)
+    _, meta, _conf = eng._ask_with_retries("data:,")
     assert "retries" not in meta and "repeat_detected" not in meta
 
 
@@ -426,7 +428,7 @@ def test_the_request_goes_through_pod_http_so_a_pods_own_ca_is_trusted(monkeypat
                                 chandra_server_url="https://10.0.0.9:8000",
                                 chandra_api_key="tok"))
     eng.load()
-    assert "ok" in eng._ask("data:image/png;base64,AA", 0.0, 0.1)
+    assert "ok" in eng._ask("data:image/png;base64,AA", 0.0, 0.1)[0]
     assert seen["url"] == "https://10.0.0.9:8000/v1/chat/completions"
     assert seen["headers"]["Authorization"] == "Bearer tok"
 

tests/test_ocr_olmocr.py

@@ -109,7 +109,10 @@ class _Stub:
     def __call__(self, messages, max_tokens, temperature):
         self.calls.append({"messages": messages, "max_tokens": max_tokens,
                            "temperature": temperature})
-        return self.replies.pop(0) if len(self.replies) > 1 else self.replies[0]
+        reply = self.replies.pop(0) if len(self.replies) > 1 else self.replies[0]
+        # (text, mean token probability) since v0.3.30 — the engines report a
+        # measured confidence now, and the ask helpers carry it.
+        return reply, 0.9
 
 
 def _engine(cfg, stub):

tests/test_olmocr_pod_tls.py

@@ -54,7 +54,7 @@ def test_a_pod_url_goes_through_pod_http(monkeypatch):
         seen["url"] = url
         return _Reply()
     monkeypatch.setattr(pod_http, "post", _post)
-    out = _engine()._ask_http([], 16, 0.0)
+    out, _conf = _engine()._ask_http([], 16, 0.0)
     assert out == "testo"
     assert seen["url"] == "https://1.2.3.4:8000/v1/chat/completions"