ocr: olmOCR has to trust a pod's certificate itself, and U+FFFD is one token...

ocr: olmOCR has to trust a pod's certificate itself, and U+FFFD is one token not two bytes (v0.3.22)

olmOCR could not reach a rented pod at all: CERTIFICATE_VERIFY_FAILED. A pod on
the direct-TCP path serves HTTPS with a certificate signed by THIS install's own
CA (codai/api/pod_tls.py), at a public IP that was not known when the certificate
was issued, so stock `requests` fails it twice over.

Surya never hit this because surya attaches through the fan-out gateway, which
learned to trust pod certificates in 0.3.16. olmOCR talks to the upstream the
dispatcher chose DIRECTLY, so it needed the same trust of its own -- and the gap
only surfaced when olmOCR was asked to cover for surya, which is exactly when a
fallback engine must not be the thing that is broken. It now goes through
codai/api/pod_http.py, which verifies https-to-a-bare-IP against our CA with the
hostname check off and hands every other URL -- localhost, a named server,
another coderai -- to requests untouched. Swept the rest of the tree: every other
raw requests call targets a localhost worker or RunPod's own public API, where
pod_http would be a pass-through anyway.

test_ocr_olmocr's HTTP-status test patched sys.modules["requests"] wholesale,
which now breaks pod_http's own import of it; it patches pod_http.post instead
and asserts the same two statuses.

Also records what the accent investigation actually established, because the
natural reading of U+FFFD is wrong and expensive. It is NOT a charset bug:
U+FFFD is a real entry in surya-ocr-2's vocabulary (id 65421 of 65425, the only
id that decodes to it), the tokenizer is WordLevel and character-level for Latin
so an accent cannot be split across tokens, and the decisive detail is the COUNT
-- a byte-level mis-decode yields TWO markers per accented letter (a-grave is two
UTF-8 bytes, each one undecodable), and what arrives is ONE. One marker is one
token. Every other hop was checked: _html_to_text is regex plus html.unescape and
survives literal and entity forms; the isolated-venv worker emits json.dumps with
ensure_ascii=True, so the pipe carries pure-ASCII escapes and the locale cannot
matter; the gateway forwards raw bytes.

And it corrects the previous commit's claim. Raising DPI is right on its own
merits -- 200 really was feeding surya 62% of the pixels it accepts -- but a
production document came back with the same markers at 200 AND at 300, which is
evidence against too-few-pixels being the whole story: a perception failure would
be stochastic, not every accent every time. The docs now say what distinguishes
the remaining candidates (the document's embedded scan resolution, which
`pdfimages -list` reports and which no rasterisation DPI can improve, versus the
serving stack) and how to tell them apart: OCR perfect synthetic Italian through
the same path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.21"
+__version__ = "0.3.22"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/ocr/olmocr.py

@@ -399,7 +399,18 @@ class OlmOcrEngine(OcrEngine):
         ``base``/``api_key``/``model_name`` override the configured server for one
         page, which is how model mode reaches the upstream the dispatcher chose.
         """
-        import requests
+        # pod_http, not requests: in model mode ``base`` is whatever the
+        # dispatcher chose, and that can be a RunPod pod on the direct-TCP path —
+        # https to a bare IP, serving a certificate signed by THIS install's own CA
+        # (codai/api/pod_tls.py). No public CA knows it and the address was not known
+        # when it was issued, so plain `requests` fails it twice over with
+        # CERTIFICATE_VERIFY_FAILED and the engine is simply unreachable. Surya does
+        # not hit this because it attaches through the fan-out gateway, which learned
+        # to trust pod certificates in 0.3.16; olmOCR talks to the upstream directly,
+        # so it needs the same trust itself. pod_http is a drop-in and hands every
+        # other URL — localhost, a named host, a remote coderai — straight to
+        # requests untouched.
+        from codai.api import pod_http
 
         payload = {
             "model": model_name or self._model_name(),
@@ -414,7 +425,7 @@ class OlmOcrEngine(OcrEngine):
             headers["Authorization"] = f"Bearer {key}"
         url = (base or self._base).rstrip("/") + "/chat/completions"
         try:
-            r = requests.post(url, json=payload, headers=headers,
+            r = pod_http.post(url, json=payload, headers=headers,
                               timeout=float(getattr(self.cfg, "olmocr_timeout", 300.0) or 300.0))
         except Exception as e:
             raise OcrError(f"olmOCR server {url} unreachable: {e}", status=503)

docs/ocr.md

@@ -232,6 +232,25 @@ instead of dying at inference.
 Engine selection on a pod follows the usual `alias or path or id`, so an entry aliased
 `olmocr` over an HF repo path is recognised.
 
+### olmOCR reaches a pod directly, so it needs the pod's CA itself
+
+olmOCR failed against a rented pod with `CERTIFICATE_VERIFY_FAILED`. A pod on the
+direct-TCP path serves HTTPS with a certificate signed by **this install's own CA**
+(`codai/api/pod_tls.py`), at a public IP that was not known when the certificate was
+issued — so stock `requests` fails it twice over.
+
+Surya never hit this because surya attaches through the fan-out gateway, which
+learned to trust pod certificates in 0.3.16. olmOCR talks to the upstream the
+dispatcher chose **directly**, so it needed the same trust of its own, and the gap
+only surfaced when olmOCR was asked to cover for surya. It now goes through
+`codai/api/pod_http.py`, which verifies *https-to-a-bare-IP* against our CA with the
+hostname check off and hands every other URL — localhost, a named server, another
+coderai — to `requests` untouched.
+
+The lesson generalises: anything that may be handed a **pod** address uses
+`pod_http`, not `requests`. A localhost worker is unaffected either way, since
+`pod_http` passes non-pod URLs straight through.
+
 ### A document is never refused because one engine is down
 
 `ocr.fallback_engines` (default `auto`) makes a failing engine hand the document to the
@@ -336,7 +355,19 @@ Surya reads its text only from `message.content`, never from the per-token
 logprobs, and coderai's fan-out gateway forwards the upstream body as raw bytes.
 
 So the model **predicted** "there is a character here I cannot identify", and
-every hop delivered that faithfully. Two things follow.
+every hop delivered that faithfully.
+
+The decisive detail is the **count**. A charset mis-decode produces *two* markers
+per accented letter, because `à` is two UTF-8 bytes and each undecodable byte
+becomes its own U+FFFD — `pi��`. What arrives is *one*: `pi�`. One marker is one
+token. Every other hop was checked and is byte-faithful too: `_html_to_text` is
+regex plus `html.unescape` and survives both literal and entity forms; the
+isolated-venv worker returns `json.dumps` with the default `ensure_ascii=True`,
+so what crosses the pipe is pure-ASCII `\uXXXX` escapes and the locale cannot
+matter; the gateway forwards raw bytes. There is no charset conversion anywhere
+in the path that could do this.
+
+Two things follow.
 
 **It is a resolution problem.** Surya caps its own input at 3072×2048 =
 6,291,456 pixels in `scale_to_fit`, which is called with **defaults only** — no
@@ -357,6 +388,19 @@ docTR have their own optima and keep `ocr.dpi`. Above ~255 nothing more reaches
 the model, though supersampling down from 300–400 can still render glyphs more
 cleanly, at the cost of rasterisation time and memory.
 
+**Raising DPI is worth doing on its own merits, but it has not been shown to fix
+U+FFFD.** A production document came back with the same markers at 200 and at
+300, which is evidence against too-few-pixels being the whole story — a
+perception failure would be stochastic, not every accent every time. Note what
+rasterisation DPI can and cannot do: if the PDF *contains a scanned image*, the
+page is re-sampled from that image, so asking for 300 DPI upscales a 150-DPI scan
+and adds no detail that was never captured. `pdfimages -list doc.pdf` reports the
+embedded resolution and is the first thing to check. The remaining candidates are
+the document's own glyph quality and the serving stack (coderai's pods run vLLM
+0.27.1, while surya's own `VLLM_DOCKER_IMAGE` pins `v0.20.1`); the way to tell
+them apart is to OCR **perfect synthetic Italian** through the same path — clean
+output indicts the document, U+FFFD on crisp vector glyphs indicts the stack.
+
 `dpi` only affects **PDF** input. An uploaded image is used at the resolution it
 arrives, so a low-resolution scan cannot be rescued by this setting.
 

tests/test_ocr_olmocr.py

@@ -189,8 +189,6 @@ def test_http_failures_map_to_useful_statuses(monkeypatch):
     eng = OlmOcrEngine(cfg)
     eng.load()
 
-    fake = types.SimpleNamespace()
-
     class _Resp:
         status_code = 500
         text = "boom"
@@ -198,8 +196,13 @@ def test_http_failures_map_to_useful_statuses(monkeypatch):
         def json(self):
             return {}
 
-    fake.post = lambda *a, **k: _Resp()
-    monkeypatch.setitem(sys.modules, "requests", fake)
+    # Patch pod_http.post, not the `requests` module: olmOCR goes through
+    # pod_http now so that a pod's own-CA certificate verifies (it used to fail
+    # with CERTIFICATE_VERIFY_FAILED), and swapping sys.modules["requests"] would
+    # break pod_http's own import of it. The statuses under test are unchanged.
+    from codai.api import pod_http
+
+    monkeypatch.setattr(pod_http, "post", lambda *a, **k: _Resp())
     with pytest.raises(OcrError) as e:
         eng.recognize_image(_page(64, 64))
     assert e.value.status == 502
@@ -207,7 +210,7 @@ def test_http_failures_map_to_useful_statuses(monkeypatch):
     def _raise(*a, **k):
         raise OSError("connection refused")
 
-    fake.post = _raise
+    monkeypatch.setattr(pod_http, "post", _raise)
     with pytest.raises(OcrError) as e:
         eng.recognize_image(_page(64, 64))
     assert e.value.status == 503

tests/test_olmocr_pod_tls.py

+"""olmOCR could not reach a pod: CERTIFICATE_VERIFY_FAILED.
+
+A pod on the direct-TCP path serves HTTPS with a certificate signed by this
+install's OWN CA (codai/api/pod_tls.py), addressed by a public IP that was not
+known when the certificate was issued. Plain `requests` fails that twice over.
+
+Surya never hit it because surya attaches through the fan-out gateway, which
+learned to trust pod certificates in 0.3.16. olmOCR talks to the chosen upstream
+directly -- it is the fallback engine, so the failure only showed up when surya
+was the one being exercised and olmOCR was asked to cover for it. Same missing
+trust, second place.
+
+`pod_http` is the drop-in that already encodes the rule: https to a bare IP is
+verified against our CA with the hostname check off; everything else goes to
+`requests` untouched.
+"""
+
+import pathlib
+import sys
+
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT))
+
+from codai.api import pod_http                                   # noqa: E402
+from codai.ocr import olmocr as om                               # noqa: E402
+
+
+class _Reply:
+    status_code = 200
+
+    def json(self):
+        return {"choices": [{"message": {"content": "testo"}}]}
+
+
+class _Cfg:
+    olmocr_api_key = ""
+    olmocr_timeout = 30.0
+    olmocr_model_id = "olmocr"
+
+
+def _engine():
+    e = om.OlmOcrEngine.__new__(om.OlmOcrEngine)
+    e.cfg = _Cfg()
+    e._base = "https://1.2.3.4:8000/v1"
+    e._model_name = lambda: "olmocr"
+    return e
+
+
+def test_a_pod_url_goes_through_pod_http(monkeypatch):
+    """The whole bug: this call used to go to requests and fail verification."""
+    seen = {}
+
+    def _post(url, **kw):
+        seen["url"] = url
+        return _Reply()
+    monkeypatch.setattr(pod_http, "post", _post)
+    out = _engine()._ask_http([], 16, 0.0)
+    assert out == "testo"
+    assert seen["url"] == "https://1.2.3.4:8000/v1/chat/completions"
+
+
+def test_it_is_not_calling_requests_directly_any_more(monkeypatch):
+    import requests
+    called = []
+    monkeypatch.setattr(requests, "post",
+                        lambda *a, **kw: called.append(1) or _Reply())
+    monkeypatch.setattr(pod_http, "post", lambda url, **kw: _Reply())
+    _engine()._ask_http([], 16, 0.0)
+    assert not called, "olmOCR bypassed pod_http and would fail a pod's TLS"
+
+
+def test_the_per_page_base_override_is_honoured(monkeypatch):
+    """Model mode passes the upstream the dispatcher chose for THIS page; a lease
+    is not a fixed address, so the override is the one that must be used."""
+    seen = {}
+    monkeypatch.setattr(pod_http, "post",
+                        lambda url, **kw: seen.update(url=url) or _Reply())
+    _engine()._ask_http([], 16, 0.0, base="https://9.9.9.9:8000/v1")
+    assert seen["url"].startswith("https://9.9.9.9:8000/v1")
+
+
+def test_an_ordinary_host_is_untouched_by_the_pinning_rule():
+    """Only https-to-bare-IP is a pod. A named server must keep normal
+    verification -- this is what makes the drop-in safe everywhere else."""
+    assert pod_http.is_pinned_url("https://1.2.3.4:8000/v1/chat/completions")
+    assert not pod_http.is_pinned_url("https://ocr.example.com/v1/chat/completions")
+    assert not pod_http.is_pinned_url("http://127.0.0.1:8000/v1/chat/completions")
+
+
+def test_a_transport_failure_is_still_a_503_with_the_url_in_it(monkeypatch):
+    from codai.ocr.base import OcrError
+
+    def _boom(url, **kw):
+        raise OSError("connection refused")
+    monkeypatch.setattr(pod_http, "post", _boom)
+    try:
+        _engine()._ask_http([], 16, 0.0)
+    except OcrError as e:
+        assert e.status == 503
+        assert "1.2.3.4" in str(e)
+    else:
+        raise AssertionError("an unreachable upstream must raise")