ocr: carry the repair receipt out with the page (v0.3.34)

SURYA READS ITALIAN AGAIN. Measured on the live install, the same synthetic page
that scored 29 markers all morning:

    MARKERS : 0    (29 baseline)
    ACCENTS : AEEIOUaeeiou (all twelve)
    La societa` ha facolta` di piu` attivita`: percio` e` cosi` che l'unita`
    perche' gia` oltre.

...but it reported `accents_repaired: null` while visibly having fixed the page,
because _page_from_dict carried `unreadable_trace` across the worker boundary and
never carried `accents_repaired`. The fix worked; only its receipt was dropped.

That matters more than a missing field: the text is no longer byte-for-byte what
the server sent, and a caller comparing the two has no way to know why. A silent
fix is nearly as bad as a silent corruption.

tests: +2 (47 in test_ocr_accent_repair.py) for the receipt crossing the boundary
and for staying absent when nothing was repaired.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
parent ccc0ac89
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API # Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here. # metadata and the admin web UI read from here.
__version__ = "0.3.33" __version__ = "0.3.34"
__tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference." __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
# Configure the CUDA caching allocator BEFORE torch is imported anywhere. # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
......
...@@ -210,4 +210,12 @@ class SubprocessOcrEngine(OcrEngine): ...@@ -210,4 +210,12 @@ class SubprocessOcrEngine(OcrEngine):
trace = d.get("unreadable_trace") trace = d.get("unreadable_trace")
if trace: if trace:
page.meta["unreadable_trace"] = trace page.meta["unreadable_trace"] = trace
# What the worker had to rebuild. Carried for the same reason the trace is:
# the text is no longer byte-for-byte what the server sent, and a caller
# comparing the two needs to know why. Missing this is why the first
# successful repair reported `accents_repaired: null` while visibly having
# fixed the page -- the fix worked and only its receipt was dropped.
repaired = d.get("accents_repaired")
if repaired:
page.meta["accents_repaired"] = repaired
return page return page
...@@ -480,3 +480,26 @@ def test_decoding_is_per_id_not_bulk(): ...@@ -480,3 +480,26 @@ def test_decoding_is_per_id_not_bulk():
src = inspect.getsource(W._decode_ids) src = inspect.getsource(W._decode_ids)
assert "for i in ids:" in src assert "for i in ids:" in src
assert "tok.decode([i]" in src assert "tok.decode([i]" in src
def test_the_repair_receipt_reaches_the_response():
"""The first successful repair reported `accents_repaired: null` while having
visibly fixed the page: the worker emitted the key and _page_from_dict never
carried it across. A silent fix is nearly as bad as a silent corruption —
nobody can tell the text was rewritten."""
from codai.ocr.subprocess_engine import SubprocessOcrEngine
page = SubprocessOcrEngine._page_from_dict({
"index": 0, "text": "società", "width": 10, "height": 10,
"lines": [], "regions": [], "tables": [],
"accents_repaired": {"blocks": 1, "recovered": 29, "how": "…"},
})
assert page.meta["accents_repaired"]["recovered"] == 29
def test_a_page_with_nothing_repaired_carries_no_receipt():
from codai.ocr.subprocess_engine import SubprocessOcrEngine
page = SubprocessOcrEngine._page_from_dict({
"index": 0, "text": "ok", "width": 1, "height": 1,
"lines": [], "regions": [], "tables": [],
})
assert "accents_repaired" not in page.meta
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment