Accented letters came back from surya as U+FFFD -- "piu`" as "pi?", "facolta`"
as "facolt?", nine of them in one page, as real EF BF BD bytes. That is exactly
what a byte-boundary decoding bug looks like: a` is C3 A0, and byte-level BPE
splitting it across two tokens is the classic cause.

It is not that. U+FFFD is a REAL ENTRY in surya-ocr-2's vocabulary -- id 65421
of 65425, the only id that decodes to it, with <unk> a separate token at 65424.
The tokenizer is WordLevel and character-level for Latin text, so a` is a single
token (2160) and an accent CANNOT be split across two: no partial-UTF-8
artefact is possible, and per-token decoding is byte-identical to decoding the
whole string. Surya reads text only from message.content, never from the
per-token logprobs, and the fan-out gateway forwards the upstream body as raw
bytes. So the model predicted "there is a character here I cannot identify" and
every hop delivered that faithfully. Two things follow.

It is a RESOLUTION problem. Surya caps its own input at 3072x2048 = 6,291,456
pixels in scale_to_fit, which is called with DEFAULTS ONLY -- no surya setting
raises it, IMAGE_DPI_HIGHRES included. Measured, for A4: 200 DPI reaches the
model as 3.89 Mpixels, 250 as 6.03, and 255 saturates it; 300 and 400 are
downscaled back to the same 6.23. The shared ocr.dpi = 200 was therefore handing
the model 62% of the pixels it accepts, and a diacritic sits above the x-height
at ~3 pixels tall. So DPI is now chosen per engine (ocr.surya_dpi, default 250)
and resolved BEFORE rasterising; paddle and docTR have their own optima and keep
ocr.dpi. The lookup is generic (f"{engine}_dpi"), so another engine only needs
the field. Resolving the engine first also refuses an unknown one before a large
PDF is rasterised rather than after.

It must not be SILENT. "pi?" indexes, searches and reads wrong for ever, and for
judicial documents a known unknown beats a quiet corruption. Lines, pages and
the document now count their unreadable glyphs and locate each one with the
confidence of the line it sits in -- the finest granularity that exists, since
surya reports one mean token probability per block, not per character, and that
is also the signal worth acting on: a low-confidence line with an unreadable
glyph is a line to re-read at a higher DPI. The key is OMITTED when there are
none, so a clean response is byte-for-byte what it always was, spots are capped
with a truncated count, and the text itself is never rewritten -- stripping the
marker would destroy the only record of where the loss is.

20 tests, including the arithmetic that chose 250, kept so a later edit has to
face it. The 8 unrelated failures in test_broker_config /
test_studio_composed_surfaces / test_whisper_server_local_models are
pre-existing -- verified identical in a clean worktree at HEAD.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW