-
Stefy Lanza (nextime / spora ) authored
chandra and olmOCR each set `meta.confidence` and `meta.confidence_measured`. surya set neither: its predictor reports confidence per LINE, so a caller switching engines was handed a different kind of answer, and nothing could tell "no line reported a confidence" from "the model was certain it was unsure". Now it reports both. Weighted by CHARACTERS rather than per line, because the other two average over tokens -- that is, over text -- and a plain line-mean lets a two-word line count as much as a full one: on [(100 chars, 0.95), (2 chars, 0.1)] it answers 0.9333 instead of 0.525. `confidence_measured` is the half that earns its place, and the reason the aggregate reads the RAW value rather than the per-line field: that field has to coerce a missing confidence to 0.0 for its own sake, which would make nothing reported -> 0.0 a real zero -> 0.0 indistinguishable. One such 0.0 has already produced a confidently wrong verdict in this subsystem. So: nothing reported -> (0.0, False) a real zero -> (0.0, True) mixed -> unreported lines skipped, never counted as zero Both keys reach the caller with no new plumbing: `_page_from_dict` forwards unknown keys by EXCLUSION (a choice made after that reassembly silently dropped `accents_repaired`, and then `model_attempts`), and both are already in the pod->orchestrator whitelist, so a page read on a rented pod reports what a local one does. Tested, including that `False` is not mistaken for the empty value that exclusion filter skips. The aggregate is extracted as `_page_confidence()` so it is testable without surya installed. Worth recording what this is NOT: on the page where the accent bug was found, surya reported ~96 % per line while every accent was a replacement character -- confident, and right to be, having emitted the correct tokens. Confidence is not a corruption detector; the byte trace is. 13 new tests. 3447 passed, 13 pre-existing failures (broker, studio, manual client) untouched. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
6053e8b1