-
Stefy Lanza (nextime / spora ) authored
CONFIDENCE. chandra hard-coded `conf=0.0` and olmOCR reported nothing at all. chandra's code even argued for it: "the model reports none, and inventing one would be read as a measurement". The first half was right and the conclusion was wrong -- the number was obtainable all along, by the route surya already uses. Both are served over an OpenAI chat endpoint, so the request now asks for logprobs and the reply's mean exp(logprob) is the confidence. It is the same computation surya's own client does, so the three VLM figures are comparable with each other instead of being three different notions of "confidence". engine scope source paddle line recogniser score doctr line recogniser score surya block mean token probability chandra page mean token probability olmocr page mean token probability The SCOPE is reported too (`/v1/ocr/engines` and `meta.confidence_scope`) and it is not cosmetic: a per-line score can single out the line to re-read, a page-level one can only say the page went badly. surya calls per layout block so its figure is genuinely per-block; chandra and olmOCR generate a whole page in one call, so chandra puts the one page figure on every line and region (which is what `unreadable`'s per-spot conf reads) and says the scope is "page" rather than letting it pass for per-line precision. A server may decline logprobs, and then conf is 0.0 -- which must NOT read as "the model was hopeless". `meta.confidence_measured` separates the two. That is the same distinction the unreadable-glyph trace needed and twice got wrong in this subsystem: a zero meaning "not measured" and a zero meaning "measured as bad" are different facts, and conflating them produces confident nonsense. coderai's in-process chat path surfaces no logprobs, so it reports not-measured rather than a 0.0 posing as one. g++. Digesta got an olmOCR pod past the TLS problem and straight into a 502: Triton JIT-compiles vLLM's kernels at run time and the launcher it builds is C++, so it invokes g++ -- and `vllm.apt` shipped only gcc. With only gcc the image looks right, vLLM starts, and then the engine core dies before it registers. Added to `vllm.apt` AND to `vllm-ocr.apt`, which was copied from it and had inherited the same gap before ever being used. BOTH vLLM pod images therefore need a rebuild to carry it: coderai-vllm (which olmOCR uses, so this is what unblocks Digesta) and coderai-vllm-ocr (pushed an hour ago and affected identically). One at a time, as ever. tests: test_ocr_confidence.py, 20 tests -- the arithmetic (exp then mean, not the geometric mean), None-not-zero for an unreporting server, the page-vs-block scope, the rotated-retry carrying the retry's figure, and a registry check so a new engine cannot arrive with no answer to "what does conf mean here". Ten existing fakes updated for the new (text, confidence) contract. 858 passed across the ocr/front/config/admin/runpod selection with the same 8 pre-existing failures. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
d834f2a2