• Stefy Lanza (nextime / spora )'s avatar
    ocr: chandra and olmOCR report a measured confidence, and vLLM pods need g++ (v0.3.30) · d834f2a2
    Stefy Lanza (nextime / spora ) authored
    CONFIDENCE. chandra hard-coded `conf=0.0` and olmOCR reported nothing at all.
    chandra's code even argued for it: "the model reports none, and inventing one
    would be read as a measurement". The first half was right and the conclusion was
    wrong -- the number was obtainable all along, by the route surya already uses.
    Both are served over an OpenAI chat endpoint, so the request now asks for
    logprobs and the reply's mean exp(logprob) is the confidence. It is the same
    computation surya's own client does, so the three VLM figures are comparable
    with each other instead of being three different notions of "confidence".
    
        engine    scope   source
        paddle    line    recogniser score
        doctr     line    recogniser score
        surya     block   mean token probability
        chandra   page    mean token probability
        olmocr    page    mean token probability
    
    The SCOPE is reported too (`/v1/ocr/engines` and `meta.confidence_scope`) and it
    is not cosmetic: a per-line score can single out the line to re-read, a
    page-level one can only say the page went badly. surya calls per layout block so
    its figure is genuinely per-block; chandra and olmOCR generate a whole page in
    one call, so chandra puts the one page figure on every line and region (which is
    what `unreadable`'s per-spot conf reads) and says the scope is "page" rather than
    letting it pass for per-line precision.
    
    A server may decline logprobs, and then conf is 0.0 -- which must NOT read as
    "the model was hopeless". `meta.confidence_measured` separates the two. That is
    the same distinction the unreadable-glyph trace needed and twice got wrong in
    this subsystem: a zero meaning "not measured" and a zero meaning "measured as
    bad" are different facts, and conflating them produces confident nonsense.
    coderai's in-process chat path surfaces no logprobs, so it reports not-measured
    rather than a 0.0 posing as one.
    
    g++. Digesta got an olmOCR pod past the TLS problem and straight into a 502:
    Triton JIT-compiles vLLM's kernels at run time and the launcher it builds is
    C++, so it invokes g++ -- and `vllm.apt` shipped only gcc. With only gcc the
    image looks right, vLLM starts, and then the engine core dies before it
    registers. Added to `vllm.apt` AND to `vllm-ocr.apt`, which was copied from it
    and had inherited the same gap before ever being used.
    
    BOTH vLLM pod images therefore need a rebuild to carry it: coderai-vllm (which
    olmOCR uses, so this is what unblocks Digesta) and coderai-vllm-ocr (pushed an
    hour ago and affected identically). One at a time, as ever.
    
    tests: test_ocr_confidence.py, 20 tests -- the arithmetic (exp then mean, not the
    geometric mean), None-not-zero for an unreporting server, the page-vs-block
    scope, the rotated-retry carrying the retry's figure, and a registry check so a
    new engine cannot arrive with no answer to "what does conf mean here". Ten
    existing fakes updated for the new (text, confidence) contract. 858 passed
    across the ocr/front/config/admin/runpod selection with the same 8 pre-existing
    failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    d834f2a2
vllm.apt 514 Bytes