• Stefy Lanza (nextime / spora )'s avatar
    ocr: empty bytes are absence, not cleanliness (v0.3.29) · 38800b30
    Stefy Lanza (nextime / spora ) authored
    CORRECTION. v0.3.26-0.3.28 reported `blame: "the serving path"` with
    `conclusive: true` and "the byte stream decoded cleanly", on data that was not
    there. Measured through the live gateway today:
    
        result.raw (what surya reads) : 3 markers  'nt ? propriet? della propriet? d'
        reassembled from logprobs     : 0 markers  'nt  propriet della propriet ...'
        entries with bytes=[]         : 257
        any byte >= 0x80 in stream    : False
    
    vLLM 0.27.1 reports `bytes: []` for the byte-level tokens of surya-ocr-2's custom
    vocabulary, and not one byte >= 0x80 appears anywhere in the stream. The accent
    bytes are not corrupted in the logprobs, they are ABSENT from them.
    
    _reassemble accepted an empty byte list as valid data, so it built a string with
    the accents silently missing, found no invalid UTF-8 in it, and called the result
    clean. That is reading absence as cleanliness -- the exact failure the envelope
    in v0.3.26 was built to prevent for MISSING logprobs, walked into one level
    further down.
    
    Fixed: an empty byte list counts as missing. The trace now counts
    `tokens_without_bytes` and, when any are present, reports `conclusive: false`
    with a `why` that says the loss is inside the serving stack (it produced neither
    usable bytes nor usable text) while stating plainly that this CANNOT say whether
    the model's own output was sound. `unreadable.blame` is then "unknown" rather
    than an accusation the data does not support.
    
    Consequence for surya_repair_accents: it CANNOT work on vLLM 0.27.1 -- there is
    nothing to rebuild from, and that is not a tuning problem. It is kept because its
    guards refuse correctly: the ASCII-skeleton check catches the mismatch, so a page
    comes back with its markers rather than with the accents quietly deleted, and the
    repair will work on any stack that does report the bytes.
    
    So on vLLM 0.27.1 the real options are `chandra` (reads the same pixels
    perfectly) or the `coderai-vllm-ocr` image pinned to the vLLM surya validates
    against.
    
    tests: +5 in test_ocr_token_trace.py (42 total) pinning that empty bytes are
    refused by both the reassembly and the repair, that the blame goes to "unknown"
    rather than the serving path, and that a FULLY reported clean stream still
    produces the genuine verdict. 643 passed across the ocr/front/config/admin
    selection with the same 8 pre-existing failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    38800b30
__init__.py 4.21 KB