• Stefy Lanza (nextime / spora )'s avatar
    surya reports a page confidence, and whether anything measured it (v0.4.15) · 6053e8b1
    Stefy Lanza (nextime / spora ) authored
    chandra and olmOCR each set `meta.confidence` and `meta.confidence_measured`.
    surya set neither: its predictor reports confidence per LINE, so a caller
    switching engines was handed a different kind of answer, and nothing could tell
    "no line reported a confidence" from "the model was certain it was unsure".
    
    Now it reports both. Weighted by CHARACTERS rather than per line, because the
    other two average over tokens -- that is, over text -- and a plain line-mean
    lets a two-word line count as much as a full one: on
    [(100 chars, 0.95), (2 chars, 0.1)] it answers 0.9333 instead of 0.525.
    
    `confidence_measured` is the half that earns its place, and the reason the
    aggregate reads the RAW value rather than the per-line field: that field has to
    coerce a missing confidence to 0.0 for its own sake, which would make
    
        nothing reported  -> 0.0
        a real zero       -> 0.0
    
    indistinguishable. One such 0.0 has already produced a confidently wrong
    verdict in this subsystem. So:
    
        nothing reported  -> (0.0, False)
        a real zero       -> (0.0, True)
        mixed             -> unreported lines skipped, never counted as zero
    
    Both keys reach the caller with no new plumbing: `_page_from_dict` forwards
    unknown keys by EXCLUSION (a choice made after that reassembly silently dropped
    `accents_repaired`, and then `model_attempts`), and both are already in the
    pod->orchestrator whitelist, so a page read on a rented pod reports what a
    local one does. Tested, including that `False` is not mistaken for the empty
    value that exclusion filter skips.
    
    The aggregate is extracted as `_page_confidence()` so it is testable without
    surya installed.
    
    Worth recording what this is NOT: on the page where the accent bug was found,
    surya reported ~96 % per line while every accent was a replacement character --
    confident, and right to be, having emitted the correct tokens. Confidence is
    not a corruption detector; the byte trace is.
    
    13 new tests. 3447 passed, 13 pre-existing failures (broker, studio, manual
    client) untouched.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    6053e8b1
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...