• Stefy Lanza (nextime / spora )'s avatar
    surya reports a page confidence, and whether anything measured it (v0.4.15) · 6053e8b1
    Stefy Lanza (nextime / spora ) authored
    chandra and olmOCR each set `meta.confidence` and `meta.confidence_measured`.
    surya set neither: its predictor reports confidence per LINE, so a caller
    switching engines was handed a different kind of answer, and nothing could tell
    "no line reported a confidence" from "the model was certain it was unsure".
    
    Now it reports both. Weighted by CHARACTERS rather than per line, because the
    other two average over tokens -- that is, over text -- and a plain line-mean
    lets a two-word line count as much as a full one: on
    [(100 chars, 0.95), (2 chars, 0.1)] it answers 0.9333 instead of 0.525.
    
    `confidence_measured` is the half that earns its place, and the reason the
    aggregate reads the RAW value rather than the per-line field: that field has to
    coerce a missing confidence to 0.0 for its own sake, which would make
    
        nothing reported  -> 0.0
        a real zero       -> 0.0
    
    indistinguishable. One such 0.0 has already produced a confidently wrong
    verdict in this subsystem. So:
    
        nothing reported  -> (0.0, False)
        a real zero       -> (0.0, True)
        mixed             -> unreported lines skipped, never counted as zero
    
    Both keys reach the caller with no new plumbing: `_page_from_dict` forwards
    unknown keys by EXCLUSION (a choice made after that reassembly silently dropped
    `accents_repaired`, and then `model_attempts`), and both are already in the
    pod->orchestrator whitelist, so a page read on a rented pod reports what a
    local one does. Tested, including that `False` is not mistaken for the empty
    value that exclusion filter skips.
    
    The aggregate is extracted as `_page_confidence()` so it is testable without
    surya installed.
    
    Worth recording what this is NOT: on the page where the accent bug was found,
    surya reported ~96 % per line while every accent was a replacement character --
    confident, and right to be, having emitted the correct tokens. Confidence is
    not a corruption detector; the byte trace is.
    
    13 new tests. 3447 passed, 13 pre-existing failures (broker, studio, manual
    client) untouched.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    6053e8b1
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
audiocraft_worker.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
chunking.py Loading commit data...
colibri_worker.py Loading commit data...
compose.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
host_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lipsync.py Loading commit data...
log.py Loading commit data...
longcat_worker.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
model_test.py Loading commit data...
models_transfer.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
pod_http.py Loading commit data...
pod_tls.py Loading commit data...
podlog.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
thread_pool.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
uploads.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...
xtts_worker.py Loading commit data...