• Stefy Lanza (nextime / spora )'s avatar
    ocr: chandra and olmOCR finish a cut-off page too, through one implementation (v0.3.62) · 639a48f8
    Stefy Lanza (nextime / spora ) authored
    Resumption was surya's alone. chandra and olmOCR stopped at their configured cap
    and returned whatever they had — and what they had is the worse half of the
    problem: chandra emits HTML, so a cut-off reply is unclosed markup that parses to
    a fragment of the page or to nothing, and olmOCR's reply opens with YAML front
    matter, so one cut off before the body closes parses to metadata and NO TEXT AT
    ALL. In the finished page both are indistinguishable from a model that could not
    read it, which is the confusion this whole area exists to remove.
    
    The rules now live once, in codai/ocr/resume.py, and they are the ones surya
    learned the hard way rather than a fresh guess:
    
    * both vLLM flags or neither — `continue_final_message` AND
      `add_generation_prompt: false`. With the second left at its default the chat
      template appends a fresh assistant turn and the model answers again about the
      same image, which reads exactly like a model repeating itself and did for a
      whole day;
    * never resume a reply that is already looping: it did not run out of room, it
      ran out of page, and resuming can only extend the loop;
    * the context window is the ceiling, not the budget. Resumption re-sends
      everything so far as INPUT, so the room shrinks by what was just produced, and
      a segment that no longer fits means the cap is too BIG, not too small;
    * progress has to be real: a segment identical to the previous one, or one that
      comes back looping, ends the loop;
    * `finish_reason` is authoritative both ways — "stop" with a full budget is a
      complete page and must not be resumed, which a token count alone cannot tell.
    
    Both engines' ask paths now return a `resume.Reply` carrying the finish reason
    and the usage counts (they were discarding both), take a `prefix` that goes back
    as the prefilled assistant turn, and report `model_attempts` in the same shape
    surya produces — including the budget in force, so a caller can tell a request
    that landed from one that was ignored. Being in-process, they have no thread
    boundary to cross: `page_options()` reaches them directly.
    
    New settings, in config.json, the admin API and the settings page:
    chandra_max_continuations / chandra_context_tokens and olmocr_max_continuations /
    olmocr_context_tokens. The UI parses them as numbers rather than falling back on
    `||`, because 0 is a real setting here (resumption off) and not "use the
    default".
    
    chandra's `detect_repeat_token` moves to being shared rather than copied: it is a
    port of chandra's upstream util and answers "has this reply fallen into a
    repetition loop", which is not chandra-specific.
    
    42 tests: tests/test_ocr_resume_shared.py (28, the rules) and
    tests/test_ocr_resume_engines.py (14, each engine's own ask path). The existing
    chandra/olmocr/confidence/pod-TLS harnesses were updated to the new ask contract
    — it changed deliberately, and the stubs say why.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    639a48f8
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...