• Stefy Lanza (nextime / spora )'s avatar
    ocr: the page budget never reached surya, so every page ran at its 12288 (v0.3.57) · 8fd4ef08
    Stefy Lanza (nextime / spora ) authored
    Reported from the Digesta side as "the pod ignores max_tokens and uses a fixed
    12288 cap, loops, and silently drops dense pages", with a pre-fix pod image
    blamed and the work blocked on a versioned one. The pod is not involved: 12288
    appears nowhere in coderai. It is `settings.SURYA_MAX_TOKENS_FULL_PAGE` in the
    surya library on the ORCHESTRATOR, and surya/recognition builds every full-page
    item with `max_tokens=` that value.
    
    surya's own client then resolves the budget as
    
        max_tokens = item.max_tokens or max_tokens_default
    
    so `item.max_tokens` was always truthy and the `max_tokens_default` the trace
    hook rewrote was never read — by anything. `ocr.surya_max_tokens` and a
    request's own `max_tokens` were both discarded on the vLLM path, which is the
    path in use. Every page ran at 12288, and on a 32,768-token window that leaves
    room for about two segments once the ~4,000-token image prefill is counted, so
    a dense page reached the ceiling mid-answer. That is the "loops and drops
    pages" half of the same report, and it is why the benchmark numbers taken at
    "max_tokens=8000/4000/2000" were all the same run.
    
    The hook now applies the cap to the item as well, which is the value that
    decides, and says so once per worker rather than once per page:
    `[surya] page budget 3072 tokens (overriding surya's own 12288)`. An item that
    refuses the assignment says that too, instead of leaving the cap quietly
    unreachable. With no cap configured surya's own budget is untouched — raising
    it costs tokens on every page.
    
    This was already on the list as "surya's own per-page item.max_tokens overrides
    the request cap — flagged, undecided". The decision: an explicit cap is an
    instruction and a per-page estimate is a guess, so the instruction wins.
    
    What was NOT true in the report, with evidence: the pod image carries coderai
    0.3.53 and `git log 42356742..HEAD -- codai/ocr codai/backends codai/api/text.py
    codai/api/ocr.py codai/models/manager.py` is empty, so it contains every OCR
    fix; the pod self-reported `coderai=0.3.53 git=42356742` at /boot. A versioned
    pod tag is still worth having — a cached `:latest` cannot be told apart from a
    current one by looking — but it was not the blocker.
    
    9 tests in tests/test_surya_item_cap.py.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    8fd4ef08
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...