• Stefy Lanza (nextime / spora )'s avatar
    a coarser scan is not a saving, it is a different page (v0.4.12) · 6f74f917
    Stefy Lanza (nextime / spora ) authored
    **The accent regression, reported from production hours after 0.4.7 shipped.**
    Italian accents stopped surviving surya again. The cause was this afternoon's
    scan passthrough, not the old detokenisation fault: its guard used an ABSOLUTE
    150-DPI floor instead of the DPI the page would have been rendered at. surya's
    own default is 250 (`surya_dpi`), and Aruba leaves it unset, so every scanned
    page holding a 150-200 DPI JPEG went to the model at native size -- 25-40 % less
    linear resolution than the render it replaced -- and surya-ocr-2 is measured to
    read badly below 250.
    
    The reasoning error is the part worth keeping: passing a scan through was
    always meant to be "the same pixels, without the work". Fewer pixels is not a
    saving. The floor is now the TARGET DPI, and `page_image_min_dpi` (default 0)
    can only RAISE it. Verified against a 250-DPI engine: a 150 or 200 DPI scan is
    rendered up as before; 250 and 300 DPI scans still pass through, which is where
    the shortcut was always legitimate.
    
    It also honours the REQUEST. `ocr_document` resolves
    `use_dpi = int(dpi) if dpi else engine_dpi(engine)` and hands that one value to
    the pod forward, the local render and (on the pod) this guard -- so "is this
    scan detailed enough" is asked about what the caller asked for, never about a
    default they overrode. Pinned, because a future change reading the engine
    default would silently hand a 200-DPI scan to a request that wanted 300.
    
    **And the rasterisation hop now encodes, instead of shipping raw pixels.**
    Measured on A4 at 250 DPI (6.04 Mpix, surya's own default):
    
        raw RGB over the pipe        18.1 MB   + 109.7 ms for the parent to encode
                                               it for the engine anyway
        PNG level 1 in the worker     0.72 MB  104.7 ms, in the WORKER
        the parent's lazy open                   0.12 ms
    
    The encode is not added, it MOVES: off the single parent process onto the
    raster pool, which is parallel. The pipe carries 25x less, the second encode
    disappears, and the parent holds 0.72 MB per page in flight instead of 18.1 MB
    -- at 48 pages that is 35 MB instead of 870 MB, which is the quantity this
    whole subsystem keeps getting into trouble over. PNG, not JPEG, unless the
    operator asked: a page about to be read must not pass through a lossy codec by
    default. Raw RGB stays the fallback, so the protocol still reads on an older
    parent.
    
    Seven existing tests had encoded the wrong floor and were corrected; one of
    them hardcoded pixel counts that assumed A4 while the fixture is 609 pt, so it
    measured 244 DPI while claiming 250.
    
    11 new tests. 3368 passed, 13 pre-existing failures (broker, studio, manual
    client) untouched.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    6f74f917
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...