• Stefy Lanza (nextime / spora )'s avatar
    chunk: POST /v1/chunk, and the OCR pipeline for all three VLM engines (v0.3.76) · 29233812
    Stefy Lanza (nextime / spora ) authored
    RETRIEVAL CHUNKING. A new endpoint that splits a document into the pieces an
    index stores, with the document's structure decided per request by a named or
    inline template. NOT embedding windowing: `embed_long_input` makes ONE input fit
    a model window and pools it into ONE vector; this produces MANY chunks, each
    indexable, each with its section, parent and references. It never pools and
    never drops text.
    
    Pure CPU -- token counts are characters / chars_per_token, so no tokenizer is
    loaded and no model, GPU or pod is involved. Every image serves it, light-core
    ones included, and a test asserts no torch/transformers import creeps in.
    
    The packing loop is reproduced verbatim, including the parts that look
    improvable: paragraphs joined with "\n\n" (hence the `+ 2`), and an overlap that
    is a RAW CHARACTER TAIL which may begin mid-word. Changing any of it moves every
    existing chunk boundary, and then a re-chunked index cannot be compared with the
    one it replaces.
    
    `conclusioni` is its own label and is never folded into `dispositivo`: in a
    ruling the dispositivo is what the judge decided, in a filing the conclusioni
    are what a party ASKED FOR, and merging them would let a demand surface as a
    holding.
    
    TWO DEVIATIONS FROM THE SPEC, both recorded in the source where someone
    comparing against the original will look:
    
      * The ceiling is target + overlap + 2 (3682), not the 3680 the prose states --
        which agrees with the spec's own "observed max is 3682" two lines later.
        Asserting 3680 would fail a correct port.
      * The verbatim patterns cannot match `**FATTO**`, yet the fidelity list asks
        for that case. The line-anchored alternatives are widened at BOTH ends
        (`^[\s*#>_]*fatto[\s*#>_]*$`), which is how the patterns that already coped
        spell it. Purely additive: every line that matched still matches, so no
        existing match moves -- but a document whose headers were emphasised used to
        fall back to flat chunking and now gets sectioned, so ITS chunks change and
        it needs re-chunking.
    
    FORWARDABLE, with the hazard that comes with it. /v1/chunk rides the EMBEDDINGS
    capability: chunks are on their way to be embedded, that image is full-core so
    it carries the chunker, and a capability of its own would be one nothing
    advertises -- or would rent a GPU to run a regex split. It was first classified
    as deliberately local on the grounds that it is cheap; that was the wrong lens,
    since the orchestrator is the box that was OOM-killed today, so WHERE the CPU is
    matters more than how much there is.
    
    But a template registered HERE does not exist on the remote: forwarded by name
    it returns 404, or worse matches a DIFFERENT template with the same name and
    returns different boundaries silently. So a named non-builtin template is
    resolved and INLINED before the request leaves. The hook never raises -- a body
    it cannot parse is forwarded untouched -- because a rewrite that can throw would
    take down every forwarded request, not just a chunk one.
    
    THE OCR PIPELINE NOW COVERS ALL THREE VLM ENGINES, not just surya. Leaving
    chandra and olmOCR on the orchestrator was an asymmetry rather than a design:
    surya needed a 1.6 GB client venv baked in, while those two speak only HTTP in
    served mode (PIL + requests, already in the main venv), so for them the pod side
    costs nothing beyond the rasteriser. Two bugs fixed on the way, both of which
    would have produced plausible WRONG text rather than an error: the dispatch
    hardcoded `engine: "surya"` in the form it posted, and the pod env was
    surya-shaped throughout -- a chandra pod was told to run surya against chandra's
    model id. Per-engine tuning and per-engine licence acceptance now travel.
    
    AND THE LOG THAT CAUSED A MISDIAGNOSIS. One pod request walks a candidate list
    until a (card, region) pair has capacity, and each attempt printed
    "provisioning pod for ...". 312 attempts for 10 pods therefore read as runaway
    provisioning and was reported as such. It now says "trying candidate N/M for
    ONE pod", so the shape of the list is visible instead of inferred.
    
    What these tests CANNOT do is prove byte-identity with the implementation this
    ports -- that needs fixtures from a live corpus. They prove the documented
    behaviour on each path; identity is what decides whether a re-chunk is safe and
    it is still outstanding.
    
    Suite at the 25 pre-existing failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    29233812
Name
Last commit
Last update
..
superpowers Loading commit data...
README.md Loading commit data...
admin-forward-auth.md Loading commit data...
chunking.md Loading commit data...
cluster.md Loading commit data...
deepseek-ds4.md Loading commit data...
dtype-auto-selection.md Loading commit data...
expressive-tts.md Loading commit data...
frontend-engine-split.md Loading commit data...
gguf-process-isolation.md Loading commit data...
glm-colibri.md Loading commit data...
install-from-packages.md Loading commit data...
kimi-k3.md Loading commit data...
ktransformers.md Loading commit data...
longcat-video.md Loading commit data...
minimax-h3.md Loading commit data...
modelli-digesta-e-container.md Loading commit data...
ocr.md Loading commit data...
orchestrator-runpod-guide.md Loading commit data...
piano-inferenza-su-coderai-prod.md Loading commit data...
process-isolation-plans.md Loading commit data...
reference-projects.md Loading commit data...
remote-execution.md Loading commit data...
reverse-proxy-nginx.md Loading commit data...
runpod-regional-storage-plan.md Loading commit data...
runpod.md Loading commit data...
tools-as-services.md Loading commit data...
unlimited-ocr-plan.md Loading commit data...
video-compose.md Loading commit data...
vllm.md Loading commit data...
zimage-lora-training.md Loading commit data...