• Stefy Lanza (nextime / spora )'s avatar
    ocr: Chandra, the document VLM that is asked for its layout (v0.3.24) · e982ff73
    Stefy Lanza (nextime / spora ) authored
    A fifth OCR engine. Chandra (datalab-to/chandra-ocr-2) is from the same lab as
    Surya and is driven the way olmOCR is — one page image over an OpenAI chat
    endpoint, three serving modes (model | vllm | server) — but its `ocr_layout`
    prompt asks the model to answer as one <div> per layout block carrying
    data-bbox (normalised 0-1000) and data-label. So it is the only engine here
    that returns text, boxes, labelled regions, tables AND page metadata in one
    pass, with no separate detector.
    
    Why it was worth adding now: it is a DIFFERENT MODEL FAMILY from surya-2
    (Qwen3-VL's tokenizer, against surya-2's own 65k WordLevel vocabulary) reached
    over the same transport, at the same pixel ceiling, from the same rasterised
    page. That makes it a cheap controlled experiment on the U+FFFD accent loss:
    accents intact under chandra and lost under surya points at the model; both
    losing them points back at the document's pixels, and then pdfimages -list is
    the next thing to read.
    
    Implementation notes:
    
    - NO new pip dependency. The two prompts are mirrored verbatim from
      chandra/prompts.py (down to the Python repr of the allowed-tag list, which is
      trained-on wording), the input scaling is chandra's own scale_to_fit
      reimplemented, and the reply is split with stdlib html.parser rather than
      pulling bs4 and markdownify into the main venv for a tag split.
    - Chandra's pixel cap is 3072x2048 = 6,291,456 — the SAME numbers as surya-2 —
      and it snaps to a 28-pixel grid for the Qwen vision tower. So chandra_dpi
      defaults to 250 for exactly the reason surya_dpi does. Boxes are
      de-normalised against the image the MODEL saw, and the size the caller
      rasterised is reported as meta.source_size when they differ: de-normalising
      against the pre-scaling size would put every box on a scaled page in the
      wrong place.
    - A repetition loop answers 200. A VLM that loses its place repeats the last
      thing it saw until the token budget runs out, and the result looks like a
      successful page. chandra_max_retries (2) re-asks with a raised temperature —
      a greedy decode falls into the same loop twice — and a page that still loops
      is returned flagged with meta.repeat_detected rather than handed on silently.
    - The parse never loses page. No blocks at all, content outside any block, an
      unclosed final div (routine at the token limit), a bbox that will not parse:
      each keeps the text. A bbox that cannot be read yields NO region rather than
      a full-page one, because a wrong box looks like a real answer.
    - Table cells get a space, not a newline, so a table does not flatten to
      "VoceImporto".
    
    License: narrower than Surya's GPL. The code is Apache-2.0 but the WEIGHTS are
    a modified OpenRAIL-M — research, personal use and companies under $2M
    funding/revenue, no use competing with Datalab's API. So it is gated on
    chandra_accept_license, and the pod path differs from surya's on purpose:
    asking a pod for surya by name IS the GPL acceptance, while chandra's is only
    ever inherited from a decision already recorded in this install's config. A pod
    cannot accept a licence on an operator's behalf.
    
    lang= is accepted and does not steer it (language_aware: false), like surya-2
    and olmOCR: 90+ languages, no language input. So it keeps ONE pool whatever is
    asked — a language-keyed pool is a second set of loaded instances holding VRAM.
    
    Wired through: engine registry, licence gate, per-engine DPI, the vLLM stop
    path (its VRAM is the subprocess's, not a pool's), the capability catalogue
    (which now reports boxes/regions/tables from chandra_layout, because with
    layout off there genuinely are none), model-name aliasing, the automatic
    fallback chain, pod planning + env seeding, the frontproxy's local-OCR check,
    models.json capability advertisement, the pipeline step's engine list, admin
    save/load and the Settings OCR card.
    
    tests/test_ocr_chandra.py: 61 tests. 788 passed across the OCR/config/admin/
    runpod selection with the same 10 pre-existing failures (broker_config x5,
    studio_composed_surfaces x3, whisper_server x2).
    
    test_ocr_fanout's catalogue test asserted a frozen engine set; it now asserts
    against the registry, which is what "lists every engine" was always supposed to
    mean.
    
    Also fixed while in there: the Settings page used `parseInt(v) || default` for
    numbers whose zero is meaningful, so a deliberate 0 saved as the default. New
    _num() helper only falls back for an empty or unparseable field.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    e982ff73
README.md 59.9 KB