ocr: Chandra, the document VLM that is asked for its layout (v0.3.24)
A fifth OCR engine. Chandra (datalab-to/chandra-ocr-2) is from the same lab as Surya and is driven the way olmOCR is — one page image over an OpenAI chat endpoint, three serving modes (model | vllm | server) — but its `ocr_layout` prompt asks the model to answer as one <div> per layout block carrying data-bbox (normalised 0-1000) and data-label. So it is the only engine here that returns text, boxes, labelled regions, tables AND page metadata in one pass, with no separate detector. Why it was worth adding now: it is a DIFFERENT MODEL FAMILY from surya-2 (Qwen3-VL's tokenizer, against surya-2's own 65k WordLevel vocabulary) reached over the same transport, at the same pixel ceiling, from the same rasterised page. That makes it a cheap controlled experiment on the U+FFFD accent loss: accents intact under chandra and lost under surya points at the model; both losing them points back at the document's pixels, and then pdfimages -list is the next thing to read. Implementation notes: - NO new pip dependency. The two prompts are mirrored verbatim from chandra/prompts.py (down to the Python repr of the allowed-tag list, which is trained-on wording), the input scaling is chandra's own scale_to_fit reimplemented, and the reply is split with stdlib html.parser rather than pulling bs4 and markdownify into the main venv for a tag split. - Chandra's pixel cap is 3072x2048 = 6,291,456 — the SAME numbers as surya-2 — and it snaps to a 28-pixel grid for the Qwen vision tower. So chandra_dpi defaults to 250 for exactly the reason surya_dpi does. Boxes are de-normalised against the image the MODEL saw, and the size the caller rasterised is reported as meta.source_size when they differ: de-normalising against the pre-scaling size would put every box on a scaled page in the wrong place. - A repetition loop answers 200. A VLM that loses its place repeats the last thing it saw until the token budget runs out, and the result looks like a successful page. chandra_max_retries (2) re-asks with a raised temperature — a greedy decode falls into the same loop twice — and a page that still loops is returned flagged with meta.repeat_detected rather than handed on silently. - The parse never loses page. No blocks at all, content outside any block, an unclosed final div (routine at the token limit), a bbox that will not parse: each keeps the text. A bbox that cannot be read yields NO region rather than a full-page one, because a wrong box looks like a real answer. - Table cells get a space, not a newline, so a table does not flatten to "VoceImporto". License: narrower than Surya's GPL. The code is Apache-2.0 but the WEIGHTS are a modified OpenRAIL-M — research, personal use and companies under $2M funding/revenue, no use competing with Datalab's API. So it is gated on chandra_accept_license, and the pod path differs from surya's on purpose: asking a pod for surya by name IS the GPL acceptance, while chandra's is only ever inherited from a decision already recorded in this install's config. A pod cannot accept a licence on an operator's behalf. lang= is accepted and does not steer it (language_aware: false), like surya-2 and olmOCR: 90+ languages, no language input. So it keeps ONE pool whatever is asked — a language-keyed pool is a second set of loaded instances holding VRAM. Wired through: engine registry, licence gate, per-engine DPI, the vLLM stop path (its VRAM is the subprocess's, not a pool's), the capability catalogue (which now reports boxes/regions/tables from chandra_layout, because with layout off there genuinely are none), model-name aliasing, the automatic fallback chain, pod planning + env seeding, the frontproxy's local-OCR check, models.json capability advertisement, the pipeline step's engine list, admin save/load and the Settings OCR card. tests/test_ocr_chandra.py: 61 tests. 788 passed across the OCR/config/admin/ runpod selection with the same 10 pre-existing failures (broker_config x5, studio_composed_surfaces x3, whisper_server x2). test_ocr_fanout's catalogue test asserted a frozen engine set; it now asserts against the registry, which is what "lists every engine" was always supposed to mean. Also fixed while in there: the Settings page used `parseInt(v) || default` for numbers whose zero is meaningful, so a deliberate 0 saved as the default. New _num() helper only falls back for an empty or unparseable field. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
Showing
codai/ocr/chandra.py
0 → 100644
tests/test_ocr_chandra.py
0 → 100644
Please
register
or
sign in
to comment