-
Stefy Lanza (nextime / spora ) authored
An OCR VLM needs a model server AND the surya pipeline that drives it. Which machine runs the second one is now chosen by the pod IMAGE, and by nothing else: coderai-vllm the pod serves the model; the ORCHESTRATOR rasterises every page and sends a ~1.06 MB base64 PNG per A4 page at 250 DPI -- and again for every resumption round. Unchanged. coderai-vllm-ocr the pod carries surya's client venv and pypdfium2, so it is sent a ONE-PAGE PDF (~474 KB, 1.7 ms to split here against ~105 ms to rasterise) and reads it on the card with its own vLLM over loopback. A separate `pod_ocr_pipeline` flag was built first and removed. Two switches that can disagree is the invisible mismatch that cost three rounds of blaming a pod image for orchestrator bugs; choosing the image IS choosing the mode, it already lives in models.json and on the model page, and an image that cannot run a pipeline can no longer be asked to. PAGE-LEVEL FAN-OUT IS UNCHANGED. One page is one request in both modes -- `placement.choose` picks an upstream per page -- so every pod in the pool shares the work and none owns a document. Sending a whole PDF to one pod would turn an 8-way parallel read into a serial one, which is why the split is per page. THE TUNING TRAVELS, not just the wiring. surya_dpi, the page budget, the continuation count, the repeat-detector thresholds, langs, the page window, the resident cap. Every one was set in response to a measured failure, and a pod reading pages at surya's own defaults would be indistinguishable from the "pre-fix pod image" that was wrongly blamed three times. A test cross-checks every CODERAI_OCR_* the plan sends against what config.py actually reads: a variable sent and never read is a setting that silently does not apply, which is the exact shape of two bugs found earlier today. FALLBACK, and what it refuses to hide: * no pod rented -> the WHOLE document is read locally, not half each way (a document read by two pipelines is the hardest kind of result to explain); * a pod answers 500 -> raised, not quietly re-read here; a broken fleet must stay visible; * the image lacks the pipeline -> the orchestrator does what it always did. The image, built and verified (14306 MB, /healthz OK): venvs/surya 1.6 GB torch 2.14.1+cpu cuda=None + ultralytics 8.4.177 venvs/vllm 11 GB main venv: pypdfium2, and no torch at all (light core intact) A CUDA torch in the surya venv would add ~4.2 GB AND put surya's small local line detector on the card serving the model; the venv check asserts torch.version.cuda is None so a resolve cannot quietly undo it. 35 tests. Two flaws of my own that they caught: four passed alone and failed in the suite because they patched sys.modules while `from codai.models import placement` resolves the name as an attribute of the already-imported package -- now patched on the real module, verified in both orders; and _plan_env used a hand-rolled stub that tested the stub, now built from a real RunpodModelConfig. Suite at the 25 pre-existing failures. NOT YET EXERCISED AGAINST A REAL POD: the orchestrator path has only run against fakes, and the image is local -- not pushed or signed. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
5813524c