-
Stefy Lanza (nextime / spora ) authored
A rented pod has no config.json and no models.json -- it creates defaults and environment is the only channel it has. CODERAI_OCR_CHANDRA_INSTANCES and CODERAI_OCR_OLMOCR_INSTANCES were sent to every pod and read by nobody, so those engines fell back to the dataclass default of ONE instance however many the operator configured, while the admin UI showed the configured value. Measured on one A40 with chandra_instances=128 and 32 pages in flight: [ocr] engine 'chandra': 1 instance(s) ready vLLM: Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6% Avg generation throughput: 54.5 tokens/s against surya's ~2000 tokens/s and ~32 concurrent sequences on the same card and the same pages. It read as "chandra is 36x slower than surya"; it was one page at a time. Surya worked only because its own env rows had been mapped -- the gap was invisible for exactly as long as surya was the engine being tested. Adds the two instance counts, plus chandra's and olmOCR's page budgets (MAX_TOKENS, MAX_CONTINUATIONS, MAX_RETRIES, DPI), which were sent and unread for the same reason: seeding an engine's wiring but not its tuning gives a pod running that engine's library defaults, which is indistinguishable from the outside from the wrong pod image. The test is deliberately generic -- for each engine and each tuning knob, if the orchestrator sends it the pod must read it. A list of expected names would have passed before this fix on surya's rows alone and said nothing about the other two engines, which is how this survived. NOTE: config.py runs ON the pod, so a pod only gets this once its image carries it. coderai-vllm-ocr must be rebuilt before chandra serves more than one page at a time. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
9632e662