-
Stefy Lanza (nextime / spora ) authored
A pod has no config.json -- its settings arrive as env vars -- and the thread pools are built before the config manager has read them. So on every pod `_configured_concurrency()` returned 0 and the default pool took its floor of 32 workers. Every page on a pod runs in that pool. The pod said so in its own log, every boot: [threads] default pool 32 workers, pod pool 64 (FLOORS -- the config was not readable from here yet ...) [vllm] Maximum concurrency for 32,768 tokens per request: 72.20x [ocr] engine 'surya': 8 instance(s) ready, 64 lane(s) each Measured 2026-10-19: 64 pages dispatched per pod, `Running: 32` on every card, GPU KV cache at 4-10%. Raising the orchestrator's dispatch width could never help; the cap was inside the pod. The sizing now falls back to CODERAI_OCR_MAX_CONCURRENCY, the env var the orchestrator already sends every pod, so a pod sizes its pools the way an install with a config file always did. Co-Authored-By:Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
aeac60df