-
Stefy Lanza (nextime / spora ) authored
server.max_parallel_requests defaults to 2 in flight and 6 queued. That is right for one local card answering chat. For an OCR install dispatching pages to rented pods it is the throughput ceiling, and nothing points at it: the pool is sized in pages (ocr.max_concurrency, max_inflight_per_pod), a pod is handed its own much larger limit through env, and the front in front of all of it keeps admitting two requests. Measured on one A40 with ocr.max_concurrency=4096 and the pod allowed 64: the pod sat at `Running: 1, Waiting: 0` -- asking for work and not getting it -- while 219 pages took 1210s, the same as reading them one at a time. Raising the gate took the pod to 8-11 concurrent sequences on the same documents. The `[queue] max_parallel_requests=2` line was printed at every boot throughout and read past, because a number with no consequence attached does not look like a problem. Two changes, neither of which overrides a deliberate choice: * When OCR is enabled and the operator has NOT written the key, the gate is derived from the page budget they did ask for (page_budget/8, floored at 16 and capped at 256) rather than taking a default chosen for a different deployment. An explicit value is obeyed however small, and so is an env override, since env is the only channel a rented pod has. The cap is there because the remaining bound is memory, not CPU: an admitted request holds its whole upload plus the per-page PDFs split from it. * When the gate is explicitly set below what the configured page budget needs, a startup diagnostic says so and names the setting to change, instead of leaving the operator to infer it from pods that look idle. ConfigManager now records which server keys were actually written, because an explicit 2 and an absent 2 are indistinguishable from the parsed value and must be treated differently. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
c3f4a2c0