Skip to content
Commit aeac60df authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

every rented pod was capped at 32 pages by its thread pool (v0.4.43)



A pod has no config.json -- its settings arrive as env vars -- and the
thread pools are built before the config manager has read them. So on
every pod `_configured_concurrency()` returned 0 and the default pool
took its floor of 32 workers. Every page on a pod runs in that pool.

The pod said so in its own log, every boot:

    [threads] default pool 32 workers, pod pool 64 (FLOORS -- the config
              was not readable from here yet ...)
    [vllm]   Maximum concurrency for 32,768 tokens per request: 72.20x
    [ocr]    engine 'surya': 8 instance(s) ready, 64 lane(s) each

Measured 2026-10-19: 64 pages dispatched per pod, `Running: 32` on every
card, GPU KV cache at 4-10%. Raising the orchestrator's dispatch width
could never help; the cap was inside the pod.

The sizing now falls back to CODERAI_OCR_MAX_CONCURRENCY, the env var
the orchestrator already sends every pod, so a pod sizes its pools the
way an install with a config file always did.

Co-Authored-By: default avatarClaude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
parent d0e305fd
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment