• Stefy Lanza (nextime / spora )'s avatar
    the pod pool was a silent ceiling on fleet size (v0.4.32) · 52a3fb1e
    Stefy Lanza (nextime / spora ) authored
    A page dispatched to a pod holds a pod-pool thread for its whole life, so the
    pool's width decides how many pods an orchestrator can actually feed:
    
        pods_feedable = pod_pool_width / max_inflight_per_pod
    
    With _MAX_POD at 2048 and a 64-page-per-pod ceiling, that was 32 pods —
    whatever max_pods said. An operator who set max_pods=128 got 96 pods' worth of
    nothing, and no line anywhere explained it: not ocr.max_concurrency (4096 here,
    so it looked generous), not max_pods, not the cgroup (pids.max 16384 with 79 in
    use on the box this was measured on). A silent fleet ceiling is a worse failure
    than a slow one, because every symptom points at the pods.
    
    Raised to 8192, which feeds 128 pods at 64 pages each. Past that an async
    client really is the right answer rather than more thread stacks.
    
    Nothing that protects the box changes. `pool_sizes()` still clamps against
    `task_budget()` — the cgroup's pids.max minus what is running minus headroom —
    still keeps _TASK_HEADROOM in reserve so a busy install can still be diagnosed,
    and still gives the pod pool away before the general pool, because an install
    that cannot answer /healthz cannot be fixed. Tests pin all three.
    
    One existing test asserted `pod == 2048` for a configured width of 2048. That
    was only true because _MAX_POD clamped there; the property it meant to protect
    is that the configured width reaches both pools, which is `want * 2`. Restated
    that way, so the next raise does not break it again.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    52a3fb1e
test_ocr_pod_pool_width.py 6.66 KB