-
Stefy Lanza (nextime / spora ) authored
A page dispatched to a pod holds a pod-pool thread for its whole life, so the pool's width decides how many pods an orchestrator can actually feed: pods_feedable = pod_pool_width / max_inflight_per_pod With _MAX_POD at 2048 and a 64-page-per-pod ceiling, that was 32 pods — whatever max_pods said. An operator who set max_pods=128 got 96 pods' worth of nothing, and no line anywhere explained it: not ocr.max_concurrency (4096 here, so it looked generous), not max_pods, not the cgroup (pids.max 16384 with 79 in use on the box this was measured on). A silent fleet ceiling is a worse failure than a slow one, because every symptom points at the pods. Raised to 8192, which feeds 128 pods at 64 pages each. Past that an async client really is the right answer rather than more thread stacks. Nothing that protects the box changes. `pool_sizes()` still clamps against `task_budget()` — the cgroup's pids.max minus what is running minus headroom — still keeps _TASK_HEADROOM in reserve so a busy install can still be diagnosed, and still gives the pod pool away before the general pool, because an install that cannot answer /healthz cannot be fixed. Tests pin all three. One existing test asserted `pod == 2048` for a configured width of 2048. That was only true because _MAX_POD clamped there; the property it meant to protect is that the configured width reaches both pools, which is `want * 2`. Restated that way, so the next raise does not break it again. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
52a3fb1e