One Python process forwards ~16-23 pages/s however many pods it has. Measured
on a 35-pod fleet: front 99% CPU, engine 101%, 14 of 16 cores idle, pod GPUs
at 0-7% KV cache. 128 pods at ~0.65 pages/s each need ~85.

ocr.workers (or CODERAI_OCR_WORKERS) = N starts N --engine-only processes
(codai/ocr_fleet_workers.py, a supervisord program) and writes nginx includes
that send exact /v1/ocr and /v1/ocr/batch to them, least_conn on a shared
zone, front as backup. Everything else still goes through the front. 0 (the
default) leaves routing exactly as before.

The main engine stays the only process that rents, scales, quarantines,
adopts or terminates. Workers (CODERAI_OCR_WORKER=i/N) follow the pod
registry, take a balanced sticky slice (runpod_fleet_share.assign), and
report per-pod in-flight/used/suspect back. The controller merges those
reports into every scale/idle/teardown decision through _load(). Drains go
to workers as flags.

Each worker takes 1/N of the per-process ceilings (max_concurrency,
max_queued_pages, *_instances, rounded up) and never writes config.json or
models.json. nginx retries another worker only on a connect error, never on
a timeout (that would read a page twice, billed twice), with no try cap so
the backup is reachable. worker_connections 1024 -> 16384.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM