-
Stefy Lanza (nextime / spora ) authored
A single pod whose vLLM failed to start (another vLLM still held 37 of its 47 GB) answered 503 to every page. _read_on_pods set the whole pool's pod pipeline aside for 300 s on that one answer, so every page in that process was read on the orchestrator. Live, that meant ~10 pages/s from 34 pods, with the orchestrator's CPU doing the reading. Now a refusal takes that pod out of rotation (lease.dead -> quarantine) and the page tries another pod. The pool is set aside only when a page's attempts were refused by at least two different pods, i.e. when it looks like the image and not one card. Co-Authored-By:
Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
595376ba
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| cluster | ||
| compose | ||
| frontproxy | ||
| models | ||
| ocr | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| util | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| main.py | ||
| ocr_fleet_workers.py | ||
| platform_paths.py | ||
| system_app.py |