Five A40s were terminated between 19:14 and 20:11 today. None of them was
broken: each pod's own log ended in `"POST /v1/chat/completions HTTP/1.1"
200 OK` at 4664 tokens/s, seconds before it was reaped, and every replacement
cost a ~6-minute image pull before it could read a page.

`httpx.RemoteProtocolError("Server disconnected without sending a response")`
was in the dead-upstream set. It does not mean the server is gone — it is what
httpx raises when a POOLED connection was closed by the far end between two
requests. uvicorn closes an idle connection after 5s and nothing here raised
that, while this gateway held pooled connections for 60s. So the first page
after any quiet gap hit a socket the pod had already closed, the gateway read
it as an empty address, and the pool terminated a rented GPU for it.

  - a closed pooled connection is retried on a fresh connection, against the
    SAME upstream: re-choosing would send the page to a different pod, or rent
    one;
  - a pod is asked before it is dropped — /healthz, then /health, on a new
    connection. Any answer under 500 counts as alive, a 401 included: that is
    a server, authenticating. Only silence drops it, because dropping is
    irreversible;
  - idle pooled connections are released after 4s
    (CODERAI_OCR_KEEPALIVE_EXPIRY_S), under the server's own keep-alive, so the
    race is rare rather than routine. Reuse within a burst is unaffected.

A pod RunPod really has reclaimed mid-document is still dropped at once, which
is what the original logic was for: left in the pool, every following page picks
the corpse and the in-flight count climbs with nothing draining. The tests pin
both halves, and a read timeout still neither probes nor drops.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW