-
Stefy Lanza (nextime / spora ) authored
/v1/runpod/status is the one surface an operator has for seeing WHY a server is busy. During a pod boot wave it was both unanswerable and, when it did answer, wrong about the only number anyone was watching. IT COULD NOT ANSWER. Measured on a live orchestrator: /healthz 200 in 2.9 ms (no thread needed) /metrics 200 in 180 ms (no thread needed) /v1/runpod/status no answer in 25 s (needs a thread) /v1/runpod/scale no answer in 25 s (needs a thread) Nothing was broken. Those two are the only endpoints that hand work to a thread, and they drew it from the DEFAULT executor -- the pool the request path also draws on. 25 s of silence reads as a dead server, and it also meant a client lease could not be RELEASED while the fleet was busy, so pods kept billing with no way to stop them but the console. Three changes, each closing one way that happened: * a CONTROL pool (16 threads, fixed) that nothing in the request path may use -- status, scale and release run there. Its whole value is being empty when everything else is full, and a test pins that no request-path module touches it. * the PDF split moves to a two-wide PDFIUM pool. pdfium admits exactly one caller, so hundreds of concurrent documents each took a general thread and then queued on the same global lock: the serialisation was unchanged while the threads they held were the control plane's. Queueing two-wide costs nothing that was not already being paid. * the snapshot's three sources now run concurrently, each with its own 5 s deadline, and whatever is late is reported as `degraded` rather than waited for. pools_schedule_status() was the half never bounded in v0.4.21 -- pods_status() gave up on a busy pool and this one waited for it -- and the ledger snapshot reads without the lock it shares with the writer that fires on every termination. Enforcement still takes that lock properly: only the snapshot may degrade. IT COULD NOT BE BELIEVED. `live_hourly_usd` summed only HEALTHY pods. RunPod bills from creation and a pod spends ~12 minutes booting without ever being healthy, so a fleet entirely in paid boot reported as free: reported $0.59/hr actual $4.72/hr reported $0.00/hr actual $7.67/hr (13 pods, 8 serving) A client polling it concluded nothing was being spent and kept feeding work. It now counts every pod in the listing, and reports the serving and booting halves separately -- their ratio is how much of the bill is boot, which is the figure that says a ramp is too steep, and it did not exist before. Both admin pages show it; "0 pod(s) · $7.67/hr" was a real reading and is self-contradictory. 29 tests. Full suite 3741 passed, 15 pre-existing failures unchanged. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
9265a7ee