• Stefy Lanza (nextime / spora )'s avatar
    runpod: cancel a boot, release a pod, and stop blocking the server (v0.3.80) · 764b5c59
    Stefy Lanza (nextime / spora ) authored
    Three reports from the same run, all of them about giving pods back.
    
    SCALE BLOCKED THE WHOLE SERVER. The endpoint called into the pool straight
    from the coroutine, so while it walked dozens of (card, region) candidates
    -- a network round trip each -- nothing else was served. /v1/runpod/status
    stopped answering during exactly the operation someone would poll it
    about, which makes a slow rental look identical to an outage. It now runs
    off the event loop and the reply is bounded: after `wait_s` (default 20)
    it answers `pending` with a request id, and the SCALING CONTINUES --
    shielded, because cancelling it would leave a half-made lease and possibly
    a pod. GET /v1/runpod/scale/{id} has the result for a caller that gave up.
    
    A BOOT COULD NOT BE CANCELLED. `_booting` is an integer, not pod objects --
    a pod being created does not exist here yet -- so `shed_now` could not
    see one, count one or stop one. A client that asked for a pod, got four on
    the way and released them was told `kept:0, needed:1, floor:1` and paid
    every boot out in full. A shed now bumps a cancel epoch; an ANTICIPATORY
    boot (warm floor, background growth) checks it during the model load and
    again just before it joins the fleet, and hands its pod straight back. A
    boot a request is blocked on is never cancellable, whatever arrives
    meanwhile: the request is what the pod is for. Booting pods also count as
    capacity in the surplus arithmetic, and the shed reports how many it
    stopped.
    
    A POD COULD NOT BE RELEASED unless some pool floor could be lowered to
    lose it. `'pool:digesta-qwen38' has no runpod block` came from code whose
    own comment says the caller cannot look that key up -- so a live pool is
    now addressable by the key /v1/runpod/status reports. And nothing anywhere
    could terminate ONE pod by id: not an adopted pod, not one left by a
    crash, not one whose pool was reconfigured underneath it. POST
    /v1/runpod/release does, admin-scoped (leasing is bounded and expires;
    terminating a named pod is neither), attributing the spend before it drops
    the pod so the money does not land nowhere.
    
    And the over-provisioning: `ensure_ready` incremented `_booting` by ONE
    and then started `want` boots, so with min_pods 3 three cards were on the
    way while the 15s scaler, `_should_grow` and the parallel-boot cap all
    believed there was one -- each topping up against a deficit already being
    filled.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    764b5c59
test_runpod_warm_floor_rerent.py 14.7 KB