• Stefy Lanza (nextime / spora )'s avatar
    runpod: one pod per pool across engines, not one per engine · b0e3cb7f
    Stefy Lanza (nextime / spora ) authored
    The second live test showed nvidia and radeon each renting their OWN pod
    for 'capability:embeddings' — two GPUs, two bills, while max_pods said 1.
    Same root cause as the reaper bug: one engine per GPU, each with its own
    pools, none able to see the others.
    
    Before renting, a pool now looks in the shared registry for a healthy pod
    another process already has for the SAME pool key, and borrows it. The
    registry entry carries the URL, recorded once the pod is serving.
    
    A borrowed pod is released, never terminated: its owner holds the
    lifecycle and the bill, and killing it would take it away from the engine
    still using it. It is also billed at 0 here, so the cost is not
    double-counted against the caps.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    b0e3cb7f
runpod_worker.py 85.8 KB