Skip to content
  • Stefy Lanza (nextime / spora )'s avatar
    df948c48
    frontproxy: fix leaked GPU-swap slot on request cancellation (queued swap never fired) · df948c48
    Stefy Lanza (nextime / spora ) authored
    The GpuSwapGate.release() was async and awaited in the dispatch `finally`
    blocks. When a request was cancelled/interrupted mid-flight (client disconnect,
    an interrupted text generation), `await self._swap_release(...)` in the finally
    could itself be cancelled BEFORE it decremented the running counter — stranding
    a gate slot. With `running` stuck > 0, `_pump()` never ran, so a video request
    queued behind the interrupted text request was never granted: the GPU never
    swapped even though the text engine had gone idle (observed: video stuck ~37min
    while the text engine was idle for 11).
    
    Fix: make release()/_pump() SYNCHRONOUS and drop the asyncio.Lock. Every critical
    section is straight-line (no await between read and write), so under asyncio's
    single thread they're already atomic — and a synchronous release from a `finally`
    always completes even while the coroutine is being cancelled. acquire() keeps its
    one `await` (t...
    df948c48
    frontproxy: fix leaked GPU-swap slot on request cancellation (queued swap never fired)
    Stefy Lanza (nextime / spora ) authored
    The GpuSwapGate.release() was async and awaited in the dispatch `finally`
    blocks. When a request was cancelled/interrupted mid-flight (client disconnect,
    an interrupted text generation), `await self._swap_release(...)` in the finally
    could itself be cancelled BEFORE it decremented the running counter — stranding
    a gate slot. With `running` stuck > 0, `_pump()` never ran, so a video request
    queued behind the interrupted text request was never granted: the GPU never
    swapped even though the text engine had gone idle (observed: video stuck ~37min
    while the text engine was idle for 11).
    
    Fix: make release()/_pump() SYNCHRONOUS and drop the asyncio.Lock. Every critical
    section is straight-line (no await between read and write), so under asyncio's
    single thread they're already atomic — and a synchronous release from a `finally`
    always completes even while the coroutine is being cancelled. acquire() keeps its
    one `await` (t...
Loading