-
Stefy Lanza (nextime / spora ) authored
The GpuSwapGate.release() was async and awaited in the dispatch `finally` blocks. When a request was cancelled/interrupted mid-flight (client disconnect, an interrupted text generation), `await self._swap_release(...)` in the finally could itself be cancelled BEFORE it decremented the running counter — stranding a gate slot. With `running` stuck > 0, `_pump()` never ran, so a video request queued behind the interrupted text request was never granted: the GPU never swapped even though the text engine had gone idle (observed: video stuck ~37min while the text engine was idle for 11). Fix: make release()/_pump() SYNCHRONOUS and drop the asyncio.Lock. Every critical section is straight-line (no await between read and write), so under asyncio's single thread they're already atomic — and a synchronous release from a `finally` always completes even while the coroutine is being cancelled. acquire() keeps its one `await` (t...
df948c48Stefy Lanza (nextime / spora ) authoredThe GpuSwapGate.release() was async and awaited in the dispatch `finally` blocks. When a request was cancelled/interrupted mid-flight (client disconnect, an interrupted text generation), `await self._swap_release(...)` in the finally could itself be cancelled BEFORE it decremented the running counter — stranding a gate slot. With `running` stuck > 0, `_pump()` never ran, so a video request queued behind the interrupted text request was never granted: the GPU never swapped even though the text engine had gone idle (observed: video stuck ~37min while the text engine was idle for 11). Fix: make release()/_pump() SYNCHRONOUS and drop the asyncio.Lock. Every critical section is straight-line (no await between read and write), so under asyncio's single thread they're already atomic — and a synchronous release from a `finally` always completes even while the coroutine is being cancelled. acquire() keeps its one `await` (t...
Loading