• Stefy Lanza (nextime / spora )'s avatar
    video: revert to model offload for injected components (the leak, not model offload, was the OOM) · 1dbf15ed
    Stefy Lanza (nextime / spora ) authored
    Follow-up to the generation-OOM leak fix: forcing 'sequential' was treating the
    symptom. The real cause of "model offload OOMs generation" was that the card
    already had ~11 GB LEAKED from the prior clip's failed forward (its exception
    traceback pinned the activations). model offload itself adds ~0 to the GPU (weights
    stay on CPU); subtract the leak and the forward fits in 24 GB.
    
    With the leak fixed (free + retry now happen outside the except, so the pinned
    activations are collectable), loads start from a clean card, so force the faster
    'model' CPU offload for injected components instead of sequential. The leak-free
    retry still degrades to sequential only if a genuinely clean-card forward OOMs.
    
    Why load-time cleanup couldn't fix it on its own: the leaked VRAM was neither a
    tracked model (already popped from the registry) nor free-able (still referenced by
    the traceback), so manager eviction had nothing to evict and empty_cache() — which
    only reclaims unreferenced memory — was a no-op. The only fix is dropping the
    reference at its source, which the traceback fix does.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    1dbf15ed
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...