• Stefy Lanza (nextime / spora )'s avatar
    video: revert to model offload for injected components (the leak, not model offload, was the OOM) · 1dbf15ed
    Stefy Lanza (nextime / spora ) authored
    Follow-up to the generation-OOM leak fix: forcing 'sequential' was treating the
    symptom. The real cause of "model offload OOMs generation" was that the card
    already had ~11 GB LEAKED from the prior clip's failed forward (its exception
    traceback pinned the activations). model offload itself adds ~0 to the GPU (weights
    stay on CPU); subtract the leak and the forward fits in 24 GB.
    
    With the leak fixed (free + retry now happen outside the except, so the pinned
    activations are collectable), loads start from a clean card, so force the faster
    'model' CPU offload for injected components instead of sequential. The leak-free
    retry still degrades to sequential only if a genuinely clean-card forward OOMs.
    
    Why load-time cleanup couldn't fix it on its own: the leaked VRAM was neither a
    tracked model (already popped from the registry) nor free-able (still referenced by
    the traceback), so manager eviction had nothing to evict and empty_cache() — which
    only reclaims unreferenced memory — was a no-op. The only fix is dropping the
    reference at its source, which the traceback fix does.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    1dbf15ed
video.py 163 KB