-
Stefy Lanza (nextime / spora ) authored
Follow-up to the generation-OOM leak fix: forcing 'sequential' was treating the symptom. The real cause of "model offload OOMs generation" was that the card already had ~11 GB LEAKED from the prior clip's failed forward (its exception traceback pinned the activations). model offload itself adds ~0 to the GPU (weights stay on CPU); subtract the leak and the forward fits in 24 GB. With the leak fixed (free + retry now happen outside the except, so the pinned activations are collectable), loads start from a clean card, so force the faster 'model' CPU offload for injected components instead of sequential. The leak-free retry still degrades to sequential only if a genuinely clean-card forward OOMs. Why load-time cleanup couldn't fix it on its own: the leaked VRAM was neither a tracked model (already popped from the registry) nor free-able (still referenced by the traceback), so manager eviction had nothing to evict and empty_cache() — which only reclaims unreferenced memory — was a no-op. The only fix is dropping the reference at its source, which the traceback fix does. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
1dbf15ed