• Stefy Lanza (nextime / spora )'s avatar
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload... · bc31e2d5
    Stefy Lanza (nextime / spora ) authored
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload for injected components
    
    Two issues from the latest debug.log on the cached VACE model:
    
    1. Generation-OOM VRAM leak (the cascade driver). The OOM/retry handler reloaded
       INSIDE `except Exception as e:`, so sys.exc_info() kept the failed forward pass's
       traceback alive — pinning its GPU activations (and the resident expert).
       _free_pipeline_vram() then reclaimed nothing, the retry reload saw the card still
       ~21 GB full, and every fallback (sequential/disk) OOM'd in turn -> 500. Same
       traceback-pinning bug fixed earlier for the load ladder. Restructured: capture
       the error, drop e.__traceback__, and do the free + gc + empty_cache + retry
       OUTSIDE the except block, where the activations are finally collectable.
    
    2. model CPU offload OOMs this dual-expert forward. With injected cached components
       we forced 'model' offload, which keeps a whole ~7 GB expert resident; the A14B
       forward then spiked past 24 GB (loaded at 11 GB, OOM mid-generate). Force
       'sequential' instead — minimal footprint that reliably fits; the cache still
       makes the LOAD fast. (offload_strategy=group remains available for more speed.)
    
    Retry policy unified: OOM -> sequential; device-mismatch -> incremental cache off
    (plain device_map build hooks everything itself). Both retry once, then 500.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    bc31e2d5
video.py 163 KB