-
Stefy Lanza (nextime / spora ) authored
video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload for injected components Two issues from the latest debug.log on the cached VACE model: 1. Generation-OOM VRAM leak (the cascade driver). The OOM/retry handler reloaded INSIDE `except Exception as e:`, so sys.exc_info() kept the failed forward pass's traceback alive — pinning its GPU activations (and the resident expert). _free_pipeline_vram() then reclaimed nothing, the retry reload saw the card still ~21 GB full, and every fallback (sequential/disk) OOM'd in turn -> 500. Same traceback-pinning bug fixed earlier for the load ladder. Restructured: capture the error, drop e.__traceback__, and do the free + gc + empty_cache + retry OUTSIDE the except block, where the activations are finally collectable. 2. model CPU offload OOMs this dual-expert forward. With injected cached components we forced 'model' offload, which keeps a whole ~7 GB expert resident; the A14B forward then spiked past 24 GB (loaded at 11 GB, OOM mid-generate). Force 'sequential' instead — minimal footprint that reliably fits; the cache still makes the LOAD fast. (offload_strategy=group remains available for more speed.) Retry policy unified: OOM -> sequential; device-mismatch -> incremental cache off (plain device_map build hooks everything itself). Both retry once, then 500. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
bc31e2d5