• Stefy Lanza (nextime / spora )'s avatar
    video: fix dual-expert OOM regression + harden pipeline VRAM teardown · e1ab02b1
    Stefy Lanza (nextime / spora ) authored
    Two fixes for the township video render failing with CUDA OOM and a
    ~17 GB "untracked teardown leak" that survived gc + empty_cache.
    
    1. Resident-experts regression. video_resident_experts now defaults to
       OFF. A dual-expert 14B model (Wan2.2-VACE-Fun: transformer +
       transformer_2, ~10 GB each at 4-bit) cannot hold both experts + text
       encoder + VAE + the activation peak in 24 GB; the resident load left
       transformer_2 on the CPU yet reported success, so the denoise loop
       (which needs both experts) OOM'd at step 0. 'model' CPU offload keeps
       only the active ~7 GB expert resident and swaps, so it fits. Also: when
       resident leaves ANY component off-GPU it is now treated as a failed
       load — the partial pipe is torn down and it falls through to model
       offload, instead of returning a half-loaded pipe that pins ~10 GB.
    
    2. Teardown leak. _free_pipeline_vram now breaks the references that
       outlived a plain component-null: reset any non-default attention
       backend, unload LoRA/PEFT adapters, run the pipe's own
       maybe_free_model_hooks()/reset_device_map() (frees accelerate offload
       hooks + their staging buffers), and drop the _coderai_* stamped attrs,
       before nulling components + gc + empty_cache.
    
    Verified live (image 0.1.33): 31 video units rendered, 0 OOM, 0 leak
    diagnostics, idle GPU back to ~6 GB.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    e1ab02b1
video.py 178 KB