-
Stefy Lanza (nextime / spora ) authored
Two fixes for the township video render failing with CUDA OOM and a ~17 GB "untracked teardown leak" that survived gc + empty_cache. 1. Resident-experts regression. video_resident_experts now defaults to OFF. A dual-expert 14B model (Wan2.2-VACE-Fun: transformer + transformer_2, ~10 GB each at 4-bit) cannot hold both experts + text encoder + VAE + the activation peak in 24 GB; the resident load left transformer_2 on the CPU yet reported success, so the denoise loop (which needs both experts) OOM'd at step 0. 'model' CPU offload keeps only the active ~7 GB expert resident and swaps, so it fits. Also: when resident leaves ANY component off-GPU it is now treated as a failed load — the partial pipe is torn down and it falls through to model offload, instead of returning a half-loaded pipe that pins ~10 GB. 2. Teardown leak. _free_pipeline_vram now breaks the references that outlived a plain component-null: reset any non-default attention backend, unload LoRA/PEFT adapters, run the pipe's own maybe_free_model_hooks()/reset_device_map() (frees accelerate offload hooks + their staging buffers), and drop the _coderai_* stamped attrs, before nulling components + gc + empty_cache. Verified live (image 0.1.33): 31 video units rendered, 0 OOM, 0 leak diagnostics, idle GPU back to ~6 GB. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
e1ab02b1