-
Stefy Lanza (nextime / spora ) authored
Two opt-out speedups for the diffusers video path (both default ON), aimed at phase-3 render time on the dual-expert Wan2.2-VACE-Fun-A14B. 1) Resident experts (video_resident_experts, default on) The injected bnb-4bit components previously forced hook-based 'model' CPU offload, which re-shuffles modules GPU<->CPU on every clip part (text-encode, VACE-encode, VAE-decode all pay a transfer). Now the whole pipeline (both 4-bit experts + text encoder + VAE, ~20 GB) is loaded RESIDENT on the 24 GB card via a new 'resident' load strategy: build with the injected components, then .to('cuda') every component, no offload hooks. The denoise loop and en/decode run on resident weights with zero per-part shuffling. Fully fallback-safe: on an activation-peak OOM the loader degrades to 'model' offload, and the generation ladder gains a ('model', True) rung as the first fallback when rung 0 was resident — i.e. exactly the previous behaviour. Set video_resident_experts=false to force it. record_vram_delta now treats 'resident' as on-GPU (not offloaded) so the measured footprint is the true GPU delta. 2) Attention backend (video_attention_backend, default 'auto') Switch the transformer(s) to a faster attention backend via diffusers 0.38's set_attention_backend, applied at the single _report_loaded chokepoint (covers initial + every fallback reload). 'auto' prefers SageAttention (INT8) if installed, else FlashAttention (flash_attn is installed), else leaves the default SDPA. Per-component try/except so an unavailable backend is a logged no-op. Self-heal: if a non-OOM generation failure occurs while a non-default backend is active (a flash/sage shape incompat on this torch build), the transformers are reset to default attention and the SAME pipe is retried once before any costly reload rung. Config: both read per-model (models.json) first, then global_args, then the default — no config change needed to get the speedups. Untested on a live render (no GPU run this session); behaviour falls back to the prior path on any failure. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
fe2cdee1