• Stefy Lanza (nextime / spora )'s avatar
    video: incremental per-component pipeline cache (load/build/cache one component at a time) · 93e2a4b9
    Stefy Lanza (nextime / spora ) authored
    A Wan2.2 A14B pipeline that only loads via offload could never be cached — diffusers
    can't save_pretrained an offloaded pipe, so every load re-quantized from bf16 shards
    (~14 min) and risked OOM. Cache it per-component instead:
    
    - pipeline_cache: per-component subdirs each with their own completion marker +
      signature (component_dir / component_valid / save_component), atomic via .building
      + sweep_stale(). A load evicted before all components are cached keeps the finished
      ones; the next load mixes cached + freshly-built (now-cached) ones, converging to a
      full cache. (Plus the earlier sweep_stale / _unsavable_reason robustness.)
    
    - hf_loading.build_cached_components: for each bnb-quantizable heavy component, load
      from the per-component cache if present, else build fresh from the model — each ONE
      AT A TIME on the GPU (where bnb quantizes it to uniform 4-bit), saved, then moved to
      CPU + empty_cache. Peak VRAM is a single component even when the whole model doesn't
      fit; uniform 4-bit by construction (no bf16/4-bit mix). Verified the bnb round-trip
      (load 4-bit on GPU -> save_pretrained -> to('cpu') -> reload).
    
    - video._load_video_pipeline: inject the cached components like GGUF components so the
      existing offload ladder assembles them. Kill-switch CODERAI_INCREMENTAL_CACHE=0.
      Fully fallback-safe: on any load failure the request handler retries once with
      use_incremental_cache=False (plain build = today's behaviour).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    93e2a4b9
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
characters.py Loading commit data...
custom_pipelines.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
images.py Loading commit data...
log.py Loading commit data...
loras.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
ratelimit.py Loading commit data...
spatial.py Loading commit data...
state.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...