• Stefy Lanza (nextime / spora )'s avatar
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload... · bc31e2d5
    Stefy Lanza (nextime / spora ) authored
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload for injected components
    
    Two issues from the latest debug.log on the cached VACE model:
    
    1. Generation-OOM VRAM leak (the cascade driver). The OOM/retry handler reloaded
       INSIDE `except Exception as e:`, so sys.exc_info() kept the failed forward pass's
       traceback alive — pinning its GPU activations (and the resident expert).
       _free_pipeline_vram() then reclaimed nothing, the retry reload saw the card still
       ~21 GB full, and every fallback (sequential/disk) OOM'd in turn -> 500. Same
       traceback-pinning bug fixed earlier for the load ladder. Restructured: capture
       the error, drop e.__traceback__, and do the free + gc + empty_cache + retry
       OUTSIDE the except block, where the activations are finally collectable.
    
    2. model CPU offload OOMs this dual-expert forward. With injected cached components
       we forced 'model' offload, which keeps a whole ~7 GB expert resident; the A14B
       forward then spiked past 24 GB (loaded at 11 GB, OOM mid-generate). Force
       'sequential' instead — minimal footprint that reliably fits; the cache still
       makes the LOAD fast. (offload_strategy=group remains available for more speed.)
    
    Retry policy unified: OOM -> sequential; device-mismatch -> incremental cache off
    (plain device_map build hooks everything itself). Both retry once, then 500.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    bc31e2d5
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...