• Stefy Lanza (nextime / spora )'s avatar
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload... · bc31e2d5
    Stefy Lanza (nextime / spora ) authored
    video: fix generation-OOM VRAM leak (traceback pin) + use sequential offload for injected components
    
    Two issues from the latest debug.log on the cached VACE model:
    
    1. Generation-OOM VRAM leak (the cascade driver). The OOM/retry handler reloaded
       INSIDE `except Exception as e:`, so sys.exc_info() kept the failed forward pass's
       traceback alive — pinning its GPU activations (and the resident expert).
       _free_pipeline_vram() then reclaimed nothing, the retry reload saw the card still
       ~21 GB full, and every fallback (sequential/disk) OOM'd in turn -> 500. Same
       traceback-pinning bug fixed earlier for the load ladder. Restructured: capture
       the error, drop e.__traceback__, and do the free + gc + empty_cache + retry
       OUTSIDE the except block, where the activations are finally collectable.
    
    2. model CPU offload OOMs this dual-expert forward. With injected cached components
       we forced 'model' offload, which keeps a whole ~7 GB expert resident; the A14B
       forward then spiked past 24 GB (loaded at 11 GB, OOM mid-generate). Force
       'sequential' instead — minimal footprint that reliably fits; the cache still
       makes the LOAD fast. (offload_strategy=group remains available for more speed.)
    
    Retry policy unified: OOM -> sequential; device-mismatch -> incremental cache off
    (plain device_map build hooks everything itself). Both retry once, then 500.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    bc31e2d5
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...