• Stefy Lanza (nextime / spora )'s avatar
    manager: sweep orphan VRAM + untracked-VRAM diagnostic before every video load · 2ab70301
    Stefy Lanza (nextime / spora ) authored
    Belt-and-suspenders for the "untracked in-use VRAM" failure class (a teardown that
    drops a model's registry entry but leaves GPU tensors resident — e.g. activations
    pinned by an exception traceback):
    
    - manager.sweep_orphan_vram(): gc.collect() + empty_cache() reclaims the
      UNREFERENCED kind (a just-dropped-but-not-collected pipeline), and logs a
      diagnostic when the card holds materially more VRAM than tracked models account
      for (the referenced kind, which can't be freed here — must be fixed at source).
    - manager._estimate_tracked_vram_gb(): sums tracked models' footprints for that
      check (offloaded models over-report, so the check is conservative, never a false
      positive on a legitimately offloaded model).
    - video: call it at the start of every load — right after request_model's eviction
      — so a model swap is verified to have actually freed the outgoing model's VRAM,
      and any stray orphan is swept before the new model loads.
    
    Same-model consecutive requests already reuse the resident (offloaded) pipe via
    request_model, so this only runs on a genuine (re)load.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    2ab70301
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...