• Stefy Lanza (nextime / spora )'s avatar
    vram: measure multi-GPU load delta across all devices (fix split models stuck at 0) · 80b21cf5
    Stefy Lanza (nextime / spora ) authored
    record_vram_delta() measured free VRAM via _free_vram_snapshot(), which only
    read the default CUDA device (device 0). A gpu_split model (e.g. gemma on two
    cards) lands most weights on the other card, so the device-0 delta came out
    ~0, tripped the 'delta <= 0' guard, and never persisted measured_vram_gb. The
    estimate stayed 0 and every load OOM-retried from scratch even with
    force_vram_update set.
    
    Sum free VRAM across every visible CUDA device (scoped per engine via
    CUDA_VISIBLE_DEVICES), plus amdgpu sysfs when cross-pooling is on or no CUDA
    device is visible -- mirroring _get_free_vram_gb(). Before/after snapshots are
    now symmetric, so the delta captures the full multi-card footprint and the
    real value is persisted; subsequent loads size correctly and stop OOM-looping.
    
    Bump version to 0.1.3.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    80b21cf5
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...