• Stefy Lanza (nextime / spora )'s avatar
    gpu: make VRAM accounting/eviction ALWAYS multi-device, not just under split · 067a797e
    Stefy Lanza (nextime / spora ) authored
    Previously pooling was gated on gpu_split, so a 2-card same-backend engine (e.g.
    2× 3090, or 2× Radeon) still measured only one device for the fit/eviction math.
    
    Now both manager._get_free_vram_gb() and vulkan._pooled_free_vram_gb():
    - ALWAYS sum every visible CUDA device (torch honours CUDA_VISIBLE_DEVICES, so it
      is scoped to this engine's NVIDIA cards) → same-backend split is accounted for
      with no flag.
    - add AMD card(s) (amdgpu sysfs) only when cross-backend split is on OR no CUDA
      device is visible (a Radeon/Vulkan engine), so a Radeon engine counts its own
      cards and an NVIDIA engine only reaches across to Radeon when split is enabled.
    
    So: 2× NVIDIA → summed across both NVIDIAs; 2× Radeon → summed across both Radeons;
    split on → summed across all NVIDIA + Radeon.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    067a797e
Name
Last commit
Last update
..
__init__.py Loading commit data...
base.py Loading commit data...
cuda.py Loading commit data...
ds4.py Loading commit data...
vulkan.py Loading commit data...