• Stefy Lanza (nextime / spora )'s avatar
    gpu: make VRAM accounting/eviction ALWAYS multi-device, not just under split · 067a797e
    Stefy Lanza (nextime / spora ) authored
    Previously pooling was gated on gpu_split, so a 2-card same-backend engine (e.g.
    2× 3090, or 2× Radeon) still measured only one device for the fit/eviction math.
    
    Now both manager._get_free_vram_gb() and vulkan._pooled_free_vram_gb():
    - ALWAYS sum every visible CUDA device (torch honours CUDA_VISIBLE_DEVICES, so it
      is scoped to this engine's NVIDIA cards) → same-backend split is accounted for
      with no flag.
    - add AMD card(s) (amdgpu sysfs) only when cross-backend split is on OR no CUDA
      device is visible (a Radeon/Vulkan engine), so a Radeon engine counts its own
      cards and an NVIDIA engine only reaches across to Radeon when split is enabled.
    
    So: 2× NVIDIA → summed across both NVIDIAs; 2× Radeon → summed across both Radeons;
    split on → summed across all NVIDIA + Radeon.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    067a797e
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...