• Stefy Lanza (nextime / spora )'s avatar
    gpu: global VRAM cap is now per physical card (nvidia/radeon), auto-enumerated · f416117a
    Stefy Lanza (nextime / spora ) authored
    The global "secondary card VRAM cap" was a single number applied to every
    non-lead card. Replace it with a per-CARD map so each physical card on the
    machine can be capped independently — e.g. RX 580 → 4 GB, RTX 3090 → 20 GB.
    
    - gpu_detect: card_key() (nvidia:<uuid> / amd:<pci>) + gpu_cards() enumerating
      every physical GPU. Stable keys the front and engines both compute identically.
    - vulkan: _per_device_card_key() (parallel to _per_device_free_vram_gb, same
      llama.cpp order) + _resolve_device_caps() combining the per-model secondary cap
      (non-main devices) with the global per-card map, lowest wins per device. Applied
      in both the auto-offload pool and the auto-split ratio.
    - config: OffloadConfig.split_card_caps_gb {key: gb}; per-model
      split_secondary_cap_gb stays a single value (one cap is enough per model).
    - manager: pass _global_card_caps to the engine; keep per-model scalar.
    - settings UI: render one cap input per detected card (name + vendor), keyed by
      the stable card key; GET returns gpu_cards + saved caps, POST saves the map.
    
    Applies on the next engine (re)load (global args are read at engine startup),
    matching the previous global cap's behavior.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    f416117a
config.py 39.9 KB