-
Stefy Lanza (nextime / spora ) authored
The global "secondary card VRAM cap" was a single number applied to every non-lead card. Replace it with a per-CARD map so each physical card on the machine can be capped independently — e.g. RX 580 → 4 GB, RTX 3090 → 20 GB. - gpu_detect: card_key() (nvidia:<uuid> / amd:<pci>) + gpu_cards() enumerating every physical GPU. Stable keys the front and engines both compute identically. - vulkan: _per_device_card_key() (parallel to _per_device_free_vram_gb, same llama.cpp order) + _resolve_device_caps() combining the per-model secondary cap (non-main devices) with the global per-card map, lowest wins per device. Applied in both the auto-offload pool and the auto-split ratio. - config: OffloadConfig.split_card_caps_gb {key: gb}; per-model split_secondary_cap_gb stays a single value (one cap is enough per model). - manager: pass _global_card_caps to the engine; keep per-model scalar. - settings UI: render one cap input per detected card (name + vendor), keyed by the stable card key; GET returns gpu_cards + saved caps, POST saves the map. Applies on the next engine (re)load (global args are read at engine startup), matching the previous global cap's behavior. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
f416117a