_free_vram_gb() used torch.cuda.mem_get_info, which returns 0 on an
AMD/Vulkan engine (no CUDA). That made the auto-offload sizing guard
(_free > 0) silently false, so n_gpu_layers stayed at -1 (all) and a
model larger than VRAM was forced entirely onto the GPU — OOM, "Failed
to load model from file" (e.g. a 13 GB gemma4 model on an 8 GB RX 580).
A 24 GB CUDA card has room for all layers, so the bug was invisible
there. Fall back to amdgpu sysfs (mem_info_vram_total - vram_used,
indexed by Vulkan device order) so AMD GPUs size partial offload too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>