-
Stefy Lanza (nextime / spora ) authored
Three related fixes for cross-GPU split of a vision model at large context: 1) gpu_detect: when exposing GPUs for cross-split (allow_cross), include only REAL hardware Vulkan devices (nvidia/amd/intel) and EXCLUDE software rasterizers (llvmpipe/lavapipe/virtio = vendor "other"). They're CPU-backed with no real VRAM, slower than native CPU offload, and their presence skewed the device list so the tensor_split ratio didn't line up with the actual cards (and layers could land on a fake GPU). 2) vulkan auto tensor_split now reserves the mmproj's size + compute margin on main_gpu (the projector always loads there), so the proportional split doesn't fill main_gpu to the brim and abort when CLIP can't allocate (GGML_ASSERT(buffer) failed). 3) auto-offload (n_gpu_layers sizing) subtracts the same mmproj reserve from the POOLED free VRAM, so when model+KV+projector exceed BOTH cards combined it reduces GPU layers and spills to CPU gracefully instead of crashing. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
b5acbc2c