-
Stefy Lanza (nextime / spora ) authored
record_vram_delta persisted measured_n_gpu_layers from every load, including VRAM-CONTENDED ones (another model resident → llama.cpp fit fewer layers). Since _learned_n_gpu_layers reuses that as a hard cap for auto (-1) configs, one unlucky contended load pinned a model half on CPU permanently — it never recovered even when the card was later free. Observed on the gemma-4-26B-A4B 32k config (alias lisa-32k): a contended load ratcheted 31→20 GPU layers and stuck, spilling ~5.8 GB of weights + KV to CPU on a 3090 with 11 GB free. Fix: in record_vram_delta, never overwrite a HIGHER known-good measured_n_gpu_layers with a LOWER value from an offloaded (CPU-spilled) load. A genuinely-too-big model still settles at its free-card maximum on an uncontended load, so keeping the higher value is safe; worst case is one offload-retry on a later truly-contended load. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
f54428cf