qwen38 (27B, n_ctx 32000, q4_0 KV) loaded with 48 of 66 layers on the GPU and
4.9 GB of weights on the CPU, for no reason: the auto-offload sizer reserves
"2 GB + n_ctx/12000" = 4.7 GB of headroom for the KV cache, while llama.cpp's own
numbers put the whole KV at 0.55 GB (CUDA0 421.88 MiB + CPU 140.62 MiB). With
20.9 GB free and ~17 GB really needed, everything fitted.

Two assumptions behind that reserve are wrong when they don't hold: that the VRAM
estimate is weights-only, and that the KV is f16. So the reserve is now 1.5 GB
(compute buffers) when the config carries a measured footprint — that number
already includes the KV, adding a context-scaled reserve on top double counts —
and 1.5 GB + n_ctx/40000 when the KV is quantized. The old heuristic stays for a
raw, unmeasured, f16 case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY