Two levers were missing from the KV plan.

The one that costs nothing: llama-cpp-python 0.3.30 accepts `swa_full`, and we
never passed it. On a sliding-window model — gemma-4-31B has a 1024-token window
on a 5:1 pattern — swa_full=True makes the sliding layers reserve the FULL context
like the global ones. Turning it off reserves only the window, and it costs no
quality: it limits context reuse across requests, not attention. So it is now both
a config passthrough and the FIRST rung of the fallback ladder, ahead of touching
precision at all.

The other: the ladder was q8_0 then q4_0, symmetric. It is now three rungs —
(q8_0,q8_0) 24.3 GB, (q8_0,q4_0) 18.6 GB, (q4_0,q4_0) 12.9 GB on tonight's
measured 45.7 GB — keeping K more precise than V in the middle, since V tolerates
quantization better.

There is no rung below that: 4.5 bits (q4_0 / iq4_nl) is llama.cpp's floor for a
KV cache. The sub-4-bit ggml types this build exposes (IQ2_*, Q2_K, IQ3_*) are
weight quantizations, not KV cache types.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY