-
Stefy Lanza (nextime / spora ) authored
measured_n_gpu_layers is reused as a cap so a reload jumps straight to the split that fit instead of rediscovering it through OOM retries, and measured_quant_sig is what stops such a measurement outliving its configuration. It covered quantization only. Setting swa_full=False on a gemma card cuts its KV reservation by about 19 GB — its 25 sliding layers stop reserving the full 178k context and hold their 1024-token window instead, at no quality cost. The learned split of 18 of 30 layers was measured with the old full-SWA KV, and nothing in the signature changed, so it would have survived and pinned the model to CPU on a card that can now hold nearly all of it. That is the layer-ratchet bug again, arriving through a setting the signature was not watching. n_ctx, swa_full, cache_type_k/v and kv_offload now form part of it. They are not "how big the weights are", which is what the docstring used to claim the hash was for, but they decide how much of the card is left for the weights and therefore how many layers fit. offload_kqv is read as a spelling of kv_offload because the loader accepts either, and a card written one way must not keep a measurement taken the other. The field list is deliberately narrow: over-invalidating discards every measurement and restores the OOM ladder on every load, which is the whole cost this mechanism exists to avoid. A test pins that both ways. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
051cc5b2