Auto: Qwen/Qwen3.5-9B is quantized — trying full-GPU regardless of the
          3.1 GB estimate (5.6 GB free)
    Error loading model Qwen/Qwen3.5-9B: … needs about 3.1 GB of VRAM to
          itself; 5.6 GB is free after evicting what could be evicted.

That reads like a bug in eviction -- 5.6 is more than 3.1 -- and is not. The
estimate was wrong in the dangerous direction: too SMALL, so the loader believed
the model fitted three times over, attempted a full-GPU load of something
needing ~6.5 GB, and failed. The refusal itself is correct and stays: a
quantized model cannot be served from CPU offload, and would answer every
request with an empty completion.

The chain, from the debug line:

    source=used_vram_gb -> 3.13 GB | used_vram_gb=7.795 measured_cfg=8.67
    load_in_4bit=True prequantized=False quant_mult=0.300

`measured_vram_gb=8.67` -- the real 4-bit footprint -- was correctly discarded,
recorded under a different quantization signature; that guard earns its keep
elsewhere and is untouched. The fallback `used_vram_gb=7.795` is a STORAGE-size
baseline that the quant factors apply on top of, so it became 7.795 x 0.300 +
reserve = 3.13 GB.

The multiplier is right: the checkpoint is 19.31 GB of bf16 with
`quantization_config: None`, verified on disk, so `prequantized=False` is also
right. 7.795 was not the size of anything. The post-download backfill writes
`cached_size * 1.2` and runs from the download thread, so an interrupted or
still-running fetch leaves a figure for a fraction of the checkpoint: 6.5 GB
downloaded x 1.2 = 7.8.

So the field can hold a number nobody checked, and the factors then scale it
down again. The checkpoint on disk is measurable fact available at that moment,
and is now a floor -- UPWARD only, with a 1.25x margin, so an operator's larger
value stays theirs and an honest small difference is tolerated. The factors
still apply to the floor, because the disk size is a storage size: 19.31 GB at
4-bit is ~5.8 GB, not 19. On the incident model that is 5.8 + reserve against
5.6 GB free, so the load makes room or is refused honestly instead of being
attempted on a figure three times under the truth.

`source=used_vram_gb(disk-floor)` in the debug line, because a floor reporting
itself as an ordinary estimate would be invisible to whoever reads these.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC