_reject_offloaded_quantized already catches a 4-bit model whose weights landed
on the CPU -- that is what stopped Qwen3.5-9B answering every request with "".
But it runs at the BOTTOM of the device_map ladder. So a full-GPU attempt that
OOMs still walks every rung: 427 weight shards across GPU and host RAM, many
minutes of it, before raising an error that was knowable when the ladder
started. The line printed immediately above the fallback says as much --
"quantized kernels cannot read offloaded weights" -- and then falls through to
the ladder anyway.

The other half is that the attempt which failed was never retried honestly. It
ran against whatever happened to be free on a shared card, which is a lottery;
another model's residency is a reason to evict, not a reason to mis-place this
one. Reproduced on the live server today: 7.2 GB free because an embedding model
and a whisper-server held the rest, and nothing asked them to move.

So a quantized model that fails full-GPU now evicts and tries once more, and
refuses immediately if that still does not fit, naming the three things that
help. Unquantized models keep the ladder: offload is legitimate for dense
weights, only slower.

This makes a failure faster and louder rather than slower and quieter, which is
a behaviour change on a live path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu