a size estimate must not be smaller than the weights on disk

    Auto: Qwen/Qwen3.5-9B is quantized — trying full-GPU regardless of the
          3.1 GB estimate (5.6 GB free)
    Error loading model Qwen/Qwen3.5-9B: … needs about 3.1 GB of VRAM to
          itself; 5.6 GB is free after evicting what could be evicted.

That reads like a bug in eviction -- 5.6 is more than 3.1 -- and is not. The
estimate was wrong in the dangerous direction: too SMALL, so the loader believed
the model fitted three times over, attempted a full-GPU load of something
needing ~6.5 GB, and failed. The refusal itself is correct and stays: a
quantized model cannot be served from CPU offload, and would answer every
request with an empty completion.

The chain, from the debug line:

    source=used_vram_gb -> 3.13 GB | used_vram_gb=7.795 measured_cfg=8.67
    load_in_4bit=True prequantized=False quant_mult=0.300

`measured_vram_gb=8.67` -- the real 4-bit footprint -- was correctly discarded,
recorded under a different quantization signature; that guard earns its keep
elsewhere and is untouched. The fallback `used_vram_gb=7.795` is a STORAGE-size
baseline that the quant factors apply on top of, so it became 7.795 x 0.300 +
reserve = 3.13 GB.

The multiplier is right: the checkpoint is 19.31 GB of bf16 with
`quantization_config: None`, verified on disk, so `prequantized=False` is also
right. 7.795 was not the size of anything. The post-download backfill writes
`cached_size * 1.2` and runs from the download thread, so an interrupted or
still-running fetch leaves a figure for a fraction of the checkpoint: 6.5 GB
downloaded x 1.2 = 7.8.

So the field can hold a number nobody checked, and the factors then scale it
down again. The checkpoint on disk is measurable fact available at that moment,
and is now a floor -- UPWARD only, with a 1.25x margin, so an operator's larger
value stays theirs and an honest small difference is tolerated. The factors
still apply to the floor, because the disk size is a storage size: 19.31 GB at
4-bit is ~5.8 GB, not 19. On the incident model that is 5.8 + reserve against
5.6 GB free, so the load makes room or is refused honestly instead of being
attempted on a figure three times under the truth.

`source=used_vram_gb(disk-floor)` in the debug line, because a floor reporting
itself as an ordinary estimate would be invisible to whoever reads these.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.5.59"
+__version__ = "0.5.60"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/models/manager.py

@@ -4500,6 +4500,40 @@ class MultiModelManager:
             return 2.67
         return 1.0
 
+    def _disk_weights_gb(self, model_key: str, resolved_name: str,
+                         cfg: dict) -> float:
+        """GB of weights actually on disk for this model, or 0.0 when unknown.
+
+        The same two sources the size-estimate passes use further down -- a local
+        weights file, else the HF hub cache for the repo id -- read here so the
+        `used_vram_gb` branch can sanity-check itself against them instead of
+        short-circuiting past them. Pure and cheap: a stat, or the memoised hub
+        cache scan. Never raises; 0.0 means "no opinion", which leaves the
+        configured value untouched.
+        """
+        try:
+            import os as _os
+            for cand in (cfg.get("path"), resolved_name, model_key):
+                p = str(cand or "")
+                if not p:
+                    continue
+                _, ext = _os.path.splitext(p)
+                if ext.lower() in (".safetensors", ".bin", ".pt", ".pth",
+                                   ".ckpt", ".gguf") and _os.path.isfile(p):
+                    return _os.path.getsize(p) / 1e9
+            from codai.models.cache import is_huggingface_model_id
+            for cand in (cfg.get("path"), resolved_name, model_key):
+                p = str(cand or "")
+                repo = p.split(":", 1)[1] if ":" in p else p
+                if not repo or not is_huggingface_model_id(repo):
+                    continue
+                n = self._hf_cached_model_size_bytes(repo)
+                if n and n > 0:
+                    return n / 1e9
+        except Exception:
+            return 0.0
+        return 0.0
+
     def _effective_quant_multiplier(self, cfg: dict,
                                     load_in_4bit: bool, load_in_8bit: bool,
                                     model_key: str = None,
@@ -5369,6 +5403,39 @@ class MultiModelManager:
         except (TypeError, ValueError):
             _explicit_val = None
         if _explicit_val is not None and _explicit_val > 0:
+            # `used_vram_gb` is a STORAGE-size baseline: the quant/precision
+            # factors are applied ON TOP of it. So a value below the weights
+            # actually on disk is scaled DOWN again, and the error compounds in
+            # the one direction that is dangerous -- the load proceeds and then
+            # fails, instead of being refused or making room.
+            #
+            # It is not always a number anyone checked. The post-download
+            # backfill writes `cached_size * 1.2`, and it runs from the download
+            # thread, so an interrupted or still-running fetch leaves a figure
+            # for a fraction of the checkpoint. Measured on zeiss 2026-10-30:
+            # Qwen/Qwen3.5-9B carried used_vram_gb=7.795 for a 19 GB bf16
+            # checkpoint (6.5 GB downloaded x 1.2), and with load_in_4bit the
+            # estimate came out 7.795 x 0.300 + reserve = 3.13 GB. The loader
+            # read 5.6 GB free, concluded it fitted three times over, tried a
+            # full-GPU load of a model needing ~6.5 GB, and failed -- reporting
+            # "needs about 3.1 GB ... 5.6 GB is free", which reads like a bug in
+            # the eviction rather than in the number.
+            #
+            # The checkpoint on disk is measurable fact, available right here,
+            # so it is used as a floor. Only ever upward: a configured value
+            # that is already larger is the operator's, and this must not
+            # quietly shrink it (the whole point of the field is that a
+            # configured number stays put).
+            _floor = self._disk_weights_gb(model_key, resolved_name, cfg)
+            if _floor and _floor > _explicit_val * 1.25:
+                print(f"  used_vram_gb={_explicit_val:.3f} GB for '{model_key}' is "
+                      f"well under the {_floor:.2f} GB of weights on disk — sizing "
+                      f"against the disk instead (a value written before the "
+                      f"download finished scales DOWN under quantization)",
+                      flush=True)
+                return _dbg_est("used_vram_gb(disk-floor)",
+                                _apply_factors(_floor)
+                                + self._config_lora_vram_gb(cfg))
             return _dbg_est("used_vram_gb",
                             _apply_factors(_explicit_val) + self._config_lora_vram_gb(cfg))
         if _explicit_val is not None and _explicit_val <= 0 and (measured or 0) > 0:

tests/test_used_vram_gb_disk_floor.py

+"""A VRAM estimate came out three times under the truth, downward.
+
+Measured on zeiss 2026-10-30, from the serving-failure watch:
+
+    Auto: Qwen/Qwen3.5-9B is quantized — trying full-GPU regardless of the
+          3.1 GB estimate (5.6 GB free)
+    Error loading model Qwen/Qwen3.5-9B: … needs about 3.1 GB of VRAM to
+          itself; 5.6 GB is free after evicting what could be evicted.
+
+which reads like a bug in eviction — 5.6 is more than 3.1 — and is not. The
+estimate was wrong, and wrong in the dangerous direction: too SMALL, so the
+loader believed it fitted three times over, attempted a full-GPU load of a model
+needing ~6.5 GB, and failed.
+
+The chain, from the debug line:
+
+    source=used_vram_gb -> 3.13 GB | used_vram_gb=7.795 measured_cfg=8.67
+    load_in_4bit=True prequantized=False quant_mult=0.300
+
+* `measured_vram_gb=8.67` (the real 4-bit footprint) was correctly DISCARDED:
+  it was recorded under a different quantization signature. That guard exists
+  because a stale measurement once forced CPU offload on a card the model fit.
+* It fell back to `used_vram_gb=7.795`, which is a STORAGE-size baseline that
+  the quant factors are applied on top of: 7.795 x 0.300 + reserve = 3.13.
+* But the checkpoint is 19.31 GB of bf16 on disk with `quantization_config:
+  None` — verified, so `prequantized=False` is right and the 0.300 multiplier
+  is right. 7.795 was simply not the size of anything: the post-download
+  backfill writes `cached_size * 1.2`, and 6.5 GB downloaded x 1.2 = 7.8.
+  The figure described a partial download.
+
+So the field can hold a number nobody checked, and the factors then scale it
+down again. The checkpoint on disk is measurable fact, available at the moment
+of the estimate, and is now used as a floor -- upward only, so a configured
+value that is already larger stays the operator's.
+"""
+
+import io
+import os
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from codai.models.manager import MultiModelManager
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+MGR = io.open(os.path.join(ROOT, 'codai/models/manager.py'), encoding='utf-8').read()
+
+
+def _bare():
+    """A manager without __init__: only the pure helper is exercised."""
+    return MultiModelManager.__new__(MultiModelManager)
+
+
+# ── the helper reads what is really on disk ──────────────────────────────────
+
+def test_a_local_weights_file_is_measured(tmp_path):
+    f = tmp_path / "model.safetensors"
+    f.write_bytes(b"\0" * 2_000_000_000)
+    gb = MultiModelManager._disk_weights_gb(_bare(), "m", "", {"path": str(f)})
+    assert 1.9 < gb < 2.1, gb
+
+
+def test_an_unknown_model_gets_no_opinion():
+    """0.0 means "no opinion", which must leave the configured value alone --
+    the alternative is a floor invented from nothing."""
+    assert MultiModelManager._disk_weights_gb(
+        _bare(), "nope/not-a-real-model-anywhere", "", {}) == 0.0
+
+
+def test_it_never_raises(tmp_path):
+    for cfg in ({}, {"path": None}, {"path": "/no/such/file.safetensors"},
+                {"path": str(tmp_path)}, {"path": 12345}):
+        assert MultiModelManager._disk_weights_gb(_bare(), "k", "", cfg) >= 0.0
+
+
+def test_a_directory_is_not_mistaken_for_weights(tmp_path):
+    assert MultiModelManager._disk_weights_gb(
+        _bare(), "k", "", {"path": str(tmp_path)}) == 0.0
+
+
+def test_an_unrecognised_extension_is_ignored(tmp_path):
+    f = tmp_path / "notes.txt"
+    f.write_bytes(b"\0" * 1000)
+    assert MultiModelManager._disk_weights_gb(
+        _bare(), "k", "", {"path": str(f)}) == 0.0
+
+
+# ── the floor, and its direction ─────────────────────────────────────────────
+
+def test_the_floor_only_moves_the_estimate_up():
+    """The field's purpose is that a configured number stays put. A floor that
+    could lower it would break exactly that."""
+    i = MGR.index('_floor = self._disk_weights_gb(')
+    block = MGR[i:i + 1200]
+    assert '_floor > _explicit_val * 1.25' in block, (
+        "the floor no longer requires the disk size to EXCEED the "
+        "configured value, so it can shrink an operator's figure")
+
+
+def test_the_margin_tolerates_an_honest_difference():
+    """A configured value a little under the disk size is ordinary (weights vs
+    checkpoint, tokenizer files, a shard counted twice). Only a gross mismatch
+    is treated as a bad number."""
+    i = MGR.index('_floor = self._disk_weights_gb(')
+    assert '* 1.25' in MGR[i:i + 1200]
+
+
+def test_it_says_what_it_did_and_why():
+    i = MGR.index('_floor = self._disk_weights_gb(')
+    block = MGR[i:i + 1600]
+    assert 'well under the' in block and 'weights on disk' in block
+    assert 'before the' in block and 'download finished' in block
+
+
+def test_the_source_is_distinguishable_in_the_debug_line():
+    """`source=` is how this subsystem is read; a floor that reported itself as
+    an ordinary used_vram_gb estimate would be invisible."""
+    assert '"used_vram_gb(disk-floor)"' in MGR
+
+
+def test_the_factors_are_still_applied_to_the_floor():
+    """The disk size is a STORAGE size: a 4-bit load of a 19 GB bf16 checkpoint
+    needs ~5.8 GB, not 19. Skipping the factors would over-estimate by 3x and
+    evict the whole card."""
+    i = MGR.index('"used_vram_gb(disk-floor)"')
+    assert '_apply_factors(_floor)' in MGR[i:i + 300]
+
+
+def test_the_ordinary_path_is_unchanged():
+    i = MGR.index('return _dbg_est("used_vram_gb",')
+    assert '_apply_factors(_explicit_val)' in MGR[i:i + 200]
+
+
+def test_the_measured_path_still_wins_and_still_skips_factors():
+    """A real measurement already reflects quant/precision/offload. This change
+    must not touch that, nor the signature guard that discards a measurement
+    taken under different quantization."""
+    assert 'Ignoring measured_vram_gb' in MGR
+    assert '_dbg_est("measured"' in MGR
+
+
+# ── the real case, if the checkpoint is present ──────────────────────────────
+
+def test_the_incident_model_now_sizes_from_its_checkpoint():
+    gb = MultiModelManager._disk_weights_gb(
+        _bare(), "Qwen3.5-9B", "Qwen/Qwen3.5-9B", {"path": "Qwen/Qwen3.5-9B"})
+    if not gb:
+        pytest.skip("Qwen/Qwen3.5-9B is not in this host's HF cache")
+    assert gb > 15.0, f"expected a ~19 GB bf16 checkpoint, got {gb:.2f}"
+    assert gb > 7.795 * 1.25, "the floor would not have fired on the real case"
+    # 4-bit of ~19 GB is ~5.8 GB, which is MORE than the 5.6 GB that was free --
+    # so the load is refused or makes room, instead of being attempted and
+    # failing on a 3.13 GB estimate.
+    assert gb * 0.30 > 5.6 * 0.95