The VRAM bar added in v0.4.67-68 reads ~90% on every pod from the moment it
boots, because that is what it was told to do: vLLM claims its
gpu_memory_utilization share up front and fills what is left after the weights
with KV cache. Measured on an A40, 2026-10-22 -- 23.22 GiB weights + 3.49
activation + 2.61 CUDA graphs + 13.27 KV, out of 48, at utilisation 0.9. A bar
pinned at 90% forever says nothing about load. (The model IS 4-bit; the
quantization is partial by design -- the whole vision tower, lm_head, the
linear_attn projections on ~48 of 63 layers and an MTP head stay FP16, which is
why weights are 23 GiB and not 14.)

What binds instead, and is now what the page shows:

* HOST RAM, as the bar. The measured ceiling: an OCR pod went 43 GB free at 32
  in-flight, 12.95 GB at 64, and was OOM-killed at 128 while the GPU was never
  the constraint. Read from CGROUP accounting first, because inside a container
  /proc/meminfo reports the HOST's memory -- a pod with a 60 GB limit on a 1 TB
  machine looks like it has hundreds of gigabytes free right until the kernel
  kills it.
* KV CACHE occupancy, with running/waiting beside it: the GPU-side headroom,
  and what decides how many more requests can be admitted.
* PREEMPTIONS, called out in red. The proof the cache ran out: vLLM evicts a
  running request and recomputes it later, so throughput collapses while every
  other counter still looks busy. Invisible in utilisation and in VRAM.

Reserved VRAM stays as static text under the RAM bar -- context, not a gauge.

Also removed a fiction: loaded_info reported `vram_gb: 60.58` and `device:
"CPU offload 8.1 GB"` for this model. Nothing measured that -- coderai never
loaded it, vLLM did, so it fell back to a config estimate for an unquantized
load that never happened. A UI claiming 8 GB of CPU offload where there is none
is worse than showing nothing; it now says "served by its own engine".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66