The share learner proposed 0.900 -> 0.460 the moment qwen38 booted on an A40:
"13.27 GiB of cache on a 0.900 share leaves room to spare; 0.460 would still
give ~2.0 GiB — using it from the next start". 2 GiB of this model's cache is
~32,000 tokens, measured from its own boot report (13.27 GiB bought 212,321
tokens), and the engine was serving a 131,072-token window. That is not one
full request, and a vLLM whose cache cannot hold max_model_len refuses to start
at all -- so the next start of that model would have failed, climbed the
escalation ladder over several six-minute boots, or come up unable to serve the
window it advertises.

The flat 2 GiB target is right for a page-at-a-time VLM, which is what it was
measured for. It is now a FLOOR, raised to whatever holds two full-length
requests of the window this engine is actually serving. Bytes-per-token come
from the boot report rather than the config, because they depend on the model's
attention shape -- this one runs linear attention on most of its layers, at
~64 KiB/token -- and the token count is captured as it streams, like the GiB
figure beside it, since both scroll out of the tail before the server is ready.

With no report to read the flat target stands: a floor invented from a guess
would refuse to shrink anything for ever, which is the waste this feature
exists to prevent. A VLM serving 4096-token pages still hands its memory back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66