A load is admitted to VRAM by estimate: ensure_vram_for sizes the model, evicts
what is in the way, and failing that a CUDA OOM is caught and reported. Host RAM
had no equivalent, and the kernel supplies its own: four engines SIGKILLed in
five hours on a 54 GB host, each one while a model staged its weights.

    Oct 29 23:03  Killed coderai-nvidia-gguf   anon-rss 46.3 GB
    Oct 30 03:10  Killed coderai-nvidia        anon-rss 47.7 GB
    Oct 30 03:36  Killed coderai-nvidia        anon-rss 31.6 GB
    Oct 30 03:49  Killed coderai-nvidia        anon-rss 32.7 GB
    front: engine#0 exited (code -9); respawning (1/5 in 120s)

The RAM watcher was awake for all of it and logged "RSS 50.6/43.0 GB (118%) --
mitigation [gc+trim+evict_idle] -> 53.7 GB": every rung of its ladder releases
memory already held, and none of them can decline the load doing the allocating.
SIGKILL then takes the engine rather than the request, so the client that asked
for the oversized model saw a dropped connection -- and so did every unrelated
request in flight, plus everything arriving during the 120-second respawn. One
bad load reads as the whole server being unreliable.

So offload.ram_admission (default on) adds the missing half. codai/models/
host_ram.py sizes a pending load -- measured host residency if the model has
loaded before, the whole footprint for an offload strategy that asks for it,
the spill past free VRAM otherwise, plus a staging allowance -- against the
tighter of MemAvailable and the headroom under max_ram_gb, less a floor kept
for the OS and the sibling engines. ensure_host_ram_for() runs the existing
pressure ladder FIRST and re-measures, so refusal is the last step; when it
comes, it is a 503 naming the model, the estimate, the budget, the shortfall
and what to change on the card. A model whose footprint cannot be determined
is never refused on a number nobody has.

And the proximate cause of the biggest of those kills was a sizing step that
reported itself as skipped:

    (auto-offload sizing skipped: cannot access local variable '_ck'
     where it is not associated with a value)

The GGUF auto-offload branch reads the configured KV-cache types to size its
headroom, from a binding made ~110 lines further down where llama_kwargs is
assembled. Every load that reached the branch raised UnboundLocalError, the
handler called it "skipped", and nothing reduced n_gpu_layers -- so llama.cpp
was asked to keep every layer of a 35B on a card that could not hold them, and
answered with a 43,562 MiB CPU KV buffer. The read is hoisted above its use
(tested on source order, not a line number), and the handler now says the load
is UNSIZED, because "skipped" invites the reader to assume nothing needed doing.

Separately, and found while checking which cards were really misconfigured:
a repo whose model_index.json declares a pipeline class diffusers does not have
is now detected and loaded with its own code when allow-listed, or refused by
name when not. The allow-list and the mechanism both already existed, but only
fired when a card named a custom_pipeline -- nothing ever did, so MOSS-Image
and Kromoss failed with "module diffusers has no attribute MOSSImagePipeline",
which names neither the cause nor the remedy and reads as a broken model.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC