Two models could not be loaded in one afternoon, and establishing why took a
diagnosis each. Neither was a memory problem; both failed late and expensively,
after other models had been turned out of the card to make room.

The second one is the reason for this. A 27B "Flash" quant passed every check a
careful operator would make -- right architecture, all 65 blocks present
including the MTP one, an mmproj beside it -- and stored 402 of its 866 tensors
in ggml type 143, a rotation-folded format from a fork. Nothing in the metadata
announces that: the type ids live in the tensor index, which no error message
reads back to you. It arrived as a generic load failure after the eviction, six
retries and a full read of the weights.

gguf_preflight answers it from the header alone and runs before the VRAM need is
even computed, so a load that was never going to work costs no other model its
residency. The bound is llama.cpp's own GGML_TYPE_COUNT rather than a list kept
here, so it tracks the installed library.

Only ids at or above that bound are refused, and the conservatism is the design.
The Python binding's GGML_TYPE_* constants are an INCOMPLETE mirror of the
compiled enum -- 0-3, 6-29, 39-42 -- so 30..38 have no constant, and 30 is BF16,
which files we serve today use and which loads perfectly. "Absent from the
binding's constants" would have rejected every one of them. An id inside the
enum that the build still cannot handle is left to llama.cpp to refuse in its own
words.

The verdict gets its own exception type because the on-demand path turns a failed
VRAM reservation into a printed warning and carries on -- correctly for a
reservation, fatally for a verdict, which would then reappear a minute later as
the original confusing failure. That path now re-raises GgufUnreadable, the same
treatment HostRamExhausted already gets, and the comment three lines below it
("a host-RAM refusal is the decision itself and must reach the caller") is why.
It reports 501, not 503: evicting and retrying cannot change the answer, so a
client that retries a 503 forever must not be invited to. The type ids ride along
in the response, being the actionable part.

The architecture is printed before the load too, so llama.cpp's "unknown model
architecture" sits beside the file it came from. Deliberately NOT a list of
supported architectures: there is no API for it, and guessing from the shared
library's strings would invent a second source of truth that can be wrong in the
direction that blocks a model which would have worked.

No fourth GGUF parser -- the tensor-type histogram rides along on the walk
gguf_mtp.read_layout already does, and a test asserts gguf_preflight has no
parser of its own. docs/serving-a-model-this-box-cannot-hold.md records the three
independent requirements, how to verify each on a new machine, the quant/size
table, why an mtp-*.gguf is not a model, and which of the three things called
"MTP support" already exist (vLLM's raw --speculative-config passthrough, ds4's
--mtp) versus which does not (a GGUF draft model, which llama-cpp-python cannot
express).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC