-
Stefy Lanza (nextime / spora ) authored
`precision` was read by the loader and listed in main.py's _LOAD_AFFECTING, but it was in neither the admin save allowlist nor the model page, so the one setting that decides how much VRAM a card actually needs could not be configured anywhere. The dtype then came from a default derived from GPU detection -- and a card's precision is a property of the card, not of the hardware it happens to land on. The select sits with the quantization boxes because that is where it is read wrongly: 4-bit and 8-bit quantize the WEIGHTS, while the embeddings, lm_head, the norms, the activations and the KV cache all stay in this dtype, so a 4-bit model left in fp32 still pays fp32 for every one of them. The hint says so, and says that fp32 also rules out Flash Attention. "" is stored as null, so "not configured" stays distinguishable from a value and the loader's own default still applies. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
be14830d