The estimator asked a 24 GB card for 74.3 GB and evicted every other model on
the box before each generation. It sizes a model by summing its whole HF cache
entry, and LongCat's holds three things that are never all resident: the bf16
DiT (51 GB, stored fp32), the INT8 DiT built beside it (13.6 GB), and an fp32
UMT5-XXL text encoder (22 GB).

_longcat_resident_gb now counts only the DiT that `variant` selects, scales each
component from its OWN stored dtype — one ratio cannot serve both an INT8 DiT
that loads as it is stored and an fp32 encoder cast to bf16 — and, when offload
is on, takes max(DiT, encoder) + VAE, because they take turns on the card. The
dtype comes from the safetensors header, which is one small read per component
and never touches the weights. Returning 0.0 leaves an unsizable model to the
generic path rather than claiming it needs nothing.

Measured against the real checkpoint: 13.9 GB for int8+offload, where the run
actually used 13.6 GB. It was 74.3 GB.

And accelerate, which upstream does not pin: without it transformers falls back
to low_cpu_mem_usage=False and materialises the whole fp32 encoder in host RAM
before casting it to bf16. Pinned in requirements-longcat.txt and asserted by
BOTH the pod image build and the local image's smoke test, so a rebuild cannot
quietly drop it.

What accelerate does NOT fix, measured after installing it: the load time. The
5 shards still take ~63 s each, because /AI is ext4 on a spinning disk that
reads at 28 MB/s cold — 22 GB of fp32 is ~5 minutes of pure I/O. accelerate
removes the host-RAM spike, not the wait. Halving the bytes read (a bf16 copy of
the encoder) or moving the checkpoint to NVMe is what would shorten it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w