LongCat had two settings that look independent and are not. A quantised text
encoder MUST be resident -- bitsandbytes quantises as weights land on CUDA and
cannot shuttle them back, and _place_pipeline returns early for a quantised
encoder without consulting the offload mode at all. So "quantise the encoder"
and "spill when VRAM runs short" are one decision: asking for both pins 22.1 GB
of a 24 GB card (13.6 DiT + 8.0 nf4 encoder + 0.5 VAE) and leaves under 2 GB for
the activations of a 13.6B video model, which OOMs at the first long clip rather
than at load and so reads as a flaky model.

offload_strategy now accepts "auto", which takes that decision once, at load,
and logs the arithmetic:

  * resident -- DiT, VAE and a quantised encoder all on the card, no swapping;
  * spill    -- DiT and VAE resident, encoder on the host, the two swapping for
                encode_prompt so only one is ever on the card.

The decision is made against the card's CAPACITY -- total VRAM less a context
reserve -- and deliberately not against the free VRAM at that instant. Free VRAM
makes the placement depend on whichever model happened to be resident when the
request arrived: a sibling holding 8 GB would park LongCat on the host, and it
would stay there for every later request, because nothing re-examines the
condition. It is also self-fulfilling -- the worker RESERVES the footprint and
the manager evicts what is in the way, so "no room right now" is not a fact
about the card but about a decision nobody has taken yet. A model that could be
resident is never left on the CPU merely because something else got there first.

What is reserved is what the chosen placement OCCUPIES, not what residency would
need: on a 24 GB card that is 14.1 GB (DiT + VAE), because reserving the 22.1 GB
residency figure would evict siblings to free 8 GB this placement never touches.
Both numbers are reported, so the one the decision was made on stays visible.

The worker and the service run the SAME pure arithmetic over the same inputs, so
they cannot disagree, and the spawn moved inside the acquire loader: it used to
run before the reservation, so the service loaded its weights onto a card the
manager had not cleared yet and the eviction meant to make room happened
afterwards. Fine for a 13.6 GB DiT, wrong for any placement that wants the card.

Every load now logs what it actually cost against what the plan predicted, so a
constant that drifts shows up as a wrong prediction instead of as an OOM at
generation time.

And it is selectable: the model page's Offload list had no "auto" entry, which
mattered more than it looks, because the page sets that select from the saved
value -- a card reading "auto" would have displayed as the first option and the
next save would have silently rewritten it to "". The text-encoder labels now
say which levels need a card bigger than 24 GB, since that is what decides
whether the quantisation is honoured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC