-
Stefy Lanza (nextime / spora ) authored
A text encoder runs ONCE per request, before the first denoising step, and is dead weight for every step after -- but it stays resident and keeps its VRAM. Measured on the cached snapshots, that is not a rounding error: Z-Image's encoder is 7.5 GiB of a 19.2 GiB pipeline, and LTX-2.3's 4-bit Gemma-3-12B is ~6.5 GiB in front of a 15 GiB transformer. PinkCherry declaring exactly 24.0 GB on a 24 GiB card is that sum, and it is why every LTX render was pushed to CPU offload. Encoding first and freeing the encoder hands that VRAM to the denoiser, so a model that "did not fit" fits. This is the sequential text-encoder offload the requirements asked for; it existed only as a side effect before, because diffusers' enable_model_cpu_offload happens to move each component out after use -- so the OFFLOADED path had it and the resident path did not. The embedding argument names differ per family and there is no common base class to ask, so they are read from the pipeline's own signature: Krea 2 takes prompt_embeds + prompt_embeds_mask and needs a second call for the negative, LTX-2 takes prompt_attention_mask and encodes both at once. A family that takes no embeddings at all is left alone rather than having its encoder freed with no way to pass the prompt back. Opt-in per model and settable from the model page, because freeing the encoder costs a host-RAM copy and only pays where the card is tight. The encoder is moved to CPU rather than deleted: the pipeline is cached and the next request needs it again. Any failure returns the request untouched -- a VRAM optimisation must never be the reason a generation fails, and a test holds that. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
65844801
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| static | ||
| templates | ||
| __init__.py | ||
| auth.py | ||
| download_worker.py | ||
| routes.py |