• Stefy Lanza (nextime / spora )'s avatar
    C5: encode the prompt, then give the card back before denoising · 65844801
    Stefy Lanza (nextime / spora ) authored
    A text encoder runs ONCE per request, before the first denoising step, and is
    dead weight for every step after -- but it stays resident and keeps its VRAM.
    Measured on the cached snapshots, that is not a rounding error: Z-Image's
    encoder is 7.5 GiB of a 19.2 GiB pipeline, and LTX-2.3's 4-bit Gemma-3-12B is
    ~6.5 GiB in front of a 15 GiB transformer. PinkCherry declaring exactly 24.0 GB
    on a 24 GiB card is that sum, and it is why every LTX render was pushed to CPU
    offload.
    
    Encoding first and freeing the encoder hands that VRAM to the denoiser, so a
    model that "did not fit" fits. This is the sequential text-encoder offload the
    requirements asked for; it existed only as a side effect before, because
    diffusers' enable_model_cpu_offload happens to move each component out after
    use -- so the OFFLOADED path had it and the resident path did not.
    
    The embedding argument names differ per family and there is no common base
    class to ask, so they are read from the pipeline's own signature: Krea 2 takes
    prompt_embeds + prompt_embeds_mask and needs a second call for the negative,
    LTX-2 takes prompt_attention_mask and encodes both at once. A family that takes
    no embeddings at all is left alone rather than having its encoder freed with no
    way to pass the prompt back.
    
    Opt-in per model and settable from the model page, because freeing the encoder
    costs a host-RAM copy and only pays where the card is tight. The encoder is
    moved to CPU rather than deleted: the pipeline is cached and the next request
    needs it again. Any failure returns the request untouched -- a VRAM
    optimisation must never be the reason a generation fails, and a test holds that.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    65844801
Name
Last commit
Last update
..
static Loading commit data...
templates Loading commit data...
__init__.py Loading commit data...
auth.py Loading commit data...
download_worker.py Loading commit data...
routes.py Loading commit data...