• Stefy Lanza (nextime / spora )'s avatar
    make a per-model setting reachable by the name it was set on · 8105c605
    Stefy Lanza (nextime / spora ) authored
    The sequential text-encoder offload reported itself "not enabled" on a card
    that plainly had `sequential_text_encoder = True`. Four defects, stacked so
    each one hid the next; the first is much wider than the feature that exposed
    it.
    
    _resolve_config_key could not resolve ANY non-text model by alias. Typed
    entries are keyed "<kind>:<name>", so the exact-id and alias-map lookups both
    miss, and the basename fallback misses too: "video:LTX-2.3-PinkCherry" has no
    slash, so basename() hands back the whole prefixed string. Every per-model
    image / video / audio / tts / embedding setting was therefore replaced by the
    global default, silently, for any model addressed by its alias -- which is the
    only name that distinguishes two configs sharing one checkpoint path. The
    resolver now tries the typed keys, ordered by the types the model is actually
    registered as. images.py (6 sites) and tts.py looked their config up with a
    raw config.get(model_key) and had the same bug; they resolve now, with a test
    that fails if a raw one reappears.
    
    The flag itself lived only in _raw_cfg, where a consumer reading
    config['sequential_text_encoder'] never saw it. Promoted to a top-level
    runtime kwarg, as gpu_split beside it already documents.
    
    With the gate finally firing, the render failed: "Must provide
    `prompt_attention_mask` when specifying `prompt_embeds`". encode_prompt's
    return order had been guessed. Read off the installed sources, LTX-2 returns
    (embeds, mask, neg_embeds, neg_mask) and Krea 2 returns (embeds, mask) --
    the mask comes SECOND, and the old code read it third, filing negative
    embeddings as the attention mask. Naming is now done by classifying the
    values (a mask is an integer tensor, embeddings are floating) instead of by
    position.
    
    That rewrite then dropped Krea 2's negative prompt, which would have been
    worse than the failure it replaced: Krea 2's __call__ encodes "" itself when
    handed no negative, against an encoder this function has just moved to the
    CPU. The negative is always encoded up front for those families now.
    
    Hardened for families nobody has tried: an SDXL-shaped pipeline returns four
    float tensors including the pooled pair, and naming only two does not degrade
    quietly -- the pipeline rejects incomplete embeddings and the request fails.
    If everything encode_prompt returned cannot be named, the optimisation is
    abandoned for that request and says so.
    
    Measured on LTX-2.3-PinkCherry (480x480, 25 frames, 8 steps, RTX 3090):
    
        without   369.4 s   peak 23718 MiB   91% util
        with      345.3 s   peak 24098 MiB   92% util   7.3 GiB freed
    
    Read as headroom, not speed: the card is 24 GiB and the denoiser fills what
    it is given, so peak sits at the ceiling either way and the 24 s is one
    sample. The 7.3 GiB is what stops these renders being pushed to CPU offload,
    where the same job took tens of minutes.
    
    Also here: a clip-audio scene no longer needs an explicit duration (B1 said
    the duration comes from the clip, and validation refused exactly the request
    the feature exists for), and _free now reports per component whether it moved,
    was already off the GPU, or REFUSED the move -- a bnb-quantized encoder can
    refuse, and all three used to arrive as "nothing was freed".
    
    test_sequential_text_encoder's fakes returned bare strings in the wrong tuple
    order, which is why the bug shipped: the test encoded the mistaken belief
    rather than the pipeline's contract. They carry dtypes and the real orders
    now. test_compose's engine-selection test patches shutil.which as well,
    because the production image ships wav2lip and sadtalker shims on PATH and the
    test was asserting the absence of a binary the real deployment has.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    8105c605
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-acestep.txt Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-latentsync.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-musetalk.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...