• Stefy Lanza (nextime / spora )'s avatar
    read the CLIP layer the caller asked for (v0.4.92) · 8f1fde51
    Stefy Lanza (nextime / spora ) authored
    clip_skip was mapped onto diffusers' argument of the same name, and that
    argument counts something else. Its SDXL path reads
    
        hidden_states[-2]                 # clip_skip is None
        hidden_states[-(clip_skip + 2)]   # otherwise
    
    so diffusers' clip_skip is the number of layers skipped BEYOND the penultimate
    one SDXL already uses. Every model card, A1111, ComfyUI and InvokeAI mean
    something simpler by the name: 1 is the final layer, 2 the penultimate, n the
    n-th from the end. So a request asking for 2 -- which is what the booru-tag
    SDXL families are trained at, and what their model cards all say -- read
    hidden_states[-4] instead: two layers too deep, the equivalent of clip-skip 4,
    which on a Pony-family model is the difference between a photograph and mush.
    It was inconsistent as well as wrong: the same 2 meant [-4] on SDXL and [-3] on
    SD 1.x, two depths from one number.
    
    The mapping now happens here, once, in the sense users mean, and clip_skip 2
    reaches the same layer whichever family the model belongs to. "Mirrors
    diffusers exactly" was the mistake in 0.4.89: mirroring is the wrong goal for a
    field a caller fills in.
    
    SDXL's pooled conditioning vector is also handled properly now. The UNet cannot
    run without it and diffusers refuses prompt_embeds that arrive without one, so
    handing it half the set is a ValueError raised in check_inputs -- an instant
    500, before any denoising. Where a pipeline does not offer the vector, it is
    rebuilt the way CLIP produces it (pooled at the end-of-text token, then
    projected through text_projection) rather than giving up the weighting and the
    window count this module exists for; only when even that is impossible does the
    encoder stand down and let the pipeline encode the prompt itself. The invalid
    combination is no longer constructible: either all four keys or None.
    
    A 500 from /v1/images/edits, /inpaint or /outfit now logs its traceback. The
    cause used to reach the caller alone, so the log showed the request and then
    the status, and a failure there could not be diagnosed afterwards at all.
    
    Test note: test_a_400_is_not_relabelled_as_a_server_fault scanned a fixed
    250-character window for the HTTPException pass-through above each catch-all;
    the new log line pushed that to 316-382 and broke a passing check on correct
    code. It now asserts structure instead of proximity -- the pass-through must be
    the nearest handler inside the same function, with the logging between it and
    the raise -- which also catches it drifting into a different function.
    
    58 tests in the prompt file; full suite 4521 passed with the same 15
    pre-existing failures as 0.4.91 (diff empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    8f1fde51
Name
Last commit
Last update
..
cache Loading commit data...
__init__.py Loading commit data...
acceleration.py Loading commit data...
capabilities.py Loading commit data...
chat_style.py Loading commit data...
gpu_lock.py Loading commit data...
gpu_query.py Loading commit data...
grammar.py Loading commit data...
hf_loading.py Loading commit data...
manager.py Loading commit data...
parser.py Loading commit data...
peft_compat.py Loading commit data...
pipeline_cache.py Loading commit data...
placement.py Loading commit data...
prompt_weighting.py Loading commit data...
quant.py Loading commit data...
ram_monitor.py Loading commit data...
templates.py Loading commit data...
text_loras.py Loading commit data...
textual_inversion.py Loading commit data...
thermal.py Loading commit data...
tmp_janitor.py Loading commit data...
tool_call_grammar.gbnf Loading commit data...
turboquant.py Loading commit data...
utils.py Loading commit data...
video_geometry.py Loading commit data...
weight_stores.py Loading commit data...