• Stefy Lanza (nextime / spora )'s avatar
    read the CLIP layer the caller asked for (v0.4.92) · 8f1fde51
    Stefy Lanza (nextime / spora ) authored
    clip_skip was mapped onto diffusers' argument of the same name, and that
    argument counts something else. Its SDXL path reads
    
        hidden_states[-2]                 # clip_skip is None
        hidden_states[-(clip_skip + 2)]   # otherwise
    
    so diffusers' clip_skip is the number of layers skipped BEYOND the penultimate
    one SDXL already uses. Every model card, A1111, ComfyUI and InvokeAI mean
    something simpler by the name: 1 is the final layer, 2 the penultimate, n the
    n-th from the end. So a request asking for 2 -- which is what the booru-tag
    SDXL families are trained at, and what their model cards all say -- read
    hidden_states[-4] instead: two layers too deep, the equivalent of clip-skip 4,
    which on a Pony-family model is the difference between a photograph and mush.
    It was inconsistent as well as wrong: the same 2 meant [-4] on SDXL and [-3] on
    SD 1.x, two depths from one number.
    
    The mapping now happens here, once, in the sense users mean, and clip_skip 2
    reaches the same layer whichever family the model belongs to. "Mirrors
    diffusers exactly" was the mistake in 0.4.89: mirroring is the wrong goal for a
    field a caller fills in.
    
    SDXL's pooled conditioning vector is also handled properly now. The UNet cannot
    run without it and diffusers refuses prompt_embeds that arrive without one, so
    handing it half the set is a ValueError raised in check_inputs -- an instant
    500, before any denoising. Where a pipeline does not offer the vector, it is
    rebuilt the way CLIP produces it (pooled at the end-of-text token, then
    projected through text_projection) rather than giving up the weighting and the
    window count this module exists for; only when even that is impossible does the
    encoder stand down and let the pipeline encode the prompt itself. The invalid
    combination is no longer constructible: either all four keys or None.
    
    A 500 from /v1/images/edits, /inpaint or /outfit now logs its traceback. The
    cause used to reach the caller alone, so the log showed the request and then
    the status, and a failure there could not be diagnosed afterwards at all.
    
    Test note: test_a_400_is_not_relabelled_as_a_server_fault scanned a fixed
    250-character window for the HTTPException pass-through above each catch-all;
    the new log line pushed that to 316-382 and broke a passing check on correct
    code. It now asserts structure instead of proximity -- the pass-through must be
    the nearest handler inside the same function, with the logging between it and
    the raise -- which also catches it drifting into a different function.
    
    58 tests in the prompt file; full suite 4521 passed with the same 15
    pre-existing failures as 0.4.91 (diff empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    8f1fde51
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...