• Stefy Lanza (nextime / spora )'s avatar
    read the CLIP layer the caller asked for (v0.4.92) · 8f1fde51
    Stefy Lanza (nextime / spora ) authored
    clip_skip was mapped onto diffusers' argument of the same name, and that
    argument counts something else. Its SDXL path reads
    
        hidden_states[-2]                 # clip_skip is None
        hidden_states[-(clip_skip + 2)]   # otherwise
    
    so diffusers' clip_skip is the number of layers skipped BEYOND the penultimate
    one SDXL already uses. Every model card, A1111, ComfyUI and InvokeAI mean
    something simpler by the name: 1 is the final layer, 2 the penultimate, n the
    n-th from the end. So a request asking for 2 -- which is what the booru-tag
    SDXL families are trained at, and what their model cards all say -- read
    hidden_states[-4] instead: two layers too deep, the equivalent of clip-skip 4,
    which on a Pony-family model is the difference between a photograph and mush.
    It was inconsistent as well as wrong: the same 2 meant [-4] on SDXL and [-3] on
    SD 1.x, two depths from one number.
    
    The mapping now happens here, once, in the sense users mean, and clip_skip 2
    reaches the same layer whichever family the model belongs to. "Mirrors
    diffusers exactly" was the mistake in 0.4.89: mirroring is the wrong goal for a
    field a caller fills in.
    
    SDXL's pooled conditioning vector is also handled properly now. The UNet cannot
    run without it and diffusers refuses prompt_embeds that arrive without one, so
    handing it half the set is a ValueError raised in check_inputs -- an instant
    500, before any denoising. Where a pipeline does not offer the vector, it is
    rebuilt the way CLIP produces it (pooled at the end-of-text token, then
    projected through text_projection) rather than giving up the weighting and the
    window count this module exists for; only when even that is impossible does the
    encoder stand down and let the pipeline encode the prompt itself. The invalid
    combination is no longer constructible: either all four keys or None.
    
    A 500 from /v1/images/edits, /inpaint or /outfit now logs its traceback. The
    cause used to reach the caller alone, so the log showed the request and then
    the status, and a failure there could not be diagnosed afterwards at all.
    
    Test note: test_a_400_is_not_relabelled_as_a_server_fault scanned a fixed
    250-character window for the HTTPException pass-through above each catch-all;
    the new log line pushed that to 316-382 and broke a passing check on correct
    code. It now asserts structure instead of proximity -- the pass-through must be
    the nearest handler inside the same function, with the logging between it and
    the raise -- which also catches it drifting into a different function.
    
    58 tests in the prompt file; full suite 4521 passed with the same 15
    pre-existing failures as 0.4.91 (diff empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    8f1fde51
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...