• Stefy Lanza (nextime / spora )'s avatar
    read the CLIP layer the caller asked for (v0.4.92) · 8f1fde51
    Stefy Lanza (nextime / spora ) authored
    clip_skip was mapped onto diffusers' argument of the same name, and that
    argument counts something else. Its SDXL path reads
    
        hidden_states[-2]                 # clip_skip is None
        hidden_states[-(clip_skip + 2)]   # otherwise
    
    so diffusers' clip_skip is the number of layers skipped BEYOND the penultimate
    one SDXL already uses. Every model card, A1111, ComfyUI and InvokeAI mean
    something simpler by the name: 1 is the final layer, 2 the penultimate, n the
    n-th from the end. So a request asking for 2 -- which is what the booru-tag
    SDXL families are trained at, and what their model cards all say -- read
    hidden_states[-4] instead: two layers too deep, the equivalent of clip-skip 4,
    which on a Pony-family model is the difference between a photograph and mush.
    It was inconsistent as well as wrong: the same 2 meant [-4] on SDXL and [-3] on
    SD 1.x, two depths from one number.
    
    The mapping now happens here, once, in the sense users mean, and clip_skip 2
    reaches the same layer whichever family the model belongs to. "Mirrors
    diffusers exactly" was the mistake in 0.4.89: mirroring is the wrong goal for a
    field a caller fills in.
    
    SDXL's pooled conditioning vector is also handled properly now. The UNet cannot
    run without it and diffusers refuses prompt_embeds that arrive without one, so
    handing it half the set is a ValueError raised in check_inputs -- an instant
    500, before any denoising. Where a pipeline does not offer the vector, it is
    rebuilt the way CLIP produces it (pooled at the end-of-text token, then
    projected through text_projection) rather than giving up the weighting and the
    window count this module exists for; only when even that is impossible does the
    encoder stand down and let the pipeline encode the prompt itself. The invalid
    combination is no longer constructible: either all four keys or None.
    
    A 500 from /v1/images/edits, /inpaint or /outfit now logs its traceback. The
    cause used to reach the caller alone, so the log showed the request and then
    the status, and a failure there could not be diagnosed afterwards at all.
    
    Test note: test_a_400_is_not_relabelled_as_a_server_fault scanned a fixed
    250-character window for the HTTPException pass-through above each catch-all;
    the new log line pushed that to 316-382 and broke a passing check on correct
    code. It now asserts structure instead of proximity -- the pass-through must be
    the nearest handler inside the same function, with the logging between it and
    the raise -- which also catches it drifting into a different function.
    
    58 tests in the prompt file; full suite 4521 passed with the same 15
    pre-existing failures as 0.4.91 (diff empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    8f1fde51
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
audiocraft_worker.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
chunking.py Loading commit data...
colibri_worker.py Loading commit data...
compose.py Loading commit data...
context_overflow.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
host_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lipsync.py Loading commit data...
log.py Loading commit data...
longcat_worker.py Loading commit data...
loop_watch.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
model_test.py Loading commit data...
models_transfer.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
pod_http.py Loading commit data...
pod_telemetry.py Loading commit data...
pod_tls.py Loading commit data...
podlog.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_fleet_share.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
serving_load.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
thread_pool.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
uploads.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...
xtts_worker.py Loading commit data...