images: stop attention slicing from silently killing every IP-Adapter load

Found by actually running it, not by reading it. Every IP-Adapter load was
failing with

  SlicedAttnProcessor.__init__() missing 1 required positional argument: 'slice_size'

and _ensure_ip_adapter_loaded() caught it, logged a warning and returned False —
so character_profiles fell back to the prompt hint on the ONE path that was
supposed to have real identity conditioning. The image came back looking
plausible, which is why it was never noticed.

Cause: we enable attention slicing for VRAM headroom, and diffusers' adapter
loader rebuilds every attention processor; re-instantiating a SlicedAttnProcessor
without its required slice_size raises. Slicing is now disabled for the load and
left off while the adapter is in use (re-enabling would swap the IP-Adapter
processors straight back out), and restored if the load fails anyway.

Verified: the SlicedAttnProcessor error is gone.

NOT yet working — the next blocker, now visible underneath: on a QUANTIZED
pipeline the load fails with "Casting a quantized model to a new dtype is
unsupported". So IP-Adapter still cannot attach to a 4-bit image model, and
identity conditioning needs an unquantized image model (or a follow-up that loads
the image encoder with an explicit dtype). Do not describe character identity as
working end to end until that is resolved.
Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
parent 279096b6
......@@ -684,15 +684,38 @@ def _ensure_ip_adapter_loaded(pipeline) -> bool:
else:
attempts.append(("h94/IP-Adapter", "models", "ip-adapter_sd15.bin"))
# Attention slicing and IP-Adapter are mutually exclusive in diffusers: the
# adapter loader rebuilds every attention processor, and re-instantiating a
# SlicedAttnProcessor without its required slice_size raises
# SlicedAttnProcessor.__init__() missing 1 required positional argument
# which is what silently killed every IP-Adapter load (we enable slicing for
# VRAM headroom). Turn it off for the load and leave it off while the adapter is
# in use — re-enabling would swap the IP-Adapter processors straight back out.
sliced = False
try:
if getattr(pipeline, '_attn_slicing_enabled', True) and \
hasattr(pipeline, 'disable_attention_slicing'):
pipeline.disable_attention_slicing()
sliced = True
except Exception:
pass
for repo, subfolder, weight_name in attempts:
try:
pipeline.load_ip_adapter(repo, subfolder=subfolder, weight_name=weight_name)
pipeline._coderai_ip_state = 'loaded'
_log.info("IP-Adapter loaded: %s/%s/%s", repo, subfolder, weight_name)
_log.info("IP-Adapter loaded: %s/%s/%s%s", repo, subfolder, weight_name,
" (attention slicing disabled for it)" if sliced else "")
return True
except Exception as e:
_log.warning("IP-Adapter load failed (%s/%s): %s", repo, weight_name, e)
if sliced: # restore the previous memory setting — the adapter isn't loaded
try:
pipeline.enable_attention_slicing()
except Exception:
pass
try:
pipeline._coderai_ip_state = 'unsupported'
except Exception:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment