a model card could cross the line the request path refuses to

_apply_sampler turns a cross-family sampler into a 400 that explains itself:
flow-matching and DDPM schedulers parameterise the trajectory differently, so
one cannot stand in for the other. The acceleration config applied its
`scheduler` with no such check, and a card is exactly where that mistake gets
made and never noticed.

Krea-2-Turbo's entry asked for EulerDiscreteScheduler on a flow-matching
model. The swap succeeded silently and the generation then died with

    EulerDiscreteScheduler.set_timesteps() got an unexpected keyword
    argument 'mu'

which names neither the card nor the model, arrives from inside diffusers, and
reads like a library bug. Now it is refused and logged, naming both
schedulers, and the model keeps its own -- acceleration is an optimisation and
has always been allowed to decline.

One set of family names, imported by both paths, so they cannot drift.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
parent fb53791e
...@@ -445,10 +445,9 @@ SAMPLERS = { ...@@ -445,10 +445,9 @@ SAMPLERS = {
#: Schedulers for flow-matching models. A pipeline whose scheduler is one of #: Schedulers for flow-matching models. A pipeline whose scheduler is one of
#: these cannot be given a classic DDPM-family sampler, and vice versa. #: these cannot be given a classic DDPM-family sampler, and vice versa.
_FLOW_SCHEDULERS = {"FlowMatchEulerDiscreteScheduler", #: Shared with the model-card path, which applies a scheduler from the
"FlowMatchHeunDiscreteScheduler", #: acceleration config and must draw the family line in the same place.
"FlowMatchLCMScheduler", from codai.models.acceleration import FLOW_SCHEDULERS as _FLOW_SCHEDULERS
"LTXEulerAncestralRFScheduler"}
def _apply_sampler(pipeline, sampler: str) -> str: def _apply_sampler(pipeline, sampler: str) -> str:
......
...@@ -225,6 +225,17 @@ def _split_lora_ref(ref: str): ...@@ -225,6 +225,17 @@ def _split_lora_ref(ref: str):
return ref, None return ref, None
#: Schedulers that parameterise a flow-matching trajectory. Swapping one of
#: these for a DDPM-family scheduler (or the reverse) does not make a worse
#: picture, it makes noise -- so both the request path (images._apply_sampler)
#: and the model-card path below refuse to cross the line. One set, so they
#: cannot disagree about where it is.
FLOW_SCHEDULERS = {"FlowMatchEulerDiscreteScheduler",
"FlowMatchHeunDiscreteScheduler",
"FlowMatchLCMScheduler",
"LTXEulerAncestralRFScheduler"}
def apply_accel_to_pipeline(pipe, accel: Optional[dict]) -> None: def apply_accel_to_pipeline(pipe, accel: Optional[dict]) -> None:
"""Fuse the distill LoRA into a diffusers pipeline and apply scheduler tweaks. """Fuse the distill LoRA into a diffusers pipeline and apply scheduler tweaks.
...@@ -252,6 +263,21 @@ def apply_accel_to_pipeline(pipe, accel: Optional[dict]) -> None: ...@@ -252,6 +263,21 @@ def apply_accel_to_pipeline(pipe, accel: Optional[dict]) -> None:
import diffusers as _diffusers import diffusers as _diffusers
sched_cls = getattr(_diffusers, sched_name, None) sched_cls = getattr(_diffusers, sched_name, None)
cur = getattr(pipe, "scheduler", None) cur = getattr(pipe, "scheduler", None)
cur_name = type(cur).__name__ if cur is not None else ""
if (sched_cls is not None and cur is not None
and (cur_name in FLOW_SCHEDULERS)
!= (sched_name in FLOW_SCHEDULERS)):
# The request path refuses this swap with an explanation; a
# model card could do it silently, and the model then fails at
# call time with something that names neither scheduler --
# Krea 2 with EulerDiscreteScheduler configured here answered
# "EulerDiscreteScheduler.set_timesteps() got an unexpected
# keyword argument 'mu'", which reads like a diffusers bug.
log.warning(
"[accel] ignoring scheduler %s: this model's scheduler is "
"%s, and flow-matching and DDPM-family samplers cannot be "
"substituted for one another", sched_name, cur_name)
sched_cls = None
if sched_cls is not None and cur is not None: if sched_cls is not None and cur is not None:
# `scheduler_config` carries the sampler's own settings, because # `scheduler_config` carries the sampler's own settings, because
# the class name alone does not identify a sampler. The samplers # the class name alone does not identify a sampler. The samplers
......
"""A model card must not be able to cross the flow/DDPM line either.
The request path (`images._apply_sampler`) refuses a cross-family sampler with
an explanation. The acceleration config had no such guard, so a card could
swap a flow-matching model's scheduler for a DDPM one silently — and the model
then failed at call time with
EulerDiscreteScheduler.set_timesteps() got an unexpected keyword
argument 'mu'
which names neither the card nor the model and reads like a diffusers bug.
Measured on Krea-2-Turbo, whose entry asked for EulerDiscreteScheduler.
"""
import logging
import pytest
from codai.models import acceleration as accel_mod
class _Sched:
def __init__(self, name, cfg=None):
self.__class__.__name__ = name
self.config = cfg or {}
class _FlowSched:
config = {"num_train_timesteps": 1000}
class _Pipe:
def __init__(self, scheduler):
self.scheduler = scheduler
def test_the_two_paths_share_one_definition_of_the_families():
"""Two copies of this set would drift, and the drift is invisible until a
model produces noise."""
from codai.api import images
assert images._FLOW_SCHEDULERS is accel_mod.FLOW_SCHEDULERS
assert "FlowMatchEulerDiscreteScheduler" in accel_mod.FLOW_SCHEDULERS
def test_a_ddpm_scheduler_is_refused_on_a_flow_matching_model(caplog):
"""Krea 2's own scheduler is FlowMatchEulerDiscreteScheduler; the card
asked for EulerDiscreteScheduler."""
pipe = _Pipe(_FlowSched())
type(pipe.scheduler).__name__ = "FlowMatchEulerDiscreteScheduler"
with caplog.at_level(logging.WARNING):
accel_mod.apply_accel_to_pipeline(
pipe, {"scheduler": "EulerDiscreteScheduler", "lora": None})
assert type(pipe.scheduler).__name__ == "FlowMatchEulerDiscreteScheduler"
assert any("cannot be substituted" in r.getMessage() for r in caplog.records)
def test_a_same_family_scheduler_is_still_applied():
"""LCM on an SD model is the reason this feature exists."""
pipe = _Pipe(_FlowSched())
type(pipe.scheduler).__name__ = "EulerDiscreteScheduler"
accel_mod.apply_accel_to_pipeline(
pipe, {"scheduler": "LCMScheduler", "lora": None})
assert type(pipe.scheduler).__name__ == "LCMScheduler"
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment