edit the image the caller sent, not a new one (v0.4.93)

Reported from a client against nesaorg/PonyRealism-Uncensored as three
symptoms -- "the edit result is garbage", "size is ignored", and a 500. Two
defects, neither of which announced itself.

The edit was never an edit. _load_img2img_pipeline picked its class by looking
for 'xl', 'sdxl', 'stable-diffusion' or 'sd' in the MODEL NAME, and
"nesaorg/ponyrealism-uncensored" contains none of them, so it fell through to
the generic DiffusionPipeline -- which faithfully honours the checkpoint's
model_index.json, and that one declares StableDiffusionXLPipeline: the
text-to-image class. Every diffusers __call__ ends in **kwargs (it carries the
deprecated callback pair), so `image` and `strength` were accepted, dropped,
and the pipeline rendered a fresh picture from the prompt alone at its default
1024x1024. One defect, all three symptoms: garbage (a prompt saying "add X,
keep everything else" rendered from scratch), size ignored (1024x1024 is the
text-to-image default, not the source's shape), and 200 OK throughout.
stable-diffusion-xl-base-1.0 worked only because its NAME contains "xl".

Both loaders now ask AutoPipelineForImage2Image / AutoPipelineForInpainting,
which read the declared class and substitute the one for this task. A guard
backs it up, on cached pipelines as well as freshly loaded ones -- an install
already holding a text-to-image pipeline would otherwise keep using it until it
restarted. A pipeline with no `image` input is converted with from_pipe (the
same components, no second copy in VRAM) or refused with 501. A 200 carrying a
picture of something else is the one outcome worth ruling out.

The 500 was a cache key used as a repo id. These endpoints keep task-specific
pipelines in MultiModelManager.models under "img2img:<repo>", and the manager's
fuzzy already-loaded pass matched one by its last path segment, then assigned
the whole key to resolved_name -- so from_pretrained was asked for a repo
called "img2img:nesaorg/PonyRealism-Uncensored" and answered "Repo id must use
alphanumeric chars" about a model that loads perfectly well. That pass now
skips task namespaces, which the endpoints look up by exact key anyway.

Writing the test for that found two more namespaces with the identical latent
bug -- depth: and segment: -- so _TASK_NAMESPACES covers four, and an invariant
test asserts every task prefix the endpoints invent is declared. A future
endpoint cannot reintroduce this quietly.

Not fixed, because it is not a bug: /v1/images/edits is plain img2img, so "add
X and keep everything else exactly the same" cannot work however correct the
pipeline is. That needs a mask through /v1/images/inpaint, or an
instruction-edit model exposed with a capability clients can prefer.

16 tests; full suite 4537 passed with the same 15 pre-existing failures as
0.4.91 (diff empty).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.92"
+__version__ = "0.4.93"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/images.py

@@ -2051,27 +2051,73 @@ def _prompt_call_kwargs(pipeline, request, model_config=None):
     return {"prompt": prompt, "negative_prompt": neg or None}
 
 
+def _accepts_image(pipeline) -> bool:
+    """Whether this pipeline's ``__call__`` takes an ``image`` at all.
+
+    Worth checking, because the answer was no and nothing said so. Every
+    diffusers ``__call__`` ends in ``**kwargs`` (it carries the deprecated
+    ``callback`` pair), so a *text-to-image* pipeline handed ``image=`` and
+    ``strength=`` accepts both, drops them, and renders a brand-new picture
+    from the prompt at its default 1024x1024. The request answers 200 with an
+    image, so it reads as a poor edit rather than as no edit at all -- and as
+    "size was ignored", since 1024x1024 is the text-to-image default rather
+    than the source's shape.
+    """
+    import inspect
+    try:
+        params = inspect.signature(pipeline.__call__).parameters
+    except (TypeError, ValueError):
+        return True          # can't introspect it; let the pipeline speak
+    if "image" in params:
+        return True
+    # Only *args/**kwargs: there is no parameter list to judge, so do not
+    # refuse on the strength of not finding a name in it.
+    named = [p for p in params.values()
+             if p.kind not in (p.VAR_POSITIONAL, p.VAR_KEYWORD)]
+    return not named
+
+
+def _as_image_task(pipeline, task: str):
+    """``pipeline`` as an img2img/inpaint pipeline, reusing its weights.
+
+    ``AutoPipelineFor*.from_pipe`` swaps the task class around the components
+    already in VRAM, so recovering from a text-to-image pipeline costs no
+    second copy of the weights. Returns None when it cannot be converted.
+    """
+    from diffusers import AutoPipelineForImage2Image, AutoPipelineForInpainting
+    Auto = AutoPipelineForInpainting if task == "inpaint" else AutoPipelineForImage2Image
+    try:
+        converted = Auto.from_pipe(pipeline)
+    except Exception as exc:
+        _log.warning("cannot use %s as a %s pipeline: %s",
+                     type(pipeline).__name__, task, exc)
+        return None
+    _log.info("converted %s to %s (same weights, no second copy)",
+              type(pipeline).__name__, type(converted).__name__)
+    return converted
+
+
 def _load_img2img_pipeline(model_name: str, global_args):
-    """Load a diffusers img2img pipeline."""
+    """Load a diffusers img2img pipeline.
+
+    The class comes from the checkpoint's own ``model_index.json``, through
+    ``AutoPipelineForImage2Image``, rather than from guessing at the model's
+    name. Guessing is what broke editing on the booru-tag fine-tunes:
+    "nesaorg/PonyRealism-Uncensored" contains none of 'xl', 'sdxl',
+    'stable-diffusion' or 'sd', so it fell through to the generic
+    ``DiffusionPipeline`` -- which faithfully honours ``_class_name``, and that
+    checkpoint declares ``StableDiffusionXLPipeline``, the TEXT-TO-IMAGE class.
+    See :func:`_accepts_image` for why that failed silently.
+    """
     import torch
-    from diffusers import (
-        StableDiffusionImg2ImgPipeline,
-        StableDiffusionXLImg2ImgPipeline,
-        DiffusionPipeline,
-    )
+    from diffusers import AutoPipelineForImage2Image
 
     device = _derive_diffusers_device(global_args)
     precision = getattr(global_args, 'image_precision', 'bf16') if global_args else 'bf16'
     dtype_map = {'bf16': torch.bfloat16, 'f16': torch.float16, 'f32': torch.float32}
     torch_dtype = dtype_map.get(precision, torch.bfloat16)
 
-    name_lower = model_name.lower()
-    if 'xl' in name_lower or 'sdxl' in name_lower or 'flux' in name_lower:
-        PipeClass = StableDiffusionXLImg2ImgPipeline
-    elif 'stable-diffusion' in name_lower or 'sd' in name_lower:
-        PipeClass = StableDiffusionImg2ImgPipeline
-    else:
-        PipeClass = DiffusionPipeline   # generic fallback
+    PipeClass = AutoPipelineForImage2Image
 
     for attempt in range(3):
         try:
@@ -2129,6 +2175,19 @@ async def create_image_edit(request: ImageEditRequest, http_request: Request = N
             raise HTTPException(status_code=500, detail=f"Failed to load img2img model: {e}")
         multi_model_manager.models[model_key] = pipe
 
+    # A pipeline that cannot take an image must never serve an edit: it would
+    # accept `image` into **kwargs, drop it, and answer 200 with a picture
+    # generated from the prompt alone. Convert it (same weights) or say so.
+    if not _accepts_image(pipe):
+        converted = _as_image_task(pipe, "img2img")
+        if converted is None:
+            raise HTTPException(status_code=501, detail=(
+                f"'{model_name}' loaded as {type(pipe).__name__}, which cannot "
+                f"edit an image: it has no `image` input. Editing needs a "
+                f"checkpoint diffusers can open for image-to-image."))
+        pipe = converted
+        multi_model_manager.models[model_key] = pipe
+
     try:
         import torch
         source_img = _decode_b64_image(request.image)
@@ -2218,23 +2277,18 @@ class ImageInpaintRequest(BaseModel):
 
 
 def _load_inpaint_pipeline(model_name: str, global_args):
+    """Load a diffusers inpainting pipeline.
+
+    Same rule as :func:`_load_img2img_pipeline`: the task class comes from the
+    checkpoint, not from substrings in its name.
+    """
     import torch
-    from diffusers import (
-        StableDiffusionInpaintPipeline,
-        StableDiffusionXLInpaintPipeline,
-        DiffusionPipeline,
-    )
+    from diffusers import AutoPipelineForInpainting
     device = _derive_diffusers_device(global_args)
     precision = getattr(global_args, 'image_precision', 'bf16') if global_args else 'bf16'
     dtype_map = {'bf16': torch.bfloat16, 'f16': torch.float16, 'f32': torch.float32}
     torch_dtype = dtype_map.get(precision, torch.bfloat16)
-    n = model_name.lower()
-    if 'xl' in n or 'sdxl' in n:
-        PClass = StableDiffusionXLInpaintPipeline
-    elif 'stable-diffusion' in n or 'inpaint' in n:
-        PClass = StableDiffusionInpaintPipeline
-    else:
-        PClass = DiffusionPipeline
+    PClass = AutoPipelineForInpainting
     for attempt in range(3):
         try:
             with _DIFFUSERS_LOAD_LOCK:
@@ -2281,6 +2335,14 @@ async def create_image_inpaint(request: ImageInpaintRequest, http_request: Reque
         except Exception as e:
             raise HTTPException(status_code=500, detail=f"Failed to load inpaint model: {e}")
         multi_model_manager.models[model_key] = pipe
+    if not _accepts_image(pipe):
+        converted = _as_image_task(pipe, "inpaint")
+        if converted is None:
+            raise HTTPException(status_code=501, detail=(
+                f"'{model_name}' loaded as {type(pipe).__name__}, which cannot "
+                f"inpaint: it has no `image` input."))
+        pipe = converted
+        multi_model_manager.models[model_key] = pipe
     try:
         import torch
         source_img = _decode_b64_image(request.image)

codai/models/manager.py

@@ -1557,6 +1557,15 @@ class ModelInstancePool:
             self.ref_counts.clear()
 
 
+#: Namespaces that other code keeps its OWN pipelines under inside
+#: ``MultiModelManager.models``, built from a task rather than from a model
+#: name. ``/v1/images/edits`` and ``/v1/images/inpaint`` each load a
+#: task-specific pipeline for a model that may also be loaded for text-to-image,
+#: so they key it by task. Name resolution has to skip these: a key is not a
+#: model id, and treating one as an id sends it to ``from_pretrained``.
+_TASK_NAMESPACES = ("img2img:", "inpaint:", "depth:", "segment:")
+
+
 class MultiModelManager:
     """
     Manages multiple models: main text model, audio transcription, and image generation.
@@ -5404,6 +5413,18 @@ class MultiModelManager:
 
             short = _norm_id(resolved_name)
             for k, v in self.models.items():
+                # Keys from another namespace are keys, not model names. The
+                # image edit endpoints keep their own pipelines in self.models
+                # under "img2img:<repo>" / "inpaint:<repo>", and matching one
+                # here set resolved_name to the KEY -- so the caller went on to
+                # ask HuggingFace for a repo literally called
+                # "img2img:nesaorg/PonyRealism-Uncensored" and got
+                # "Repo id must use alphanumeric chars" on a model that loads
+                # perfectly well. Those endpoints look their keys up exactly;
+                # this fuzzy pass must leave them alone.
+                if (k.startswith(_TASK_NAMESPACES)
+                        and not (type_prefix and k.startswith(type_prefix))):
+                    continue
                 k_name = k[len(type_prefix):] if type_prefix and k.startswith(type_prefix) else k
                 if (k_name == resolved_name or k_name.split("/")[-1] == resolved_name.split("/")[-1]
                         or _norm_id(k_name) == short):

tests/test_image_edit_uses_an_image_pipeline.py

+"""`/v1/images/edits` has to load a pipeline that can take an image.
+
+Reported from a client on 2026-10-25 against `nesaorg/PonyRealism-Uncensored`
+as "the edit result is garbage", "`size` is ignored" and a 500 — three
+symptoms of two defects, neither of which announced itself.
+
+**The edit was never an edit.** The img2img loader picked its pipeline class by
+looking for substrings in the model's name — 'xl', 'sdxl', 'stable-diffusion',
+'sd' — and that name contains none of them, so it fell through to the generic
+`DiffusionPipeline`. That faithfully honours the checkpoint's own
+`model_index.json`, which for this model declares `StableDiffusionXLPipeline`:
+the **text-to-image** class. Every diffusers `__call__` ends in `**kwargs` (it
+carries the deprecated `callback` pair), so `image=` and `strength=` were
+accepted, silently dropped, and the pipeline rendered a fresh picture from the
+prompt alone at its default 1024x1024. Hence "garbage" (a prompt saying "add X,
+keep everything else" rendered from scratch), hence "`size` ignored" (1024x1024
+is the text-to-image default, not the source's shape), and hence 200 OK
+throughout.
+
+**The 500 was a key used as a name.** These endpoints keep their task-specific
+pipelines in `MultiModelManager.models` under `img2img:<repo>` and
+`inpaint:<repo>`. The manager's fuzzy "is this already loaded?" pass matched
+one of those keys by its last path segment and then assigned the whole key to
+`resolved_name`, so the endpoint went on to ask HuggingFace for a repo called
+`img2img:nesaorg/PonyRealism-Uncensored` — "Repo id must use alphanumeric
+chars" — on a model that loads perfectly well.
+"""
+import inspect
+
+import pytest
+
+from codai.api import images as images_mod
+
+
+# --------------------------------------------------------------------------- #
+#  a pipeline that cannot take an image
+# --------------------------------------------------------------------------- #
+
+def _with_signature_of(func):
+    """An object whose ``__call__`` is ``func``, for signature checks only."""
+    return type("Pipe", (), {"__call__": func})()
+
+
+def test_a_text2img_pipeline_is_recognised_as_unable_to_edit():
+    from diffusers import StableDiffusionXLPipeline
+    pipe = _with_signature_of(StableDiffusionXLPipeline.__call__)
+    assert images_mod._accepts_image(pipe) is False
+
+
+@pytest.mark.parametrize("cls_name", [
+    "StableDiffusionXLImg2ImgPipeline",
+    "StableDiffusionXLInpaintPipeline",
+    "StableDiffusionImg2ImgPipeline",
+])
+def test_an_image_pipeline_is_recognised_as_able(cls_name):
+    import diffusers
+    cls = getattr(diffusers, cls_name)
+    assert images_mod._accepts_image(_with_signature_of(cls.__call__)) is True
+
+
+def test_kwargs_is_why_this_failed_silently():
+    """The reason the wrong class answered 200 instead of raising: `image` and
+    `strength` land in **kwargs and are dropped, not rejected."""
+    from diffusers import StableDiffusionXLPipeline
+    params = inspect.signature(StableDiffusionXLPipeline.__call__).parameters
+    assert "image" not in params and "strength" not in params
+    assert any(p.kind is p.VAR_KEYWORD for p in params.values()), \
+        "without **kwargs this would have been a TypeError and never shipped"
+
+
+def test_a_pipeline_with_nothing_to_judge_is_given_the_benefit_of_the_doubt():
+    """A ``(*args, **kwargs)`` signature carries no parameter list, so "image
+    is not in it" means nothing — refusing on that would turn an odd but
+    working pipeline into a 501."""
+    class Odd:
+        __call__ = staticmethod(lambda *a, **k: None)
+    assert images_mod._accepts_image(Odd()) is True
+
+
+def test_a_real_parameter_list_without_image_is_a_refusal():
+    """Whereas a signature that names its inputs and does not name `image` is
+    evidence, and the only safe reading of it is "cannot edit"."""
+    class Text2Img:
+        def __call__(self, prompt=None, height=None, width=None, **kwargs):
+            return None
+    assert images_mod._accepts_image(Text2Img()) is False
+
+
+# --------------------------------------------------------------------------- #
+#  the class comes from the checkpoint, not from its name
+# --------------------------------------------------------------------------- #
+
+def test_the_loaders_no_longer_guess_from_the_model_name():
+    """The regression in one line: a name with no 'xl' in it got a
+    text-to-image pipeline."""
+    src = open("codai/api/images.py").read()
+    head = src.index("def _load_img2img_pipeline")
+    body = src[head:head + 2500]
+    assert "AutoPipelineForImage2Image" in body
+    for guess in ("'xl' in name_lower", "'sdxl' in name_lower",
+                  "'sd' in name_lower"):
+        assert guess not in body, f"still guessing: {guess}"
+
+    head = src.index("def _load_inpaint_pipeline")
+    body = src[head:head + 2000]
+    assert "AutoPipelineForInpainting" in body
+    assert "'xl' in n" not in body
+
+
+def test_the_reported_checkpoint_maps_to_an_img2img_class():
+    """What AutoPipelineFor* does that the name heuristics could not: read the
+    declared class and swap in the one for this task."""
+    from diffusers.pipelines.auto_pipeline import (
+        AUTO_IMAGE2IMAGE_PIPELINES_MAPPING,
+        AUTO_INPAINT_PIPELINES_MAPPING,
+        _get_task_class,
+    )
+    declared = "StableDiffusionXLPipeline"      # PonyRealism's model_index.json
+    assert _get_task_class(AUTO_IMAGE2IMAGE_PIPELINES_MAPPING, declared).__name__ \
+        == "StableDiffusionXLImg2ImgPipeline"
+    assert _get_task_class(AUTO_INPAINT_PIPELINES_MAPPING, declared).__name__ \
+        == "StableDiffusionXLInpaintPipeline"
+
+
+def test_conversion_reuses_the_weights_rather_than_loading_them_twice():
+    """A text-to-image pipeline already in VRAM is recovered with from_pipe,
+    which swaps the task class around the same components."""
+    from diffusers import AutoPipelineForImage2Image, AutoPipelineForInpainting
+    assert hasattr(AutoPipelineForImage2Image, "from_pipe")
+    assert hasattr(AutoPipelineForInpainting, "from_pipe")
+    src = open("codai/api/images.py").read()
+    assert "from_pipe" in src
+
+
+def test_an_unconvertible_pipeline_is_refused_not_served():
+    """501, not a 200 carrying a picture of something else."""
+    src = open("codai/api/images.py").read()
+    assert "cannot \nedit an image" in src or "cannot " in src
+    assert src.count("status_code=501") >= 2, \
+        "both edits and inpaint must refuse rather than silently generate"
+    assert "has no `image` input" in src
+
+
+@pytest.mark.parametrize("endpoint", ["create_image_edit", "create_image_inpaint"])
+def test_the_guard_runs_on_a_cached_pipeline_too(endpoint):
+    """A pipeline loaded by an earlier version is already in the manager's
+    models dict, so checking only after a fresh load would leave every running
+    install serving text-to-image until it restarted."""
+    src = open("codai/api/images.py").read()
+    head = src.index(f"async def {endpoint}")
+    body = src[head:head + 4000]
+    guard = body.index("_accepts_image(pipe)")
+    cached_lookup = body.index("multi_model_manager.models.get(")
+    assert guard > cached_lookup
+    # and it is not nested inside the `if pipe is None:` load branch
+    line_start = body.rindex("\n", 0, guard) + 1
+    indent = len(body[line_start:guard]) - len(body[line_start:guard].lstrip())
+    assert indent <= 4, "the guard is inside the load branch, so a cached " \
+                        "text-to-image pipeline would still be used"
+
+
+# --------------------------------------------------------------------------- #
+#  a task key is not a model name
+# --------------------------------------------------------------------------- #
+
+def test_the_task_namespaces_are_declared():
+    from codai.models.manager import _TASK_NAMESPACES
+    assert "img2img:" in _TASK_NAMESPACES
+    assert "inpaint:" in _TASK_NAMESPACES
+
+
+def test_every_namespace_the_endpoints_invent_is_declared():
+    """The invariant that keeps this from coming back: if a future endpoint
+    keys a pipeline by task, name resolution has to know to skip it, or that
+    key becomes a repo id again."""
+    import re
+    from codai.models.manager import _TASK_NAMESPACES
+    src = open("codai/api/images.py").read()
+    used = set(re.findall(r'f"([a-z0-9_]+):\{model_name\}"', src))
+    # The manager owns these and strips them itself during resolution; every
+    # other prefix is one an endpoint made up for a task.
+    manager_own = {"image:", "audio:", "tts:", "vision:", "video:",
+                   "audio_gen:", "embedding:", "spatial:", "text:"}
+    invented = {f"{u}:" for u in used} - manager_own
+    assert invented, "expected the edit endpoints to key pipelines by task"
+    missing = invented - set(_TASK_NAMESPACES)
+    assert not missing, f"not declared in _TASK_NAMESPACES: {sorted(missing)}"
+
+
+def test_name_resolution_skips_a_foreign_task_key():
+    """The 500, as the guard that prevents it: the fuzzy already-loaded pass
+    must leave img2img:/inpaint: keys alone, because it assigns the matched key
+    to resolved_name and that value goes to from_pretrained."""
+    src = open("codai/models/manager.py").read()
+    head = src.index("Fuzzy fallback:")
+    body = src[head:head + 2200]
+    assert "_TASK_NAMESPACES" in body
+    # the skip has to come before the match, not after it
+    assert body.index("_TASK_NAMESPACES") < body.index("resolved_name = k_name")
+    assert "continue" in body[body.index("_TASK_NAMESPACES"):
+                              body.index("resolved_name = k_name")]