seed an offloaded pipeline, and stop advertising a direction a model cannot go

**A seeded request to an offloaded pipeline failed as a load error.** Sequential
CPU offload leaves every module on the META device until its own forward pulls
it in, and a pipeline whose modules are all on meta reports meta as its device.
torch.Generator(device=pipeline.device) then reached torch's accelerator hooks
with a meta device:

    RuntimeError: META device type not an accelerator.

The image route reported that as "Failed to load image model
'Tongyi-MAI/Z-Image-Turbo'" -- naming the load, which had in fact succeeded
(footprint measured, 1.56 GB offloaded to host RAM) -- so the model read as
unloadable. And only a SEEDED request failed, so it looked broken to anyone who
passes a seed and perfectly fine to anyone who does not.

diffusers already knows the answer: _execution_device is where an offloaded
pipeline actually computes. models.utils.generator_device() prefers it, rejects
meta and lazy, and falls back to cuda/cpu. Four sites constructed a Generator
from a pipeline's own device -- text-to-image, img2img, edit, and audio
generation -- and fixing one would have left the same failure reachable by three
other routes, so all four go through it.

**Four working edit models were reported as broken because of their capability
list.** The Qwen-Image-Edit siblings carried image_generation and modes: null,
while diffusers maps no text-to-image class for the family and the request path
says so outright:

    501 QwenImageEditPlusPipeline cannot generate from a prompt alone:
        diffusers offers no text-to-image pipeline for this family.

The runtime was right and the capability list was wrong, which is the wrong way
round. The instruction-edit branch of the heuristic lumped two different kinds
of model together: FLUX.2 is unified -- one checkpoint that generates from text
AND edits -- while Qwen-Image-Edit, FLUX.1 Kontext, Step1X-Edit and the
Rapid-AIO fine-tunes are edit-only. They are split now, so only a unified family
claims image_generation.

And the listing publishes the directions. `modes` is how a client learns a model
has no text-to-image path without sending one and reading a 501, and most cards
never set it; when a card says nothing, the modes are now inferred from the
capabilities the model already publishes (t2i/i2i, t2v/i2v/v2v). Inferred, never
invented: every mode is backed by a flag that was already there, and a card
that states its own modes always wins.

Verified live while testing this: Wan2.2-I2V DOES serve a prompt-only request
(it returned an mp4), so that row was collateral from the VRAM contention
rather than a missing path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.5.34"
+__version__ = "0.5.35"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/audio_gen.py

@@ -31,6 +31,7 @@ from typing import Optional
 from fastapi import APIRouter, HTTPException, Request
 
 from codai.models.manager import multi_model_manager
+from codai.models import utils as model_utils
 from codai.pydantic.audiogenrequest import AudioGenerationRequest, AudioGenerationResponse
 from codai.tasks import task_registry, TaskCancelled
 
@@ -448,7 +449,8 @@ def _generate_audio(pipe, model_name: str, request: AudioGenerationRequest, task
         import torch
         _gen = None
         if getattr(request, 'seed', None) is not None:
-            _gen = torch.Generator(pipe.device).manual_seed(int(request.seed))
+            _gen = torch.Generator(
+                model_utils.generator_device(pipe)).manual_seed(int(request.seed))
         result = pipe(
             request.prompt,
             negative_prompt=getattr(request, 'negative_prompt', None),

codai/api/images.py

@@ -32,6 +32,7 @@ from fastapi import APIRouter, HTTPException, Request
 
 _log = logging.getLogger(__name__)
 from PIL import Image
+from codai.models import utils as model_utils
 from pydantic import BaseModel, ConfigDict
 
 # Import from codai modules
@@ -1641,7 +1642,8 @@ async def _generate_with_diffusers(pipeline, request, global_args, http_request=
     seed = request.seed if request.seed is not None else getattr(global_args, 'image_seed', None)
     generator = None
     if seed is not None:
-        generator = torch.Generator(device=pipeline.device).manual_seed(seed)
+        generator = torch.Generator(
+            device=model_utils.generator_device(pipeline)).manual_seed(seed)
 
     quality = request.quality or "standard"
     # Acceleration/distillation defaults (Lightning / Turbo / LCM / Hyper-SD): when
@@ -3182,7 +3184,8 @@ async def create_image_edit(request: ImageEditRequest, http_request: Request = N
                        for im in sources]
 
         seed = request.seed or getattr(global_args, 'image_seed', None)
-        generator = torch.Generator(device=pipe.device).manual_seed(seed) if seed else None
+        generator = (torch.Generator(device=model_utils.generator_device(pipe))
+                     .manual_seed(seed) if seed else None)
         quality = request.quality or "standard"
         _instruction = _is_instruction_edit(pipe)
         # A distilled instruction model is tuned for 4 steps at CFG 1.0;
@@ -3332,7 +3335,8 @@ async def create_image_inpaint(request: ImageInpaintRequest, http_request: Reque
                 source_img = source_img.resize((w, h))
                 mask_img = mask_img.resize((w, h))
         seed = request.seed or getattr(global_args, 'image_seed', None)
-        generator = torch.Generator(device=pipe.device).manual_seed(seed) if seed else None
+        generator = (torch.Generator(device=model_utils.generator_device(pipe))
+                     .manual_seed(seed) if seed else None)
         quality = request.quality or "standard"
         num_steps = request.steps or (30 if quality == "standard" else 50)
         cfg_scale = request.guidance_scale or (getattr(global_args, 'image_cfg_scale', 7.5) if quality == "standard" else 9.0)

codai/models/capabilities.py

@@ -246,14 +246,31 @@ def detect_model_capabilities(model_name: str) -> ModelCapabilities:
     # `strength`, which is exactly what images._is_instruction_edit decides on,
     # so the runtime already treated it as an instruction edit and only the
     # capability list disagreed.
-    if (any(x in n for x in ['qwen-image-edit', 'qwen_image_edit',
-                             'flux.1-kontext', 'flux-kontext', 'flux_kontext',
-                             'flux.2', 'flux-2', 'flux2',
-                             'image-edit-rapid', 'firered-image-edit'])
+    #
+    # Two kinds of model land here and they do NOT have the same directions.
+    # FLUX.2 is unified: one checkpoint makes a picture from text and edits one
+    # from an instruction. Qwen-Image-Edit, FLUX.1 Kontext, Step1X-Edit and the
+    # Rapid-AIO fine-tunes are edit-ONLY -- diffusers maps no text-to-image
+    # class for those families, and the request path says so outright:
+    #
+    #   501 QwenImageEditPlusPipeline cannot generate from a prompt alone:
+    #       diffusers offers no text-to-image pipeline for this family.
+    #
+    # Advertising image_generation for them anyway is what sent clients down
+    # that path: a capability sweep read "image_generation", posted a prompt,
+    # and filed the model as broken. The runtime was right and the capability
+    # list was wrong, which is the wrong way round -- the listing has to agree
+    # with what the model will actually do.
+    _UNIFIED_EDIT = ['flux.2', 'flux-2', 'flux2']
+    _EDIT_ONLY = ['qwen-image-edit', 'qwen_image_edit',
+                  'flux.1-kontext', 'flux-kontext', 'flux_kontext',
+                  'image-edit-rapid', 'firered-image-edit']
+    if (any(x in n for x in _UNIFIED_EDIT + _EDIT_ONLY)
             or re.search(r'\bstep1x-edit', n)):
         caps.instruction_edit = True
         caps.image_to_image = True      # it is still an image-in, image-out edit
-        caps.image_generation = True
+        # Only a unified family can start from a prompt alone.
+        caps.image_generation = any(x in n for x in _UNIFIED_EDIT)
         return caps
 
     # ── Image: generation ────────────────────────────────────────────────────

codai/models/manager.py

@@ -1581,6 +1581,41 @@ class ModelInstancePool:
 _TASK_NAMESPACES = ("img2img:", "inpaint:", "depth:", "segment:")
 
 
+def _modes_from_caps(caps, model_type: str = "") -> Optional[list]:
+    """Which directions a model generates in, inferred from its capabilities.
+
+    A fallback for cards that say nothing. ``modes`` is how a client learns that
+    a model has no text-to-image path WITHOUT sending one and reading a 501, and
+    most cards never set it -- four Qwen-Image-Edit siblings published
+    ``modes: null`` while advertising image_generation, so a capability sweep
+    posted a prompt to an edit-only model and filed it as broken.
+
+    Inferred, never invented: each mode here is backed by a capability flag the
+    model already publishes, so this adds no new claim. A card that states its
+    own modes always wins.
+    """
+    try:
+        flags = set(caps.to_list() if hasattr(caps, "to_list") else (caps or []))
+    except Exception:
+        return None
+    out = []
+    if model_type == "image" or flags & {"image_generation", "image_to_image",
+                                         "instruction_edit", "inpainting"}:
+        if "image_generation" in flags:
+            out.append("t2i")
+        if flags & {"image_to_image", "instruction_edit"}:
+            out.append("i2i")
+    if model_type == "video" or flags & {"video_generation", "image_to_video",
+                                         "video_to_video"}:
+        if "video_generation" in flags:
+            out.append("t2v")
+        if "image_to_video" in flags:
+            out.append("i2v")
+        if "video_to_video" in flags:
+            out.append("v2v")
+    return out or None
+
+
 def _group_ids(models, canonical_by_id: Dict[str, str]) -> None:
     """Mark which published ids are the same model, in place.
 
@@ -7151,7 +7186,11 @@ class MultiModelManager:
                 # to remember which a given model type uses.
                 modes=((meta.get("defaults") or {}).get("modes")
                        or meta.get("modes")
-                       or meta.get("video_subtypes") or None),
+                       or meta.get("video_subtypes")
+                       # Nothing on the card: infer from the capabilities it
+                       # already publishes, so "can this take a prompt alone?"
+                       # is answerable without sending one.
+                       or _modes_from_caps(caps, resolved_type)),
                 license=((meta.get("defaults") or {}).get("license")
                          or meta.get("license") or meta.get("licence") or None),
                 available=meta.get("available"),

codai/models/utils.py

@@ -23,6 +23,42 @@ from typing import Optional, Any
 _global_args = None
 
 
+def generator_device(pipeline):
+    """A device a ``torch.Generator`` can actually be built on.
+
+    A pipeline's own ``.device`` is not always one. Sequential CPU offload
+    leaves every module on the META device until its own forward pulls it in,
+    and a pipeline whose modules are all on meta reports meta as its device --
+    so ``torch.Generator(device=pipeline.device)`` reached torch's accelerator
+    hooks with a meta device and raised:
+
+        RuntimeError: META device type not an accelerator.
+
+    Measured on Z-Image-Turbo (bf16, sequential_text_encoder): the image route
+    reported it as "Failed to load image model 'Tongyi-MAI/Z-Image-Turbo'",
+    naming the load -- which had in fact succeeded, footprint and all -- rather
+    than the seed. Only a SEEDED request failed, so the model looked broken to
+    anyone who passes a seed and fine to anyone who does not.
+
+    diffusers already knows the answer: ``_execution_device`` is the device an
+    offloaded pipeline will actually compute on. Use it when present, and
+    otherwise fall back to anything that is not meta.
+    """
+    import torch
+    for candidate in (getattr(pipeline, "_execution_device", None),
+                      getattr(pipeline, "device", None)):
+        try:
+            dev = torch.device(candidate) if candidate is not None else None
+        except (TypeError, ValueError, RuntimeError):
+            dev = None
+        if dev is not None and dev.type not in ("meta", "lazy"):
+            return dev
+    try:
+        return torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    except Exception:
+        return torch.device("cpu")
+
+
 def set_global_args(args: Any):
     """Set the global arguments object for use in utility functions."""
     global _global_args

tests/test_edit_only_model_directions.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""What a client is told about which directions a model generates in.
+
+Four Qwen-Image-Edit siblings were published with ``image_generation`` in their
+capabilities and ``modes: null``. diffusers maps no text-to-image class for that
+family, and the request path says so plainly:
+
+    501 QwenImageEditPlusPipeline cannot generate from a prompt alone:
+        diffusers offers no text-to-image pipeline for this family.
+
+So the runtime was right and the capability list was wrong, which is the wrong
+way round: a capability sweep read image_generation, posted a prompt, got the
+501 and filed four working edit models as broken.
+"""
+
+import io
+import os
+
+import pytest
+
+from codai.models.capabilities import detect_model_capabilities
+from codai.models.manager import _modes_from_caps
+
+_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+
+def _src(rel):
+    return io.open(os.path.join(_ROOT, rel), encoding="utf-8").read()
+
+
+# ---------------------------------------------------------------------------
+# edit-only vs unified
+# ---------------------------------------------------------------------------
+
+@pytest.mark.parametrize("name", [
+    "Qwen-Image-Edit-2511",
+    "Qwen/Qwen-Image-Edit-2511",
+    "Qwen-Image-Edit-Rapid-AIO-v23",
+    "FLUX.1-Kontext-dev",
+    "black-forest-labs/FLUX.1-Kontext-dev",
+    "step1x-edit",
+    "firered-image-edit",
+])
+def test_an_edit_only_family_does_not_claim_text_to_image(name):
+    caps = detect_model_capabilities(name).to_list()
+    assert "instruction_edit" in caps
+    assert "image_to_image" in caps
+    assert "image_generation" not in caps, name
+
+
+@pytest.mark.parametrize("name", [
+    "black-forest-labs/FLUX.2-klein-4B",
+    "FLUX.2-dev",
+    "flux2-pro",
+])
+def test_a_unified_family_keeps_text_to_image(name):
+    """FLUX.2 is one checkpoint that both generates and edits — taking its
+    text-to-image away would be the same mistake in the other direction."""
+    caps = detect_model_capabilities(name).to_list()
+    assert "image_generation" in caps
+    assert "instruction_edit" in caps
+
+
+def test_an_ordinary_generator_is_untouched():
+    caps = detect_model_capabilities(
+        "stabilityai/stable-diffusion-xl-base-1.0").to_list()
+    assert "image_generation" in caps
+    assert "instruction_edit" not in caps
+
+
+# ---------------------------------------------------------------------------
+# the modes a client reads
+# ---------------------------------------------------------------------------
+
+def test_an_edit_only_model_publishes_i2i_alone():
+    assert _modes_from_caps(
+        detect_model_capabilities("Qwen-Image-Edit-2511"), "image") == ["i2i"]
+
+
+def test_a_unified_model_publishes_both_directions():
+    assert _modes_from_caps(
+        detect_model_capabilities("FLUX.2-klein-4B"), "image") == ["t2i", "i2i"]
+
+
+def test_an_image_to_video_model_publishes_i2v():
+    assert _modes_from_caps(
+        detect_model_capabilities("Wan-AI/Wan2.2-I2V-A14B-Diffusers"),
+        "video") == ["i2v"]
+
+
+def test_every_inferred_mode_is_backed_by_a_capability():
+    """Inferred, never invented: the point of this fallback is to restate what
+    the model already claims, not to add a claim."""
+    pairs = {"t2i": "image_generation", "i2v": "image_to_video",
+             "t2v": "video_generation", "v2v": "video_to_video"}
+    for name in ("Qwen-Image-Edit-2511", "FLUX.2-klein-4B",
+                 "stabilityai/stable-diffusion-xl-base-1.0",
+                 "Wan-AI/Wan2.2-I2V-A14B-Diffusers", "Lightricks/LTX-2.5"):
+        caps = detect_model_capabilities(name)
+        flags = set(caps.to_list())
+        for mode in (_modes_from_caps(caps) or []):
+            if mode in pairs:
+                assert pairs[mode] in flags, (name, mode)
+            else:
+                assert mode == "i2i"
+                assert flags & {"image_to_image", "instruction_edit"}
+
+
+def test_a_model_with_no_generation_capability_gets_no_modes():
+    assert _modes_from_caps(detect_model_capabilities("bge-m3"), "embedding") is None
+    assert _modes_from_caps([], "") is None
+    assert _modes_from_caps(None, "") is None
+
+
+def test_the_cards_own_modes_always_win():
+    """Inference is a fallback. An operator who stated the directions -- for a
+    model whose name says nothing, or one deliberately restricted -- keeps
+    them."""
+    src = _src("codai/models/manager.py")
+    head = src.index("modes=((meta.get(\"defaults\") or {}).get(\"modes\")")
+    body = src[head:head + 700]
+    # The card's three spellings are consulted BEFORE the inference.
+    assert body.index("meta.get(\"modes\")") < body.index("_modes_from_caps(")
+    assert body.index("meta.get(\"video_subtypes\")") < body.index("_modes_from_caps(")

tests/test_model_defaults_and_samplers.py

@@ -183,10 +183,24 @@ def test_modes_and_licence_are_read_from_either_place():
     src = inspect.getsource(manager)
     anchor = 'defaults=(meta.get("defaults") or None)'
     i = src.index(anchor)
-    # The enclosing ModelInfo(...) call: from the nearest preceding
-    # `models.append(ModelInfo(` to the `))` that closes it.
+    # The enclosing ModelInfo(...) call, delimited by BALANCED parentheses
+    # rather than by the first "))" after the anchor. Scanning for "))" found
+    # the end of any nested call that happened to sit inside the block -- a
+    # mode-inference fallback written as
+    # `or _modes_from_caps(caps, resolved_type))` closed two parens and
+    # truncated the slice, so fields further down went missing and this test
+    # failed for a field it was not about.
     start = src.rindex("models.append(ModelInfo(", 0, i)
-    end = src.index("))", i)
+    depth, end = 0, None
+    for pos in range(start, len(src)):
+        if src[pos] == "(":
+            depth += 1
+        elif src[pos] == ")":
+            depth -= 1
+            if depth == 0:
+                end = pos
+                break
+    assert end is not None, "unbalanced ModelInfo(...) call"
     block = src[start:end]
     for spelling in ('meta.get("modes")', 'meta.get("video_subtypes")',
                      'meta.get("license")', 'meta.get("licence")',

tests/test_seeded_generator_device.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""Where a seeded request's random generator lives.
+
+Sequential CPU offload leaves every module on the META device until its own
+forward pulls it in, and a pipeline whose modules are all on meta reports meta
+as its device. ``torch.Generator(device=pipeline.device)`` then reached torch's
+accelerator hooks with a meta device:
+
+    RuntimeError: META device type not an accelerator.
+
+Measured on Z-Image-Turbo (bf16, sequential_text_encoder=true). The image route
+reported it as "Failed to load image model 'Tongyi-MAI/Z-Image-Turbo'" -- naming
+the load, which had in fact succeeded (footprint measured, 1.56 GB offloaded) --
+and only SEEDED requests failed, so the model looked broken to anyone who
+passes a seed and perfectly fine to anyone who does not.
+"""
+
+import io
+import os
+
+import torch
+
+from codai.models.utils import generator_device
+
+_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+
+def _src(rel):
+    return io.open(os.path.join(_ROOT, rel), encoding="utf-8").read()
+
+
+class _Pipe:
+    def __init__(self, device=None, execution_device=None):
+        if device is not None:
+            self.device = torch.device(device)
+        if execution_device is not None:
+            self._execution_device = torch.device(execution_device)
+
+
+def test_a_meta_device_is_never_handed_to_a_generator():
+    """The exact failure: every module offloaded, so .device is meta."""
+    dev = generator_device(_Pipe(device="meta"))
+    assert dev.type != "meta"
+    # And it is usable, which is the only thing that matters here.
+    torch.Generator(device=dev).manual_seed(1234)
+
+
+def test_the_execution_device_wins_when_diffusers_offers_one():
+    """diffusers already knows where an offloaded pipeline computes."""
+    dev = generator_device(_Pipe(device="meta", execution_device="cpu"))
+    assert dev == torch.device("cpu")
+
+
+def test_an_ordinary_pipeline_is_left_exactly_alone():
+    assert generator_device(_Pipe(device="cpu")) == torch.device("cpu")
+
+
+def test_a_pipeline_with_no_device_still_yields_one():
+    dev = generator_device(object())
+    assert dev.type in ("cuda", "cpu")
+    torch.Generator(device=dev).manual_seed(1)
+
+
+def test_a_nonsense_device_does_not_raise():
+    class _Bad:
+        device = "not-a-device"
+    dev = generator_device(_Bad())
+    assert dev.type in ("cuda", "cpu")
+
+
+def test_a_lazy_device_is_refused_too():
+    """Same class of device as meta: no accelerator hooks behind it."""
+    class _Lazy:
+        device = torch.device("lazy")
+    assert generator_device(_Lazy()).type != "lazy"
+
+
+def test_every_pipeline_generator_goes_through_it():
+    """Four sites constructed a Generator from a pipeline's own device: the
+    text-to-image path, img2img, edit, and audio generation. One of them being
+    fixed would leave the same failure reachable by three other routes."""
+    for rel in ("codai/api/images.py", "codai/api/audio_gen.py"):
+        src = _src(rel)
+        assert "torch.Generator(device=pipeline.device)" not in src
+        assert "torch.Generator(device=pipe.device)" not in src
+        assert "torch.Generator(pipe.device)" not in src
+    assert _src("codai/api/images.py").count(
+        "model_utils.generator_device(") == 3
+    assert _src("codai/api/audio_gen.py").count(
+        "model_utils.generator_device(") == 1