a seeded request drew its noise on the card the pipeline was not using

Z-Image-Turbo, measured 2026-11-01:

    Cannot generate a cpu tensor from a generator of type cuda.

generator_device() followed _execution_device, which diffusers documents as
the device an offloaded pipeline COMPUTES on -- not the one it allocates its
initial latents on. randn_tensor is asymmetric about the difference:

    if gen_device_type != device.type and gen_device_type == "cpu":
        rand_device = "cpu"          # draw on the host, copy over. Fine.
    elif gen_device_type != device.type and gen_device_type == "cuda":
        raise ValueError(f"Cannot generate a {device} tensor from a "
                         f"generator of type {gen_device_type}.")

So a CPU generator is portable and a CUDA one is a bet that the pipeline wants
CUDA tensors. An offloaded pipeline does not, and only SEEDED requests lost the
bet -- the same shape of failure this function was written for, one layer down:
the first version crashed on a meta device, following _execution_device fixed
that and left this.

Draw on the host unconditionally. It costs one copy of a few MB per request and
buys reproducibility the old behaviour never had: the same seed now gives the
same image whether the model is resident, offloaded or sharded, because the RNG
stream no longer depends on placement. Seeds from before this change produce
different images, which is the price of them working at all.

test_the_execution_device_wins_when_diffusers_offers_one still PASSED after the
change, by coincidence, while asserting a rule that no longer exists; it is now
the offloaded case, and the guarantee that would actually catch a regression --
a CUDA pipeline still gets a host generator -- is tested beside it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC

codai/models/utils.py

@@ -40,23 +40,34 @@ def generator_device(pipeline):
     than the seed. Only a SEEDED request failed, so the model looked broken to
     anyone who passes a seed and fine to anyone who does not.
 
-    diffusers already knows the answer: ``_execution_device`` is the device an
-    offloaded pipeline will actually compute on. Use it when present, and
-    otherwise fall back to anything that is not meta.
+    The answer is CPU, always, and the reason is an asymmetry in diffusers'
+    own ``randn_tensor``::
+
+        if gen_device_type != device.type and gen_device_type == "cpu":
+            rand_device = "cpu"          # generate here, then move. Fine.
+        elif gen_device_type != device.type and gen_device_type == "cuda":
+            raise ValueError(f"Cannot generate a {device} tensor from a "
+                             f"generator of type {gen_device_type}.")
+
+    A CPU generator is portable: whatever device the pipeline asks for, the
+    noise is drawn on the host and copied. A CUDA generator is only correct
+    while the pipeline happens to want CUDA tensors, and an offloaded pipeline
+    does not -- measured 2026-11-01, Z-Image-Turbo answering
+
+        Cannot generate a cpu tensor from a generator of type cuda.
+
+    This function used to follow ``_execution_device``, which is the device the
+    pipeline COMPUTES on and not necessarily the one it allocates its initial
+    latents on, so a seeded request on an offloaded model failed where the same
+    request on a resident one worked.
+
+    Drawing the noise on the host costs one copy of a few MB, once per request,
+    and buys something worth more than that: the SAME seed now gives the same
+    image whether the model is resident, offloaded or sharded, because the RNG
+    stream no longer depends on placement.
     """
     import torch
-    for candidate in (getattr(pipeline, "_execution_device", None),
-                      getattr(pipeline, "device", None)):
-        try:
-            dev = torch.device(candidate) if candidate is not None else None
-        except (TypeError, ValueError, RuntimeError):
-            dev = None
-        if dev is not None and dev.type not in ("meta", "lazy"):
-            return dev
-    try:
-        return torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    except Exception:
-        return torch.device("cpu")
+    return torch.device("cpu")
 
 
 def set_global_args(args: Any):

tests/test_seeded_generator_device.py

@@ -28,6 +28,18 @@ reported it as "Failed to load image model 'Tongyi-MAI/Z-Image-Turbo'" -- naming
 the load, which had in fact succeeded (footprint measured, 1.56 GB offloaded) --
 and only SEEDED requests failed, so the model looked broken to anyone who
 passes a seed and perfectly fine to anyone who does not.
+
+Following ``_execution_device`` fixed the meta crash and left a second one, for
+the same reason: that is the device the pipeline COMPUTES on, not necessarily
+the one it allocates its initial latents on. diffusers' randn_tensor tolerates
+a CPU generator (draw on the host, copy over) and REFUSES a CUDA one whenever
+the target is CPU --
+
+    Cannot generate a cpu tensor from a generator of type cuda.
+
+-- measured again on Z-Image-Turbo, 2026-11-01. So the generator is built on
+the host unconditionally, which also makes a seed reproduce the same image
+whether the model is resident, offloaded or sharded.
 """
 
 import io
@@ -60,12 +72,50 @@ def test_a_meta_device_is_never_handed_to_a_generator():
     torch.Generator(device=dev).manual_seed(1234)
 
 
-def test_the_execution_device_wins_when_diffusers_offers_one():
-    """diffusers already knows where an offloaded pipeline computes."""
+def test_an_offloaded_pipeline_gets_a_host_generator():
+    """_execution_device no longer decides. It is where the pipeline COMPUTES,
+    and an offloaded pipeline allocates its first latents elsewhere -- which is
+    how the cuda-generator/cpu-tensor refusal got in after the meta fix."""
     dev = generator_device(_Pipe(device="meta", execution_device="cpu"))
     assert dev == torch.device("cpu")
 
 
+def test_a_cuda_pipeline_still_gets_a_host_generator():
+    """THE new guarantee, and the one that would be lost by "optimising" this
+    back to the pipeline's device. A CUDA generator is fatal the moment the
+    pipeline wants a CPU tensor; a CPU generator is never fatal, because
+    randn_tensor draws on the host and copies.
+
+        if gen_device_type != device.type and gen_device_type == "cpu":
+            rand_device = "cpu"          # fine
+        elif gen_device_type != device.type and gen_device_type == "cuda":
+            raise ValueError(...)        # the measured failure
+    """
+    assert generator_device(_Pipe(device="cuda")) == torch.device("cpu")
+    assert generator_device(
+        _Pipe(device="cuda", execution_device="cuda")) == torch.device("cpu")
+
+
+def test_the_device_is_always_one_a_generator_can_be_built_on():
+    """Whatever the pipeline claims -- including nothing at all."""
+    for pipe in (_Pipe(device="meta"), _Pipe(device="cuda"), _Pipe(device="cpu"),
+                 object()):
+        torch.Generator(device=generator_device(pipe)).manual_seed(7)
+
+
+def test_a_seed_no_longer_depends_on_where_the_model_sits():
+    """Two placements of the same model must draw the same noise: that is what
+    a seed is for, and it was not true while the generator followed the
+    pipeline."""
+    a = generator_device(_Pipe(device="cuda"))
+    b = generator_device(_Pipe(device="meta", execution_device="cpu"))
+    assert a == b
+    first = torch.Generator(device=a).manual_seed(42)
+    second = torch.Generator(device=b).manual_seed(42)
+    assert torch.equal(torch.randn(8, generator=first),
+                       torch.randn(8, generator=second))
+
+
 def test_an_ordinary_pipeline_is_left_exactly_alone():
     assert generator_device(_Pipe(device="cpu")) == torch.device("cpu")