reconcile a converter and a class that disagree about twelve names

The LTX-2.3 GGUFs failed with "Cannot copy out of meta tensor; no data!", and
the obvious reading -- parameters the checkpoint lacks -- was wrong.

diffusers 0.38 converts an LTX-2 checkpoint prompt-adaln weights to
prompt_adaln_single.* and audio_prompt_adaln_single.*, while
LTX2VideoTransformer3DModel expects prompt_adaln.* and audio_prompt_adaln.*.
Twelve names, one-to-one, the same tensors. They were present all along, under
a name nobody reconciled, so they never materialised and stayed on meta.

Measured identically on LTX-2.3-10Eros v1.5 and PinkCherry v1.8: 4444 GGUF
tensors, 4186 expected parameters, and after the rename 0 missing and 0
unused on both.

The tempting fix was to initialise those twelve. That would have invented
weights to paper over a rename -- precisely what the surrounding refusal
exists to stop -- so it is NOT what this does. A key is moved only when the
class does not want its current name AND does want the renamed one AND nothing
already occupies the target, which makes it a rename of a tensor the
checkpoint has and never the creation of one it lacks. A test pins that: a
genuinely absent parameter must stay absent and the load must still fail.

The converter is a third-party global, so it is wrapped for the duration of
one from_single_file call and restored in a finally.

Version-robust: 9 pass against the image diffusers 0.38, including the real
checkpoint; against 0.37 the class has no prompt_adaln modules at all, so the
repair is correctly a no-op and that test skips rather than asserting a fact
about someone else is version.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZnmWyKjWZerwsb27jZ

codai/models/hf_loading.py

@@ -487,6 +487,104 @@ def build_single_file_pipeline(model_name: str, cfg: Optional[Dict[str, Any]],
           "from_single_file can infer.")
 
 
+#: Segment suffixes diffusers' own converters sometimes carry when the model
+#: class does not. Checked one at a time, both directions, and only ever
+#: applied when BOTH of these hold: the produced key is not wanted by the
+#: class, and the renamed key IS. That makes it a rename of a tensor the
+#: checkpoint already contains -- never the invention of one it lacks.
+_KEY_ALIAS_SUFFIXES = ("_single",)
+
+
+def _rename_unexpected_keys(state, expected):
+    """Rename converter keys the class does not want but would under an alias.
+
+    Returns ``(new_state, [(old, new), ...])``. A key is only moved when the
+    class does not expect its current name AND does expect the renamed one AND
+    nothing already occupies the target, so this can neither overwrite a real
+    weight nor fabricate a missing one.
+    """
+    moved = []
+    out = dict(state)
+    for key in list(out):
+        if key in expected:
+            continue
+        for suf in _KEY_ALIAS_SUFFIXES:
+            parts = key.split(".")
+            hit = None
+            for i, seg in enumerate(parts):
+                if seg.endswith(suf) and len(seg) > len(suf):
+                    cand = ".".join(parts[:i] + [seg[: -len(suf)]] + parts[i + 1:])
+                    if cand in expected and cand not in out:
+                        hit = cand
+                        break
+            if hit:
+                out[hit] = out.pop(key)
+                moved.append((key, hit))
+                break
+    return out, moved
+
+
+def _patch_converter_key_aliases(cls_name, cls, cfg, comp_name):
+    """Repair a converter/class naming disagreement for one load.
+
+    diffusers 0.38 converts an LTX-2 checkpoint's prompt-adaln weights to
+    ``prompt_adaln_single.*`` and ``audio_prompt_adaln_single.*`` while
+    LTX2VideoTransformer3DModel expects ``prompt_adaln.*`` and
+    ``audio_prompt_adaln.*``. Twelve names, one-to-one, the same tensors --
+    so the load failed with "Cannot copy out of meta tensor" for parameters
+    that were present all along, under a name nobody reconciled.
+
+    Measured identically on LTX-2.3-10Eros v1.5 and PinkCherry v1.8.
+
+    The patch wraps that class's registered converter for the duration of ONE
+    ``from_single_file`` call and restores it afterwards; the returned callable
+    is that restore. Returns None when there is nothing to do, which is the
+    case for every model whose converter and class already agree.
+    """
+    try:
+        import json as _json
+        import os as _os
+
+        import torch
+
+        from diffusers.loaders.single_file_model import (
+            SINGLE_FILE_LOADABLE_CLASSES)
+        entry = SINGLE_FILE_LOADABLE_CLASSES.get(cls_name)
+        if not entry or "checkpoint_mapping_fn" not in entry:
+            return None
+        conf = _os.path.join(_component_dir(cfg, comp_name) or "", "config.json")
+        if not _os.path.isfile(conf):
+            return None
+        c = _json.load(open(conf))
+        c.pop("_class_name", None)
+        c.pop("_diffusers_version", None)
+        with torch.device("meta"):
+            probe = cls.from_config(c)
+        expected = set(dict(probe.named_parameters())) | set(
+            dict(probe.named_buffers()))
+        original = entry["checkpoint_mapping_fn"]
+
+        def _wrapped(checkpoint, **kw):
+            out = original(checkpoint, **kw)
+            fixed, moved = _rename_unexpected_keys(out, expected)
+            if moved:
+                groups = sorted({a.split(".")[0] for a, _ in moved})
+                print(f"  [gguf] reconciled {len(moved)} key name(s) this "
+                      f"diffusers' converter and {cls_name} disagree on "
+                      f"({', '.join(groups)}); the tensors are the "
+                      f"checkpoint's own, only the names differed")
+            return fixed
+
+        entry["checkpoint_mapping_fn"] = _wrapped
+
+        def _restore():
+            entry["checkpoint_mapping_fn"] = original
+
+        return _restore
+    except Exception:
+        return None
+
+
 def _missing_tensor_hint(exc, name: str, path: str, cfg) -> str:
     """Turn a meta-tensor failure into the list of parameters actually absent.
 
@@ -603,6 +701,7 @@ def build_gguf_pipeline_components(model_name: str, cfg: Optional[Dict[str, Any]
     # freed a third of what the load actually took. A silent downgrade to
     # something that does not fit is worse than a refusal.
     problems: List[str] = []
+    _undo = None
     for name, path in gguf_entries.items():
         spec = specs.get(name)
         if not spec or spec[0] != 'diffusers':
@@ -617,6 +716,7 @@ def build_gguf_pipeline_components(model_name: str, cfg: Optional[Dict[str, Any]
             continue
         try:
             print(f"  Loading GGUF component '{name}' from {path}")
+            _undo = _patch_converter_key_aliases(cls_name, cls, cfg, name)
             # `config` names the architecture the file's tensors belong to.
             # Without it, from_single_file INFERS one -- and its fallback is
             # Stable Diffusion 1.5, so a Qwen-Image / Flux / Wan transformer
@@ -635,6 +735,13 @@ def build_gguf_pipeline_components(model_name: str, cfg: Optional[Dict[str, Any]
         except Exception as e:
             _hint = _missing_tensor_hint(e, name, path, cfg)
             problems.append(f"{name} ({path}): {e}{_hint}")
+        finally:
+            try:
+                if _undo:
+                    _undo()
+            except Exception:
+                pass
+            _undo = None
     if problems:
         raise RuntimeError(
             "this model configures GGUF component(s) that could not be loaded, "

tests/test_gguf_key_alias_repair.py

+"""A converter and its model class disagreeing about a name is not a missing weight.
+
+diffusers 0.38 converts an LTX-2 checkpoint's prompt-adaln weights to
+`prompt_adaln_single.*` / `audio_prompt_adaln_single.*`, while
+LTX2VideoTransformer3DModel expects `prompt_adaln.*` / `audio_prompt_adaln.*`.
+Twelve names, one-to-one, the same tensors. The load failed with "Cannot copy
+out of meta tensor" for parameters that were present all along -- and the
+tempting "fix" was to initialise them, which would have invented weights to
+paper over a rename.
+
+Measured identically on LTX-2.3-10Eros v1.5 and PinkCherry v1.8: after the
+rename, 0 parameters missing and 0 unused.
+"""
+import pytest
+
+from codai.models.hf_loading import _rename_unexpected_keys
+
+
+def test_a_suffix_only_mismatch_is_renamed():
+    state = {"prompt_adaln_single.linear.weight": 1,
+             "prompt_adaln_single.linear.bias": 2,
+             "blocks.0.attn.to_q.weight": 3}
+    expected = {"prompt_adaln.linear.weight", "prompt_adaln.linear.bias",
+                "blocks.0.attn.to_q.weight"}
+    out, moved = _rename_unexpected_keys(state, expected)
+    assert len(moved) == 2
+    assert set(out) == expected
+    assert out["prompt_adaln.linear.weight"] == 1
+
+
+def test_a_key_the_class_already_wants_is_left_alone():
+    state = {"prompt_adaln.linear.weight": 1}
+    expected = {"prompt_adaln.linear.weight"}
+    out, moved = _rename_unexpected_keys(state, expected)
+    assert moved == []
+    assert out == state
+
+
+def test_nothing_is_invented_for_a_genuinely_absent_parameter():
+    """The property that matters.
+
+    A rename moves a tensor the checkpoint HAS. If the class wants something
+    the checkpoint simply lacks, this must leave it absent so the load still
+    fails loudly -- that is the difference between repairing a name and
+    fabricating a weight.
+    """
+    state = {"blocks.0.attn.to_q.weight": 1}
+    expected = {"blocks.0.attn.to_q.weight", "audio_proj_in.weight"}
+    out, moved = _rename_unexpected_keys(state, expected)
+    assert moved == []
+    assert "audio_proj_in.weight" not in out
+
+
+def test_it_never_overwrites_an_existing_target():
+    # If both spellings are present, the real one wins and nothing is clobbered.
+    state = {"prompt_adaln.linear.weight": "real",
+             "prompt_adaln_single.linear.weight": "other"}
+    expected = {"prompt_adaln.linear.weight"}
+    out, moved = _rename_unexpected_keys(state, expected)
+    assert out["prompt_adaln.linear.weight"] == "real"
+    assert moved == []
+
+
+def test_the_suffix_is_matched_per_segment_not_as_a_substring():
+    # "single" inside a longer word must not be stripped, and only a whole
+    # path segment ending in the suffix is a candidate.
+    state = {"my_single_block.weight": 1}
+    expected = {"my_block.weight"}
+    out, moved = _rename_unexpected_keys(state, expected)
+    assert moved == []
+
+
+def test_an_empty_or_unrelated_state_is_a_no_op():
+    assert _rename_unexpected_keys({}, set()) == ({}, [])
+    s = {"a.b": 1}
+    assert _rename_unexpected_keys(s, {"a.b"}) == (s, [])
+
+
+def test_the_patch_restores_the_converter_it_wrapped():
+    # It wraps a THIRD-PARTY global for the duration of one load; leaving it
+    # wrapped would change every later load in the process.
+    import inspect
+
+    from codai.models import hf_loading as H
+    src = inspect.getsource(H._patch_converter_key_aliases)
+    assert "_restore" in src
+    assert 'entry["checkpoint_mapping_fn"] = original' in src
+    caller = inspect.getsource(H.build_gguf_pipeline_components)
+    assert "finally:" in caller and "_undo()" in caller
+
+
+def test_it_returns_none_when_there_is_nothing_to_reconcile():
+    from codai.models import hf_loading as H
+    # An unknown class has no registered converter, so no patch is installed.
+    assert H._patch_converter_key_aliases("NoSuchModelClass", None, {}, "x") is None
+
+
+@pytest.mark.parametrize("real_gguf", [
+    "/AI/hfcache/guffcache/10Eros_v1.5-Q4_K_M.gguf",
+])
+def test_the_real_checkpoint_maps_exactly_after_the_rename(real_gguf):
+    """End to end on the actual file, when it is present."""
+    import json
+    import os
+    import glob
+    if not os.path.isfile(real_gguf):
+        pytest.skip("checkpoint not on this machine")
+    torch = pytest.importorskip("torch")
+    diffusers = pytest.importorskip("diffusers")
+    gguf = pytest.importorskip("gguf")
+    if not hasattr(diffusers, "LTX2VideoTransformer3DModel"):
+        pytest.skip("this diffusers has no LTX-2 transformer")
+    from diffusers.loaders.single_file_utils import (
+        convert_ltx2_transformer_to_diffusers)
+    conf = glob.glob("/AI/hfcache/huggingface/hub/"
+                     "models--diffusers--LTX-2.3-Diffusers/snapshots/*/"
+                     "transformer/config.json")
+    if not conf:
+        pytest.skip("LTX-2.3 base not on this machine")
+    c = json.load(open(conf[0]))
+    c.pop("_class_name", None)
+    c.pop("_diffusers_version", None)
+    with torch.device("meta"):
+        m = diffusers.LTX2VideoTransformer3DModel.from_config(c)
+    expected = set(dict(m.named_parameters())) | set(dict(m.named_buffers()))
+    sd = {t.name: torch.zeros(1) for t in gguf.GGUFReader(real_gguf).tensors}
+    converted = convert_ltx2_transformer_to_diffusers(sd, config={})
+    before_missing = expected - set(converted)
+    fixed, moved = _rename_unexpected_keys(converted, expected)
+    after_missing = expected - set(fixed)
+
+    # Version-independent invariants. The exact count (12, against diffusers
+    # 0.38) depends on which diffusers is installed -- 0.37 ignores several of
+    # this config's attributes and so builds a different module set -- so what
+    # is asserted here is the PROPERTY, not the number.
+    if not moved:
+        # Nothing to reconcile on THIS diffusers. The mismatch is specific to
+        # versions whose converter and class disagree (0.38 does; 0.37 has no
+        # prompt_adaln modules at all, so the question does not arise). The
+        # repair being a no-op is the correct outcome there, not a failure.
+        pytest.skip(f"diffusers {diffusers.__version__}: converter and class "
+                    f"already agree on every key")
+    for old_key, new_key in moved:
+        assert new_key in expected, new_key          # renamed onto a real slot
+        assert old_key not in expected               # and away from a dead one
+        assert fixed[new_key] is converted[old_key]  # the same tensor, moved
+    assert after_missing < before_missing, "the rename must reduce the gap"
+    assert not (after_missing - before_missing), "nothing new went missing"
+    # And nothing was conjured: every key present afterwards came from the
+    # converter, none were created.
+    assert set(fixed) <= set(converted) | {n for _, n in moved}