stop discarding the reason a GGUF would not load, and give every TTS backend the card's path

**Five models "won't load" and llama.cpp had said why.** The load-phase log
callback printed only lines matching a keyword list about layers, buffers and
devices. llama.cpp writes its diagnosis -- an unknown architecture, a missing
tensor -- in none of those words, so it matched nothing and was dropped, while
llama-cpp-python raised its one generic sentence: "Failed to load model from
file: <path>". That string is all the client got, and all the log had.

So every line is now kept in a 60-line ring, lines that look like a diagnosis
are printed whether or not they mention a buffer, and the raised error carries
llama.cpp's own words. A model this build genuinely cannot read is no longer
indistinguishable from one that did not fit, a bad path, or a corrupt download.

The retry ladder stops earlier too. It exists to answer one question -- does it
fit? -- by moving layers to the host, and an attempt with NO layers on the GPU
has already answered it. Three MTP/next-N checkpoints each burned six attempts
down to n_gpu_layers=0, every one failing in a fraction of a second with the
same message: 18 loads whose only effect was to push the reason further up the
log. (What those three have in common is visible in their metadata --
qwen35.nextn_predict_layers=1, blk.N.layer_output_scale.weight -- and with the
reason no longer discarded, the next attempt will say so outright instead of
leaving it to be inferred.)

**XTTS and parler were handed an alias instead of a repo id.** _load_backend
passed `model_path or model_name` to Kokoro and the bare `model_name` to
everyone else, and for a card registered under an alias that name is the alias.
Each isolated worker then failed in its own vocabulary, neither mentioning the
cause:

  XTTS:   "XTTS venv could not load 'XTTS-v2': ValueError: not enough values to
          unpack (expected 4, got 1)" -- coqui splits a model name into
          type/lang/dataset/model, and the bare alias has no separator at all.
  parler: "parler-tts-mini-multilingual is not the path to a directory
          containing a file named config.json" -- the org is missing, so the
          real repo is never looked up, and the worker reports a model that IS
          cached as one to download first.

Every backend that resolves a checkpoint now gets the path. The XTTS worker's
own translation table is fixed to match on the last path segment as well, since
it is reached by other callers too: it required a "/" before, which is precisely
the case that does not apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.5.29"
+__version__ = "0.5.30"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/tts_backends.py

@@ -632,12 +632,30 @@ def load_backend(model_name: str, model_path: Optional[str], config: Optional[di
 def _load_backend(model_name: str, model_path: Optional[str], config: Optional[dict]):
     fam = _family(model_name)
     config = config or {}
+    # Every backend that has to RESOLVE a checkpoint gets the card's path, with
+    # the request's name only as a fallback. `model_name` is whatever the caller
+    # asked for, which for a card registered under an alias is the alias -- and
+    # an alias is not a repo id. Kokoro already did this; the others took
+    # `model_name`, and both isolated workers failed on it, each in its own way:
+    #
+    #   XTTS:   "XTTS venv could not load 'XTTS-v2': ValueError: not enough
+    #            values to unpack (expected 4, got 1)" -- coqui splits a model
+    #            name into type/lang/dataset/model, and the bare alias has no
+    #            separator at all (the HF id 'coqui/XTTS-v2' has one, which is
+    #            the case the worker's translation table was written for).
+    #   parler: "parler-tts-mini-multilingual is not the path to a directory
+    #            containing a file named config.json" -- the org is missing, so
+    #            the real repo 'parler-tts/parler-tts-mini-multilingual' is
+    #            never looked up, and the worker reports it as "not in the cache,
+    #            download it from the model interface first" for a model that IS
+    #            in the cache.
+    _ckpt = model_path or model_name
     if fam == "kokoro":
-        return _KokoroBackend(model_path or model_name, config)
+        return _KokoroBackend(_ckpt, config)
     if fam == "coqui":
-        return _CoquiBackend(model_name, config)
+        return _CoquiBackend(_ckpt, config)
     if fam == "bark":
-        return _BarkBackend(model_name, config)
+        return _BarkBackend(_ckpt, config)
     if fam == "melo":
         # MeloTTS is OpenVoice V2's base speaker model. Its pins conflict with this
         # stack the way parler's do, so it runs in an isolated venv worker and is
@@ -645,9 +663,9 @@ def _load_backend(model_name: str, model_path: Optional[str], config: Optional[d
         if config.get("service_url"):
             return _RemoteSpeechBackend(config, family="melo")
         from codai.api import melotts_worker
-        url = melotts_worker.ensure_service(model_name)
+        url = melotts_worker.ensure_service(_ckpt)
         return _RemoteSpeechBackend({**config, "service_url": url}, family="melo",
-                                    managed_model=model_name)
+                                    managed_model=_ckpt)
     if fam == "parler":
         # An explicit service_url points at an externally-run service. Otherwise
         # coderai fully manages the worker: bootstrap its venv, spawn it, and
@@ -655,10 +673,10 @@ def _load_backend(model_name: str, model_path: Optional[str], config: Optional[d
         if config.get("service_url"):
             return _RemoteSpeechBackend(config, family="parler")
         from codai.api import parler_worker
-        url = parler_worker.ensure_service(model_name)
+        url = parler_worker.ensure_service(_ckpt)
         return _RemoteSpeechBackend({**config, "service_url": url},
-                                    family="parler", managed_model=model_name)
-    return _TransformersBackend(model_name, config)
+                                    family="parler", managed_model=_ckpt)
+    return _TransformersBackend(_ckpt, config)
 
 
 # --------------------------------------------------------------------------- #

codai/backends/vulkan.py

@@ -22,6 +22,7 @@ import json
 import re
 import threading
 import time
+from collections import deque
 from typing import AsyncIterator, Optional, Union, List, Dict, Any
 from pathlib import Path
 
@@ -611,6 +612,33 @@ _KV_OBSERVED: dict = {"gpu": 0.0, "cpu": 0.0}
 _KV_BUF_RE = re.compile(
     r"llama_kv_cache[^:]*:\s*(\S+)\s+KV buffer size\s*=\s*([0-9.]+)\s*MiB")
 
+#: The last lines llama.cpp logged during the CURRENT load, whatever they said.
+#: llama-cpp-python raises one generic sentence for every failure -- "Failed to
+#: load model from file: <path>" -- while the REASON is written by llama.cpp to
+#: its log callback. The callback below used to print only lines matching a
+#: keyword list about layers and buffers, so an error naming the architecture or
+#: the missing tensor matched nothing and was dropped. Four models then failed
+#: with no reason anywhere, through six retries each.
+_LOG_TAIL: deque = deque(maxlen=60)
+
+#: Lines worth surfacing even though they say nothing about layers or buffers.
+_LOG_TROUBLE = ("error", "failed", "unsupported", "unknown", "missing",
+                "not supported", "cannot", "invalid", "corrupt", "assert")
+
+
+def llama_log_tail(limit: int = 6, trouble_only: bool = True) -> str:
+    """The tail of llama.cpp's own log for the load just attempted.
+
+    ``trouble_only`` keeps the lines that look like a diagnosis, which is almost
+    always what a failure needs; the whole tail is there when it is not.
+    """
+    lines = list(_LOG_TAIL)
+    if trouble_only:
+        picked = [ln for ln in lines
+                  if any(t in ln.lower() for t in _LOG_TROUBLE)]
+        lines = picked or lines
+    return " | ".join(lines[-max(1, int(limit)):])
+
 
 def _install_layer_log_callback():
     """Replace llama.cpp's log callback with one that prints load-time layer/buffer
@@ -625,6 +653,7 @@ def _install_layer_log_callback():
         return None
     _KV_OBSERVED["gpu"] = 0.0
     _KV_OBSERVED["cpu"] = 0.0
+    _LOG_TAIL.clear()
 
     # Keywords that identify interesting load-phase messages
     _KEEP = (
@@ -642,7 +671,13 @@ def _install_layer_log_callback():
             if _m:
                 _where = "cpu" if _m.group(1).upper().startswith("CPU") else "gpu"
                 _KV_OBSERVED[_where] += float(_m.group(2)) / 1024.0
-            if msg and any(k in msg for k in _KEEP):
+            if msg:
+                # EVERY line is kept, so a failure can be explained afterwards
+                # even when the line said nothing this filter was looking for.
+                _LOG_TAIL.append(msg)
+            _low = msg.lower()
+            if msg and (any(k in msg for k in _KEEP)
+                        or any(t in _low for t in _LOG_TROUBLE)):
                 print(f"  [llama.cpp] {msg}", flush=True)
         except Exception:
             pass
@@ -1938,10 +1973,28 @@ class VulkanBackend(ModelBackend):
                     break
                 except Exception as e:
                     _last_err = e
+                    _why = llama_log_tail()
                     print(f"Error loading GGUF model "
-                          f"(attempt {_attempt + 1}/{_max_attempts}): {e}")
+                          f"(attempt {_attempt + 1}/{_max_attempts}): {e}"
+                          + (f"\n  llama.cpp said: {_why}" if _why else ""))
                     if _attempt + 1 >= _max_attempts:
                         break
+                    # The ladder exists to answer ONE question -- does it fit? --
+                    # by moving layers to the host. An attempt with NO layers on
+                    # the GPU has already answered it: whatever is wrong is not
+                    # VRAM, and four more attempts cannot discover that. Measured
+                    # on three MTP/next-N checkpoints: six attempts each, down to
+                    # n_gpu_layers=0, every one failing in ~0.2 s with the same
+                    # generic "Failed to load model from file" -- 18 loads whose
+                    # only effect was to bury the reason further up the log.
+                    _was = llama_kwargs.get('n_gpu_layers', self.n_gpu_layers)
+                    if _was == 0:
+                        print(f"  not retrying: this attempt had every layer on "
+                              f"the CPU and still failed, so the model is not "
+                              f"too big for the GPU — llama.cpp cannot load this "
+                              f"file at all"
+                              + (f" ({_why})" if _why else ""))
+                        break
                     _cur = llama_kwargs.get('n_gpu_layers', self.n_gpu_layers)
                     _tot = _block_count or (_cur if (_cur and _cur > 0) else 40)
                     if _cur is None or _cur < 0:
@@ -1953,7 +2006,17 @@ class VulkanBackend(ModelBackend):
                     print(f"  Retrying with n_gpu_layers={_new} "
                           f"({max(0, _tot - _new)}/{_tot} layers on CPU)…")
             if self.model is None:
-                print(f"Error loading model after {_max_attempts} attempts: {_last_err}")
+                # Raise with llama.cpp's own words attached. The exception
+                # llama-cpp-python produces names the file and nothing else, and
+                # that string is what reaches the client -- so a model that this
+                # build genuinely cannot read was indistinguishable from one that
+                # did not fit, from a bad path, or from a corrupt download.
+                _why = llama_log_tail(limit=4)
+                print(f"Error loading model after {_max_attempts} attempts: "
+                      f"{_last_err}" + (f"\n  llama.cpp said: {_why}" if _why else ""))
+                if _why and str(_why) not in str(_last_err):
+                    raise RuntimeError(f"{_last_err} — llama.cpp reported: {_why}") \
+                        from _last_err
                 raise _last_err
         finally:
             _params_mod.llama_model_default_params = _orig_params

tests/test_full_pass_regressions.py

@@ -202,3 +202,117 @@ def test_an_exact_name_wins_even_unhealthy(registry_with_a_split_engine):
     up = _FakeEngine("nvidia-gguf", "nvidia", healthy=True)
     reg._engines = {1: up, 2: down}
     assert reg.by_name("nvidia") is down
+
+
+# ---------------------------------------------------------------------------
+# "Five models won't load" -- llama.cpp said why and the filter dropped it
+# ---------------------------------------------------------------------------
+
+def test_llama_cpp_log_lines_are_all_kept():
+    """The callback printed only lines matching a keyword list about layers and
+    buffers. An error naming the architecture or a missing tensor matches none
+    of them, so three MTP checkpoints failed with no reason anywhere in the log
+    and llama-cpp-python's one generic sentence reached the client."""
+    from codai.backends import vulkan as V
+    V._LOG_TAIL.clear()
+    V._LOG_TAIL.extend([
+        "llama_model_loader: loaded meta data",
+        "load_tensors: layer 0 assigned to device CUDA0",
+        "error loading model: missing tensor 'blk.0.nextn.weight'",
+    ])
+    tail = V.llama_log_tail()
+    assert "missing tensor" in tail
+
+
+def test_the_tail_prefers_lines_that_look_like_a_diagnosis():
+    """A 60-line ring ending in routine chatter must still surface the error."""
+    from codai.backends import vulkan as V
+    V._LOG_TAIL.clear()
+    V._LOG_TAIL.append("unknown model architecture: 'qwen35moe'")
+    for i in range(40):
+        V._LOG_TAIL.append(f"load_tensors: layer {i} assigned to device CUDA0")
+    tail = V.llama_log_tail()
+    assert "unknown model architecture" in tail
+
+
+def test_the_whole_tail_is_available_when_nothing_looks_wrong():
+    from codai.backends import vulkan as V
+    V._LOG_TAIL.clear()
+    V._LOG_TAIL.extend(["routine one", "routine two"])
+    assert "routine two" in V.llama_log_tail()
+    assert "routine two" in V.llama_log_tail(trouble_only=False)
+
+
+def test_an_empty_log_yields_no_claim():
+    from codai.backends import vulkan as V
+    V._LOG_TAIL.clear()
+    assert V.llama_log_tail() == ""
+
+
+def test_the_ladder_stops_once_cpu_only_has_failed():
+    """The retry ladder answers "does it fit?" by moving layers to the host. An
+    attempt with no layers on the GPU has answered it — the three MTP models
+    each burned six attempts down to n_gpu_layers=0, every one failing in a
+    fraction of a second with the same generic message."""
+    src = _src("codai/backends/vulkan.py")
+    assert "not retrying: this attempt had every layer on" in src
+    head = src.index("Error loading GGUF model ")
+    body = src[head:head + 2000]
+    assert "if _was == 0:" in body
+
+
+def test_the_raised_error_carries_llama_cpps_words():
+    src = _src("codai/backends/vulkan.py")
+    assert "llama.cpp reported:" in src
+
+
+def test_trouble_lines_are_printed_even_when_no_keyword_matches():
+    src = _src("codai/backends/vulkan.py")
+    head = src.index("def _install_layer_log_callback")
+    body = src[head:src.index("async def _aiter_blocking")]
+    assert "_LOG_TROUBLE" in body
+    assert "_LOG_TAIL.append(msg)" in body
+
+
+# ---------------------------------------------------------------------------
+# "XTTS-v2 and parler don't load" -- both got an alias, not a repo id
+# ---------------------------------------------------------------------------
+
+def test_every_tts_backend_is_given_the_cards_path():
+    """Kokoro already took `model_path or model_name`; coqui, bark, parler, melo
+    and the transformers fallback took the request's name, which for a card
+    registered under an alias is the alias."""
+    src = _src("codai/api/tts_backends.py")
+    head = src.index("def _load_backend(")
+    body = src[head:src.index("# encoding", head)]
+    assert "_ckpt = model_path or model_name" in body
+    assert "_CoquiBackend(_ckpt" in body
+    assert "_BarkBackend(_ckpt" in body
+    assert "_KokoroBackend(_ckpt" in body
+    assert "_TransformersBackend(_ckpt" in body
+    assert "parler_worker.ensure_service(_ckpt)" in body
+    assert "melotts_worker.ensure_service(_ckpt)" in body
+    # And no backend is left taking the bare request name.
+    assert "_CoquiBackend(model_name" not in body
+    assert "_BarkBackend(model_name" not in body
+
+
+def test_the_xtts_worker_recognises_an_org_less_alias():
+    """'XTTS venv could not load XTTS-v2: not enough values to unpack
+    (expected 4, got 1)' — coqui splits a name into four parts and the bare
+    alias has no separator at all. The old test required a '/', which is
+    exactly the case that does not apply."""
+    import importlib.util
+    spec = importlib.util.spec_from_file_location(
+        "_xtts_svc", os.path.join(_ROOT, "tools/xtts_service.py"))
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    four = "tts_models/multilingual/multi-dataset/xtts_v2"
+    assert mod._coqui_name("XTTS-v2") == four
+    assert mod._coqui_name("coqui/XTTS-v2") == four
+    assert mod._coqui_name("xtts_v2") == four
+    assert mod._coqui_name("xtts_v1.1").endswith("xtts_v1.1")
+    assert mod._coqui_name("coqui/xtts-v1.1").endswith("xtts_v1.1")
+    # Already coqui's own four-part naming, and an unrelated model: untouched.
+    assert mod._coqui_name(four) == four
+    assert mod._coqui_name("some/other-model") == "some/other-model"

tools/xtts_service.py

@@ -48,7 +48,17 @@ def _coqui_name(name: str) -> str:
     mapped = _HF_TO_COQUI.get(n.lower())
     if mapped:
         return mapped
-    if "/" in n and "xtts" in n.lower():
+    # Match on the LAST path segment, so an org-less alias is recognised too.
+    # The previous test required a "/", which is exactly the case that does not
+    # apply to a card registered as "XTTS-v2": coqui then split a name with no
+    # separator and reported "not enough values to unpack (expected 4, got 1)",
+    # an error about its own four-part naming that says nothing about the alias
+    # that caused it. The caller passes the card's path now, but the worker is
+    # reached by other paths too and must not depend on that.
+    tail = n.rsplit("/", 1)[-1].lower().replace("_", "-")
+    if "xtts" in tail:
+        if "v1.1" in tail or "v1-1" in tail:
+            return "tts_models/multilingual/multi-dataset/xtts_v1.1"
         # An unknown XTTS repo: the v2 architecture is the safe assumption.
         return "tts_models/multilingual/multi-dataset/xtts_v2"
     return n