give the lip-sync engines an operator, and let a component be one file

Two things that were only reachable from a python -c.

The lip-sync engines had no management surface at all: no status, no install,
no removal. They are not models.json entries on purpose -- each runs in its own
interpreter as a subprocess, so a row in a model section would put them in the
VRAM ledger as something the manager can load, which it cannot -- but that is a
reason to give them their own surface, not none. So: a tab on the models page,
three admin endpoints behind require_admin like every other model action, and
an install that runs in the background while the page polls. The log is kept
because when an install fails, WHERE it failed is the whole answer: a wheel
that will not build reads nothing like a 401. "Rebuild venv" keeps the weights,
since a dependency change does not alter six gigabytes.

`component_models` could only name a repo. A community fine-tune is very often
ONE file -- a retrained transformer with no repo around it, the base supplying
the VAE, the text encoder and the scheduler -- and from_pretrained cannot open
one, so such a component could only be named if somebody had published a
converted directory tree for it. It now accepts a single-file or civitai:
reference.

And the Civitai resolver picked the LARGEST file of a version. Real Dream's
Krea 2 V3 publishes the same weights three times: fp8 13.46 GB, int8 13.18 GB,
nvfp4 8.12 GB. Largest-wins lands on fp8 by a 0.28 GB margin nobody chose, and
could as easily have landed on nvfp4, which needs Blackwell and cannot load on
an Ampere card at all. A reference may now name the file, and without one the
choice is made among the builds this GPU can run, by precision rather than size.

Two smaller ones found by using it. urllib replays headers across a redirect,
so Civitai's signed R2 URL received a bearer token as well and answered 400 --
which looks exactly like a bad token, and meant adding a token BROKE downloads
that had worked without one. curl strips it; now so does this. And the cache
has its own root (CODERAI_CIVITAI_CACHE) instead of living under the HF cache,
which is how a second GGUF directory grew inside one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu

codai/admin/routes.py

@@ -5925,3 +5925,68 @@ async def api_hf_model_info(model_id: str, username: str = Depends(require_admin
         "gguf_files": gguf_files,
         "file_count": len(all_files),
     }
+
+
+# --------------------------------------------------------------------------- #
+# Lip-sync engines (LatentSync / MuseTalk)
+# --------------------------------------------------------------------------- #
+# These are not models.json entries -- they are subprocess engines with their
+# own interpreter, so giving them a row in a model section would put them in
+# the VRAM ledger as something the manager can load, which it cannot. They do
+# need the rest of what every other engine gets: see what is installed, install
+# it, take it away again. Without these three the only way to bring one up was
+# a python -c, which is not a way to run a server.
+
+def _lipsync_engines():
+    from codai.api import lipsync_engines
+    return lipsync_engines
+
+
+@router.get("/admin/api/lipsync-engines", summary="Lip-sync engine status")
+async def api_lipsync_engines(username: str = Depends(require_admin)):
+    """Every managed lip-sync engine, whether it is installed, and what it is
+    short of when it is not."""
+    try:
+        return {"success": True, "engines": _lipsync_engines().all_states()}
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.post("/admin/api/lipsync-engine-install", summary="Install a lip-sync engine")
+async def api_lipsync_engine_install(request: Request,
+                                     username: str = Depends(require_admin)):
+    """Start the install and return at once.
+
+    It is minutes of downloading and a pip resolve, so the caller polls
+    /admin/api/lipsync-engines for the state and the tail of the log. Asking
+    twice joins the running job rather than starting a second one.
+    """
+    data = await request.json()
+    name = (data.get("engine") or data.get("name") or "").strip().lower()
+    try:
+        return {"success": True, "state": _lipsync_engines().start_install(name)}
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.post("/admin/api/lipsync-engine-remove", summary="Remove a lip-sync engine")
+async def api_lipsync_engine_remove(request: Request,
+                                    username: str = Depends(require_admin)):
+    """Delete an engine's venv, and its weights unless `keep_weights` is set.
+
+    Keeping them is the useful default for rebuilding a venv after a dependency
+    change: the weights are gigabytes and have not changed.
+    """
+    data = await request.json()
+    name = (data.get("engine") or data.get("name") or "").strip().lower()
+    keep = bool(data.get("keep_weights"))
+    try:
+        return {"success": True, **_lipsync_engines().uninstall(name, keep_weights=keep)}
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+    except RuntimeError as e:
+        raise HTTPException(status_code=409, detail=str(e))
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))

codai/admin/templates/models.html

@@ -45,6 +45,7 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
 <div class="tabs">
   <button class="tab active" onclick="switchTab('local', this)">Local models</button>
   <button class="tab" onclick="switchTab('search', this)">Find on HuggingFace</button>
+  <button class="tab" onclick="switchTab('lipsync', this); loadLipsyncEngines()">Lip-sync engines</button>
 </div>
 
 <!-- active downloads strip (all tabs) -->
@@ -1841,6 +1842,21 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
   </div>
   <div id="info-body" style="padding:1.25rem 1.5rem"></div>
 </div>
+
+<!-- LIP-SYNC ENGINES -->
+<div id="tab-lipsync" class="tab-panel">
+  <div class="card">
+    <div class="card-title">Lip-sync engines</div>
+    <p style="color:var(--text-2);font-size:13px;margin:.25rem 0 1rem">
+      Video-driven presenter engines. They are not models.json entries: each runs in
+      its own interpreter as a subprocess, so the model manager cannot load one into
+      VRAM and they carry no row in the VRAM ledger. The inference code ships in the
+      repo; installing fetches the weights and builds the engine's venv, which takes
+      minutes and several GB.
+    </p>
+    <div id="lipsync-list">Loading…</div>
+  </div>
+</div>
 {% endblock %}
 
 {% block scripts %}
@@ -1887,6 +1903,90 @@ function switchTab(name,btn){
   document.getElementById('tab-'+name).classList.add('active');
   btn.classList.add('active');
 }
+
+/* ── Lip-sync engines ────────────────────────────────── */
+let _lipsyncPoll = null;
+
+function _lipsyncBadge(e){
+  if (e.state === 'running') return '<span class="badge" style="background:#8a6d1f22;color:#8a6d1f">installing…</span>';
+  if (e.state === 'failed')  return '<span class="badge" style="background:#a3321f22;color:#a3321f">install failed</span>';
+  if (e.available)           return '<span class="badge" style="background:#1f6f4322;color:#1f6f43">ready</span>';
+  if (e.installed)           return '<span class="badge" style="background:#8a6d1f22;color:#8a6d1f">weights present, venv not usable</span>';
+  return '<span class="badge" style="background:#5c616922;color:var(--text-2)">not installed</span>';
+}
+
+function renderLipsyncEngines(engines){
+  const el = document.getElementById('lipsync-list');
+  if (!engines || !engines.length){ el.innerHTML = '<div style="color:var(--text-2)">No managed engines.</div>'; return; }
+  el.innerHTML = engines.map(e => {
+    const busy = e.state === 'running';
+    const missing = (e.missing || []).length
+      ? `<div style="font-size:12px;color:var(--text-2);margin-top:.35rem">Missing: ${e.missing.map(m=>`<code>${m}</code>`).join(', ')}</div>` : '';
+    const err = e.error
+      ? `<div style="font-size:12px;color:#a3321f;margin-top:.35rem">${e.error}</div>` : '';
+    const log = (e.log && e.log.length && (busy || e.state === 'failed'))
+      ? `<pre style="margin:.5rem 0 0;padding:.5rem;background:var(--raised);border-radius:6px;font-size:11px;max-height:9rem;overflow:auto">${e.log.join('\n').replace(/[<&]/g, c => c === '<' ? '&lt;' : '&amp;')}</pre>` : '';
+    return `
+    <div style="border:1px solid var(--border);border-radius:8px;padding:.875rem;margin-bottom:.75rem">
+      <div style="display:flex;align-items:center;gap:.6rem;flex-wrap:wrap">
+        <strong style="font-size:15px">${e.name}</strong>
+        ${_lipsyncBadge(e)}
+        <span style="margin-left:auto;display:flex;gap:.4rem">
+          <button class="btn btn-sm btn-secondary" ${busy?'disabled':''} onclick="installLipsync('${e.name}')">
+            ${e.installed ? 'Reinstall' : 'Install'}</button>
+          <button class="btn btn-sm btn-ghost" ${busy||!e.installed?'disabled':''} onclick="removeLipsync('${e.name}', true)">Rebuild venv</button>
+          <button class="btn btn-sm btn-ghost" ${busy||!e.installed?'disabled':''} onclick="removeLipsync('${e.name}', false)">Remove</button>
+        </span>
+      </div>
+      <div style="font-size:12px;color:var(--text-2);margin-top:.4rem">
+        python ${e.python} · <code>${e.install_dir}</code> · venv <code>${e.venv}</code>
+      </div>
+      <div style="font-size:12px;color:var(--text-2);margin-top:.2rem">
+        weights: ${(e.weights||[]).map(w=>`<code>${w}</code>`).join(', ')}
+      </div>
+      ${missing}${err}${log}
+    </div>`;
+  }).join('');
+}
+
+async function loadLipsyncEngines(){
+  try {
+    const r = await fetch('/admin/api/lipsync-engines');
+    const j = await r.json();
+    renderLipsyncEngines(j.engines || []);
+    // Poll only while something is installing, so an idle page is quiet.
+    const busy = (j.engines || []).some(e => e.state === 'running');
+    if (busy && !_lipsyncPoll) _lipsyncPoll = setInterval(loadLipsyncEngines, 3000);
+    if (!busy && _lipsyncPoll){ clearInterval(_lipsyncPoll); _lipsyncPoll = null; }
+  } catch (err) {
+    document.getElementById('lipsync-list').innerHTML =
+      '<div style="color:#a3321f">Could not read engine status: ' + err + '</div>';
+  }
+}
+
+async function installLipsync(name){
+  const ok = await showConfirm('Install ' + name + '?',
+    'This downloads several GB of weights and builds a dedicated venv. It runs in the background; you can leave this page.');
+  if (!ok) return;
+  await fetch('/admin/api/lipsync-engine-install', {
+    method:'POST', headers:{'Content-Type':'application/json'},
+    body: JSON.stringify({engine:name})});
+  loadLipsyncEngines();
+}
+
+async function removeLipsync(name, keepWeights){
+  const what = keepWeights
+    ? 'Delete the venv and the engine code, keeping the downloaded weights?'
+    : 'Delete the venv, the code AND the downloaded weights?';
+  const ok = await showConfirm('Remove ' + name, what);
+  if (!ok) return;
+  const r = await fetch('/admin/api/lipsync-engine-remove', {
+    method:'POST', headers:{'Content-Type':'application/json'},
+    body: JSON.stringify({engine:name, keep_weights:!!keepWeights})});
+  if (!r.ok){ const j = await r.json().catch(()=>({})); showAlert('Could not remove', j.detail || r.statusText); }
+  loadLipsyncEngines();
+}
+
 function openModal(id){document.getElementById(id).classList.add('show')}
 function closeModal(id){document.getElementById(id).classList.remove('show')}
 

codai/api/lipsync_engines.py

@@ -41,6 +41,7 @@ import shutil
 import subprocess
 import sys
 import threading
+import time
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Callable, Dict, List, Optional, Sequence, Tuple
@@ -701,3 +702,122 @@ def run(name: str, video_path: str, audio_path: str, out_path: str,
         why = (proc.stdout or proc.stderr or "").strip()[-600:]
         raise RuntimeError(
             f"{sp.name} exited 0 but produced no video. Last output: {why}") from None
+
+
+# --------------------------------------------------------------------------- #
+# Management: install as something an operator does, not a Python call
+# --------------------------------------------------------------------------- #
+# An engine coderai can install but has not is neither installed nor absent, and
+# until there was somewhere to act on that distinction the only way to bring one
+# up was `python -c "from codai.api import lipsync_engines; ..."`. Everything
+# else coderai serves is configurable from the models page; these were not.
+
+#: name -> {"state","started","finished","error","log":[...]}. An install is
+#: minutes of downloading and a pip resolve, so it runs in a thread and the
+#: caller polls; the log is kept because when one fails, WHERE it failed is the
+#: whole answer (a wheel that would not build reads nothing like a 401).
+_installs: Dict[str, dict] = {}
+_install_lock = threading.RLock()
+_INSTALL_LOG_LINES = 200
+
+
+def install_state(name: str) -> dict:
+    """Everything the UI needs for one engine, without touching the network."""
+    sp = spec(name)
+    with _install_lock:
+        job = dict(_installs.get(sp.name) or {})
+    job.pop("thread", None)
+    installed = is_installed(sp.name)
+    return {
+        "name": sp.name,
+        "installed": installed,
+        "available": is_available(sp.name) if installed else False,
+        "missing": missing_pieces(sp.name),
+        "install_dir": str(install_dir(sp.name)),
+        "venv": str(venv_dir(sp.name)),
+        "python": sp.py_version,
+        "weights": [ws.repo_id for ws in sp.weights],
+        "state": job.get("state", "idle"),
+        "error": job.get("error"),
+        "log": job.get("log", [])[-20:],
+    }
+
+
+def all_states() -> List[dict]:
+    return [install_state(n) for n in MANAGED]
+
+
+def start_install(name: str) -> dict:
+    """Begin (or re-join) an install. Returns the state immediately."""
+    sp = spec(name)
+    with _install_lock:
+        job = _installs.get(sp.name)
+        if job and job.get("state") == "running":
+            return install_state(sp.name)
+        job = {"state": "running", "started": time.time(), "error": None,
+               "log": []}
+        _installs[sp.name] = job
+
+        def _log(line: str) -> None:
+            with _install_lock:
+                job["log"].append(str(line))
+                del job["log"][:-_INSTALL_LOG_LINES]
+            print(line, flush=True)
+
+        def _run():
+            try:
+                ensure_installed(sp.name, log=_log)
+                with _install_lock:
+                    job["state"] = "done"
+            except Exception as exc:
+                _log(f"[{sp.name}] install failed: {type(exc).__name__}: {exc}")
+                with _install_lock:
+                    job["state"] = "failed"
+                    job["error"] = f"{type(exc).__name__}: {exc}"
+            finally:
+                with _install_lock:
+                    job["finished"] = time.time()
+
+        t = threading.Thread(target=_run, name=f"install-{sp.name}", daemon=True)
+        job["thread"] = t
+        t.start()
+    return install_state(sp.name)
+
+
+def uninstall(name: str, keep_weights: bool = False) -> dict:
+    """Remove an engine's venv, and its install directory unless asked not to.
+
+    ``keep_weights`` exists because the weights are the slow part: an operator
+    rebuilding a venv after a dependency change should not have to fetch 6 GB
+    again. The vendored code is re-seeded from the repo either way.
+    """
+    sp = spec(name)
+    with _install_lock:
+        job = _installs.get(sp.name)
+        if job and job.get("state") == "running":
+            raise RuntimeError(
+                f"{sp.name} is being installed right now; wait for it to finish")
+        _installs.pop(sp.name, None)
+        _ready.pop(sp.name, None)
+    removed = []
+    venv = venv_dir(sp.name)
+    if venv.is_dir():
+        shutil.rmtree(venv, ignore_errors=True)
+        removed.append(str(venv))
+    root = install_dir(sp.name)
+    if root.is_dir():
+        if keep_weights:
+            # The code only: everything the vendored tree put there.
+            for item in sorted(vendor_dir(sp.name).iterdir()):
+                target = root / item.name
+                if target.is_dir():
+                    shutil.rmtree(target, ignore_errors=True)
+                elif target.exists():
+                    target.unlink()
+            _seed_stamp(root).unlink(missing_ok=True)
+            removed.append(f"{root} (code only)")
+        else:
+            shutil.rmtree(root, ignore_errors=True)
+            removed.append(str(root))
+    return {"name": sp.name, "removed": removed,
+            "state": install_state(sp.name)}

tests/test_civitai_single_file.py

@@ -160,3 +160,112 @@ def test_resolve_single_file_routes_civitai_without_touching_hf(monkeypatch, tmp
     monkeypatch.setattr(H, "resolve_civitai", _fake)
     assert H.resolve_single_file("civitai:4242") == "/tmp/x.safetensors"
     assert called["ref"] == "civitai:4242"
+
+
+# ===================================== choosing WHICH file of a version to get
+# A Civitai version is not one checkpoint. Real Dream's Krea 2 V3 publishes the
+# same weights three times -- fp8 13.46 GB, int8 13.18 GB, nvfp4 8.12 GB -- and
+# they are not interchangeable: nvfp4 needs Blackwell, so on an Ampere card it
+# is 8 GB downloaded to produce a load error. Taking the largest, as this used
+# to, lands on fp8 by a 0.28 GB margin nobody chose.
+
+REAL_DREAM = [
+    {"id": 3286566, "name": "fp8.safetensors", "sizeKB": 13.46e6,
+     "metadata": {"fp": "fp8"}},
+    {"id": 3289346, "name": "nvfp4.safetensors", "sizeKB": 8.12e6,
+     "metadata": {"fp": "nvfp4"}},
+    {"id": 3304483, "name": "int8.safetensors", "sizeKB": 13.18e6,
+     "metadata": {"fp": "int8"}},
+]
+
+
+@pytest.fixture
+def ampere(monkeypatch):
+    """A card that is not Blackwell — which is every card this runs on today."""
+    import codai.models.hf_loading as H
+    monkeypatch.setattr(H, "_fp_is_runnable_here",
+                        lambda fp: (fp or "").lower() not in ("nvfp4", "fp4"))
+
+
+def test_an_explicit_file_id_is_obeyed(ampere):
+    from codai.models.hf_loading import _pick_civitai_file
+    got = _pick_civitai_file(REAL_DREAM, "3396368", "3286566")
+    assert got["name"] == "fp8.safetensors"
+
+
+def test_without_a_file_id_precision_decides_rather_than_size(ampere):
+    from codai.models.hf_loading import _pick_civitai_file
+    got = _pick_civitai_file(REAL_DREAM, "3396368", None)
+    assert got["name"] == "int8.safetensors"
+
+
+def test_a_build_this_gpu_cannot_run_is_never_chosen(ampere):
+    from codai.models.hf_loading import _pick_civitai_file
+    only_nvfp4 = [f for f in REAL_DREAM if f["metadata"]["fp"] == "nvfp4"]
+    with pytest.raises(RuntimeError) as e:
+        _pick_civitai_file(only_nvfp4, "3396368", None)
+    assert "nvfp4" in str(e.value)
+    # ...but naming it explicitly still works, for the operator who knows better.
+    assert _pick_civitai_file(only_nvfp4, "3396368", "3289346")["name"] \
+        == "nvfp4.safetensors"
+
+
+def test_a_file_id_that_is_not_there_lists_what_is(ampere):
+    from codai.models.hf_loading import _pick_civitai_file
+    with pytest.raises(RuntimeError) as e:
+        _pick_civitai_file(REAL_DREAM, "3396368", "123")
+    msg = str(e.value)
+    assert "3304483" in msg and "int8" in msg
+
+
+def test_an_unlabelled_file_is_not_rejected(ampere):
+    from codai.models.hf_loading import _pick_civitai_file
+    plain = [{"id": 1, "name": "x.safetensors", "sizeKB": 1e6, "metadata": {}}]
+    assert _pick_civitai_file(plain, "1", None)["name"] == "x.safetensors"
+
+
+@pytest.mark.parametrize("ref,vid,fid", [
+    ("civitai:3396368", "3396368", None),
+    ("civitai:3396368:3304483", "3396368", "3304483"),
+])
+def test_a_reference_may_name_the_file_as_well_as_the_version(ref, vid, fid):
+    from codai.models.hf_loading import is_single_file_ref
+    assert is_single_file_ref(ref) is True
+
+
+def test_a_non_numeric_file_id_is_refused():
+    with pytest.raises(ValueError) as e:
+        resolve_civitai("civitai:3396368:abc")
+    assert "fileId" in str(e.value)
+
+
+def test_two_precisions_of_one_version_cache_side_by_side(monkeypatch, tmp_path):
+    """Without a per-file directory the "already downloaded?" check answers with
+    whichever build happens to be biggest — the bug one level down."""
+    import codai.models.hf_loading as H
+    monkeypatch.setattr(H, "civitai_cache_root", lambda: tmp_path)
+    for fid, name, size in (("3304483", "int8.safetensors", 2_000_000),
+                            ("3286566", "fp8.safetensors", 3_000_000)):
+        d = tmp_path / "3396368" / fid
+        d.mkdir(parents=True)
+        (d / name).write_bytes(b"x" * size)
+    import urllib.request
+    monkeypatch.setattr(urllib.request, "urlopen",
+                        lambda *a, **k: pytest.fail("re-downloaded a cached file"))
+    assert resolve_civitai("civitai:3396368:3304483").endswith("int8.safetensors")
+    assert resolve_civitai("civitai:3396368:3286566").endswith("fp8.safetensors")
+
+
+def test_the_cache_root_can_be_pointed_at_the_weights_disk(monkeypatch):
+    from codai.models.hf_loading import civitai_cache_root
+    monkeypatch.setenv("CODERAI_CIVITAI_CACHE", "/AI/civitaicache")
+    assert str(civitai_cache_root()) == "/AI/civitaicache"
+
+
+def test_the_cache_root_is_not_derived_from_the_hf_cache(monkeypatch):
+    """Putting multi-GB checkpoints under the HF cache root is how a second GGUF
+    directory grew inside one."""
+    from codai.models.hf_loading import civitai_cache_root
+    monkeypatch.delenv("CODERAI_CIVITAI_CACHE", raising=False)
+    monkeypatch.setenv("HF_HOME", "/AI/hfcache/huggingface")
+    assert "hfcache" not in str(civitai_cache_root())

tests/test_lipsync_engines.py

@@ -370,3 +370,145 @@ def test_the_sweep_never_removes_the_run_being_set_up(le, tmp_path):
 
 def test_the_sweep_is_silent_when_there_is_nothing_to_sweep(le, tmp_path):
     assert le._sweep_run_dirs(tmp_path / "never-made") == 0
+
+
+# ── management: an operator's surface, not a python -c ───────────────────────
+
+def test_an_engine_reports_everything_the_ui_needs_without_the_network(le,
+                                                                       monkeypatch,
+                                                                       tmp_path):
+    monkeypatch.setattr(le, "install_dir", lambda n: tmp_path / n)
+    monkeypatch.setattr(le, "venv_dir", lambda n: tmp_path / (n + "_venv"))
+    import urllib.request
+    monkeypatch.setattr(urllib.request, "urlopen",
+                        lambda *a, **k: pytest.fail("status touched the network"))
+    s = le.install_state("latentsync")
+    assert s["name"] == "latentsync"
+    assert s["installed"] is False and s["available"] is False
+    assert s["state"] == "idle"
+    assert s["python"] == "3.11"
+    assert any("LatentSync" in w for w in s["weights"])
+    assert "checkpoints/latentsync_unet.pt" in s["missing"]
+
+
+def test_every_managed_engine_has_a_state(le):
+    assert {s["name"] for s in le.all_states()} == set(le.MANAGED)
+
+
+def test_an_install_runs_in_the_background_and_reports_as_it_goes(le, monkeypatch):
+    import threading
+    released = threading.Event()
+
+    def _slow(name, log=None):
+        log("working")
+        released.wait(5)
+        log("finished")
+        return ("root", "py")
+
+    monkeypatch.setattr(le, "ensure_installed", _slow)
+    monkeypatch.setattr(le, "_installs", {}, raising=False)
+    st = le.start_install("latentsync")
+    assert st["state"] == "running"
+    # Asking again joins the running job rather than starting a second.
+    assert le.start_install("latentsync")["state"] == "running"
+    released.set()
+    for _ in range(100):
+        if le.install_state("latentsync")["state"] == "done":
+            break
+        import time as _t
+        _t.sleep(0.05)
+    done = le.install_state("latentsync")
+    assert done["state"] == "done"
+    assert "working" in done["log"] and "finished" in done["log"]
+
+
+def test_a_failed_install_keeps_the_reason_where_it_can_be_read(le, monkeypatch):
+    def _boom(name, log=None):
+        log("fetching")
+        raise RuntimeError("no wheel for this host")
+
+    monkeypatch.setattr(le, "ensure_installed", _boom)
+    monkeypatch.setattr(le, "_installs", {}, raising=False)
+    le.start_install("musetalk")
+    for _ in range(100):
+        if le.install_state("musetalk")["state"] in ("failed", "done"):
+            break
+        import time as _t
+        _t.sleep(0.05)
+    s = le.install_state("musetalk")
+    assert s["state"] == "failed"
+    assert "no wheel for this host" in s["error"]
+    assert any("fetching" in line for line in s["log"])
+
+
+def test_the_state_never_carries_a_thread_object(le, monkeypatch):
+    """It is serialised to JSON for the admin page."""
+    import json
+    monkeypatch.setattr(le, "ensure_installed", lambda name, log=None: ("r", "p"))
+    monkeypatch.setattr(le, "_installs", {}, raising=False)
+    le.start_install("latentsync")
+    json.dumps(le.install_state("latentsync"))      # must not raise
+
+
+def test_removing_an_engine_takes_the_venv_and_the_weights(le, monkeypatch,
+                                                            tmp_path):
+    root, venv = tmp_path / "latentsync", tmp_path / "latentsync_venv"
+    (root / "checkpoints").mkdir(parents=True)
+    (root / "checkpoints" / "latentsync_unet.pt").write_bytes(b"w")
+    venv.mkdir()
+    monkeypatch.setattr(le, "install_dir", lambda n: root)
+    monkeypatch.setattr(le, "venv_dir", lambda n: venv)
+    monkeypatch.setattr(le, "_installs", {}, raising=False)
+    le.uninstall("latentsync")
+    assert not root.exists() and not venv.exists()
+
+
+def test_rebuilding_a_venv_keeps_the_gigabytes(le, monkeypatch, tmp_path):
+    """The weights are the slow part and a dependency change does not alter
+    them, so a venv rebuild must not re-download 6 GB."""
+    root, venv = tmp_path / "latentsync", tmp_path / "latentsync_venv"
+    (root / "checkpoints").mkdir(parents=True)
+    kept = root / "checkpoints" / "latentsync_unet.pt"
+    kept.write_bytes(b"w")
+    (root / "scripts").mkdir()
+    (root / "scripts" / "inference.py").write_text("x")
+    le._seed_stamp(root).write_text("stamp")
+    venv.mkdir()
+    monkeypatch.setattr(le, "install_dir", lambda n: root)
+    monkeypatch.setattr(le, "venv_dir", lambda n: venv)
+    monkeypatch.setattr(le, "_installs", {}, raising=False)
+    le.uninstall("latentsync", keep_weights=True)
+    assert kept.exists(), "the weights were deleted on a venv rebuild"
+    assert not venv.exists()
+    assert not (root / "scripts").exists(), "the code should be re-seeded"
+    assert not le._seed_stamp(root).exists()
+
+
+def test_an_engine_cannot_be_removed_while_it_installs(le, monkeypatch):
+    monkeypatch.setattr(le, "_installs", {"latentsync": {"state": "running"}},
+                        raising=False)
+    with pytest.raises(RuntimeError) as e:
+        le.uninstall("latentsync")
+    assert "being installed" in str(e.value)
+
+
+def test_the_admin_routes_exist_and_are_admin_only():
+    import inspect
+    from codai.admin import routes
+    src = inspect.getsource(routes)
+    for path in ("/admin/api/lipsync-engines",
+                 "/admin/api/lipsync-engine-install",
+                 "/admin/api/lipsync-engine-remove"):
+        assert path in src, path
+    # Each one behind require_admin, like every other model action.
+    tail = src[src.index("/admin/api/lipsync-engines"):]
+    assert tail.count("Depends(require_admin)") >= 3
+
+
+def test_the_models_page_offers_the_engines():
+    page = (ROOT / "codai" / "admin" / "templates" / "models.html").read_text(
+        encoding="utf-8")
+    assert 'id="tab-lipsync"' in page
+    assert "loadLipsyncEngines" in page
+    assert "lipsync-engine-install" in page
+    assert "lipsync-engine-remove" in page