thermal: attribute CPU heat per-engine + graceful colibri pause; add credits

Fixes the "only radeon was stopped while everything under throttle" case and gives
colibri a clean pause instead of SIGSTOP.

Thermal (engine_supervisor):
- Per-engine CPU gating: measure each engine's process-group CPU (rolling max) and
  apply the GLOBAL cpu_hot/cpu_warm term ONLY to real CPU-heat sources. A GPU-bound
  engine (embeddings) is never paused/stranded in the resume<CPU<high dead-band that
  a CPU-heavy engine (colibri) keeps the shared CPU parked in.
- Targeted SIGSTOP: _thermal_signal now freezes the engine's native compute CHILDREN
  (colibri/ds4/whisper-server), leaving the Python HTTP server alive to ack
  pause/resume — killing the resume-fail / SIGSTOP↔SIGCONT thrash. Falls back to
  killpg for in-process (torch) engines.

colibri graceful PAUSE/RESUME:
- packaging/patch-colibri.py: idempotent serve-mux patch adding PAUSE/RESUME control
  frames — colibri idles the decode loop between tokens (keeping KV) instead of being
  frozen; build.sh applies it after clone.
- MuxEngine.pause()/resume() + colibri_worker.pause_all()/resume_all(); the engine's
  /internal/thermal-pause|-resume now drives them, so the cooperative thermal throttle
  can cool the box without SIGSTOP.

Credits (requested): clear acknowledgements for the brilliant engines coderai builds
on — colibri (JustVugg), ds4/DwarfStar (antirez), llama.cpp/whisper.cpp (Gerganov),
and VPR/geo research — in the admin settings UI, README, and docs (new
docs/glm-colibri.md, ds4 doc footer).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty

README.md

@@ -694,8 +694,23 @@ Merge requests welcome.
 
 ## Acknowledgments
 
+CoderAI stands on the shoulders of remarkable open-source work. Special, heartfelt
+thanks to the developers of the native inference engines that let it punch far above
+its hardware — running models that otherwise simply couldn't run on a single GPU:
+
+- **[colibri](https://github.com/JustVugg/colibri)** by **JustVugg** — a brilliant
+  pure-C MoE engine that streams **GLM-5.2 (744B params)** across VRAM/RAM/disk to run
+  it on a single consumer GPU.
+- **[ds4 / DwarfStar](https://github.com/antirez/ds4)** by **Salvatore Sanfilippo
+  (antirez)** — a superb from-scratch **DeepSeek-V4** inference engine.
+- **[llama.cpp](https://github.com/ggml-org/llama.cpp)** &
+  **[whisper.cpp](https://github.com/ggml-org/whisper.cpp)** by **Georgi Gerganov**
+  and contributors — the foundational GGUF LLM inference and Whisper STT engines.
+
+And the libraries, models and research CoderAI builds on:
+
 - [FastAPI](https://fastapi.tiangolo.com/)
-- [HuggingFace Transformers](https://huggingface.co/docs/transformers/) — NVIDIA text backend
+- [HuggingFace Transformers](https://huggingface.co/docs/transformers/) & [Diffusers](https://github.com/huggingface/diffusers) — NVIDIA text/image backends
 - [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) — Vulkan/CUDA GGUF text backend
 - [stable-diffusion-cpp-python](https://github.com/william-murray1204/stable-diffusion-cpp-python) — GGUF image backend
 - [InsightFace](https://github.com/deepinsight/insightface) — face swap
@@ -704,3 +719,4 @@ Merge requests welcome.
 - [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) — image/video upscaling
 - [Wav2Lip](https://github.com/Rudrabha/Wav2Lip) — audio-driven lip sync
 - [SadTalker](https://github.com/OpenTalker/SadTalker) — talking head / lip sync generation
+- Visual place recognition & geolocation research — [EigenPlaces](https://github.com/gmberton/EigenPlaces) (Gabriele Berton et al.), [DINOv2-SALAD](https://github.com/serizba/salad) (Sergio Izquierdo, Javier Civera), [GeoCLIP](https://github.com/VicenteVivan/geo-clip) (Vicente Vivanco et al.), [DINOv2](https://github.com/facebookresearch/dinov2) (Meta AI)

build.sh

@@ -806,6 +806,12 @@ build_colibri() {
         git clone --depth 1 https://github.com/JustVugg/colibri "$COLIBRI_DIR" || {
             echo -e "${YELLOW}Warning: could not clone colibri; skipping.${NC}"; return 0; }
     fi
+    # Apply coderai's serve-mux PAUSE/RESUME patch (idempotent) so thermal throttle can
+    # gracefully idle in-flight generation instead of SIGSTOP-freezing the engine.
+    if [ -f "$(dirname "$0")/packaging/patch-colibri.py" ]; then
+        python3 "$(dirname "$0")/packaging/patch-colibri.py" "$COLIBRI_DIR/c/colibri.c" || \
+            echo -e "${YELLOW}Warning: colibri PAUSE/RESUME patch failed (building unpatched).${NC}"
+    fi
     # Default to a PORTABLE CUDA arch (SASS for Ampere..Blackwell + PTX fallback), like
     # ds4's cuda-generic, so the bundled binary isn't locked to the build host's GPU.
     # Requires a real nvcc (a CUDA *runtime* has /usr/local/cuda but no compiler); the

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.1.74"
+__version__ = "0.1.75"
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
 # expandable_segments lets the allocator return freed pages to the driver even

codai/admin/templates/settings.html

@@ -667,6 +667,32 @@
     </div>
   </div>
 </div>
+
+<!-- Credits & acknowledgements -->
+<div class="card">
+  <div class="card-title">Credits &amp; acknowledgements</div>
+  <p class="form-hint" style="margin-bottom:.7rem">CoderAI stands on the shoulders of remarkable open-source work. Heartfelt thanks to the developers whose engines and research make these capabilities possible.</p>
+
+  <div style="display:flex;flex-direction:column;gap:.6rem">
+    <div style="border-left:3px solid var(--accent,#5b8cff);padding:.15rem 0 .15rem .7rem">
+      <div style="font-weight:600">colibri <span class="muted" style="font-weight:400">— GLM-5.2 on consumer hardware</span></div>
+      <div class="form-hint">A brilliant pure-C MoE engine by <b>JustVugg</b> that streams a 744B-parameter model across VRAM/RAM/disk to run it on a single GPU. <a href="https://github.com/JustVugg/colibri" target="_blank" rel="noopener">github.com/JustVugg/colibri</a></div>
+    </div>
+    <div style="border-left:3px solid var(--accent,#5b8cff);padding:.15rem 0 .15rem .7rem">
+      <div style="font-weight:600">ds4 / DwarfStar <span class="muted" style="font-weight:400">— DeepSeek-V4 native engine</span></div>
+      <div class="form-hint">A superb from-scratch DeepSeek-V4 inference engine by <b>Salvatore Sanfilippo (antirez)</b>. <a href="https://github.com/antirez/ds4" target="_blank" rel="noopener">github.com/antirez/ds4</a></div>
+    </div>
+    <div style="border-left:3px solid var(--border,#444);padding:.15rem 0 .15rem .7rem">
+      <div style="font-weight:600">llama.cpp &amp; whisper.cpp <span class="muted" style="font-weight:400">— GGUF LLM inference &amp; Whisper STT</span></div>
+      <div class="form-hint">The foundational C/C++ inference engines by <b>Georgi Gerganov</b> and contributors. <a href="https://github.com/ggml-org/llama.cpp" target="_blank" rel="noopener">llama.cpp</a> · <a href="https://github.com/ggml-org/whisper.cpp" target="_blank" rel="noopener">whisper.cpp</a></div>
+    </div>
+    <div style="border-left:3px solid var(--border,#444);padding:.15rem 0 .15rem .7rem">
+      <div style="font-weight:600">Visual place recognition &amp; geolocation research</div>
+      <div class="form-hint"><b>EigenPlaces</b> (Gabriele Berton et&nbsp;al.) · <b>DINOv2-SALAD</b> (Sergio Izquierdo, Javier Civera) · <b>GeoCLIP</b> (Vicente Vivanco et&nbsp;al.) · <b>DINOv2</b> (Meta AI) — the models that power visual matching &amp; geolocation.</div>
+    </div>
+  </div>
+  <p class="form-hint" style="margin-top:.7rem;font-size:11px">…and the wider open-source ecosystem (PyTorch, Hugging Face Transformers/Diffusers, and many more) that CoderAI builds on.</p>
+</div>
 {% endblock %}
 
 {% block scripts %}

codai/api/app.py

@@ -479,6 +479,15 @@ async def internal_thermal_pause(request: Request):
         _therm.set_external_pause(True, reason=str(data.get("reason") or "thermal"))
     except Exception as e:
         return {"ok": False, "error": str(e)}
+    # Gracefully idle any in-flight colibri generation via the PAUSE mux frame — it
+    # decodes autonomously once submitted, so the between-tokens external-pause flag
+    # alone can't stop it; this lets the cooperative pause actually cool the box
+    # without the front escalating to SIGSTOP.
+    try:
+        from codai.api import colibri_worker
+        colibri_worker.pause_all()
+    except Exception:
+        pass
     return {"ok": True, "paused": True}
 
 
@@ -490,6 +499,11 @@ async def internal_thermal_resume(request: Request):
         _therm.set_external_pause(False)
     except Exception as e:
         return {"ok": False, "error": str(e)}
+    try:
+        from codai.api import colibri_worker
+        colibri_worker.resume_all()
+    except Exception:
+        pass
     return {"ok": True, "paused": False}
 
 

codai/api/colibri_worker.py

@@ -435,6 +435,30 @@ class MuxEngine:
         finally:
             self._slots.put(slot)
 
+    def pause(self):
+        """Gracefully idle the decode loop (thermal throttle) via the PAUSE mux frame
+        — colibri stops issuing forward passes but keeps all KV/slot state, so no
+        tokens are lost and the process stays alive/responsive (unlike SIGSTOP).
+        Best-effort; needs the coderai PAUSE/RESUME colibri patch (packaging/
+        patch-colibri.py) — an unpatched engine simply ignores the unknown line."""
+        try:
+            if self.process.poll() is None:
+                with self.write_lock:
+                    self.process.stdin.write(b"PAUSE\n")
+                    self.process.stdin.flush()
+        except Exception:
+            pass
+
+    def resume(self):
+        """Resume decoding after :meth:`pause` (RESUME mux frame)."""
+        try:
+            if self.process.poll() is None:
+                with self.write_lock:
+                    self.process.stdin.write(b"RESUME\n")
+                    self.process.stdin.flush()
+        except Exception:
+            pass
+
     def close(self):
         with self.pending_lock:
             if self.closed:
@@ -606,5 +630,39 @@ def stop_all() -> None:
         stop_service(mid)
 
 
+def pause_all() -> int:
+    """Gracefully idle every live colibri engine (thermal throttle). Returns the count
+    signalled. Used by the engine's cooperative /internal/thermal-pause handler so a
+    hot box can throttle in-flight colibri generation without SIGSTOP."""
+    with _lock:
+        engines = list(_services.values())
+    n = 0
+    for eng in engines:
+        try:
+            eng.pause()
+            n += 1
+        except Exception:
+            pass
+    if n:
+        print(f"[colibri] thermal pause → idled {n} engine(s)", flush=True)
+    return n
+
+
+def resume_all() -> int:
+    """Resume every live colibri engine after :func:`pause_all` (thermal cooled)."""
+    with _lock:
+        engines = list(_services.values())
+    n = 0
+    for eng in engines:
+        try:
+            eng.resume()
+            n += 1
+        except Exception:
+            pass
+    if n:
+        print(f"[colibri] thermal resume → woke {n} engine(s)", flush=True)
+    return n
+
+
 import atexit as _atexit
 _atexit.register(stop_all)

codai/frontproxy/engine_supervisor.py

@@ -643,10 +643,142 @@ class EngineSupervisor:
         return [c for c in all_cards
                 if c.get("vendor") in sels or (c.get("uuid") and c["uuid"] in sels)]
 
+    # An engine's process group using at least this many CPU cores is a real source
+    # of CPU heat; below it, pausing the engine can't meaningfully cool the CPU.
+    _CPU_RELEVANT_CORES = 1.5
+    _CPU_HIST_LEN = 6          # rolling window (~poll_seconds × 6) for CPU relevance
+
+    @staticmethod
+    def _scan_pgroup_cpu() -> dict:
+        """One pass over /proc → {pgid: total (utime+stime) ticks}. Used to attribute
+        CPU load to each engine's process group (its own PID + children like
+        colibri/ds4/whisper-server, which inherit the engine's setsid pgid)."""
+        out: dict = {}
+        try:
+            for name in os.listdir("/proc"):
+                if not name.isdigit():
+                    continue
+                try:
+                    with open("/proc/" + name + "/stat", "rb") as f:
+                        data = f.read()
+                    # comm (field 2) is parenthesised and may contain spaces/parens;
+                    # fields after the last ')' are stable and space-separated.
+                    fields = data[data.rindex(b")") + 2:].split()
+                    pgid = int(fields[2])                      # field 5: pgrp
+                    out[pgid] = out.get(pgid, 0) + int(fields[11]) + int(fields[12])
+                except Exception:
+                    continue
+        except Exception:
+            pass
+        return out
+
+    def _engine_cpu_cores(self, engine, pg_ticks: dict, now: float):
+        """CPU cores this engine's process group used since the last poll, or None
+        (first sample / unmeasurable)."""
+        proc = getattr(engine, "proc", None)
+        pid = getattr(proc, "pid", None) if proc is not None else None
+        if not pid:
+            return None
+        try:
+            pgid = os.getpgid(pid)
+        except Exception:
+            return None
+        ticks = pg_ticks.get(pgid)
+        if ticks is None:
+            return None
+        cache = getattr(self, "_cpu_sample", None)
+        if cache is None:
+            cache = self._cpu_sample = {}
+        prev = cache.get(engine.id)
+        cache[engine.id] = (ticks, now)
+        if not prev:
+            return None
+        dt = now - prev[1]
+        if dt <= 0:
+            return None
+        try:
+            clk = os.sysconf("SC_CLK_TCK")
+        except Exception:
+            clk = 100.0
+        return max(0.0, (ticks - prev[0]) / clk / dt)
+
+    def _engine_cpu_relevant(self, engine, pg_ticks: dict, now: float) -> bool:
+        """True when this engine is a meaningful CPU-heat source, so the shared-CPU
+        thermal term applies to it. Uses a rolling MAX so an engine that just got
+        SIGSTOP-ed/idled (e.g. colibri between throttle cycles) still counts as
+        CPU-heavy and keeps its normal hysteresis, while a persistently GPU-bound
+        engine (Vulkan embeddings, ~0 CPU) is exempt and can't be stranded by another
+        engine's CPU heat. Fails safe to True when it can't be measured."""
+        cores = self._engine_cpu_cores(engine, pg_ticks, now)
+        hist = getattr(self, "_cpu_hist", None)
+        if hist is None:
+            hist = self._cpu_hist = {}
+        lst = hist.setdefault(engine.id, [])
+        if cores is not None:
+            lst.append(cores)
+            del lst[:-self._CPU_HIST_LEN]
+        if not lst:
+            return True
+        return max(lst) >= self._CPU_RELEVANT_CORES
+
     def _thermal_signal(self, engine, sig) -> None:
-        """Send a signal to the engine's whole process group (it runs in its own
-        session via setsid), so a SIGSTOP/SIGCONT freezes/resumes the engine and any
-        children (whisper-server, ds4) in one shot."""
+        """Freeze/resume this engine's heavy native compute WITHOUT freezing its
+        Python HTTP server, so it can still ack pause/resume and report health.
+
+        The old behaviour SIGSTOP-ed the whole process group (``os.killpg``), which
+        froze the engine's own process too — for a subprocess backend (colibri/ds4/
+        whisper-server) that left the front unable to talk to it, causing the resume
+        message to fail and a SIGSTOP/SIGCONT thrash. Instead, signal the engine's
+        native compute CHILDREN (everything in the group except the engine's own PID);
+        that stops the CPU/GPU heat source while the parent stays responsive. When
+        there are no such children (an in-process torch engine), fall back to killpg
+        so those still stop."""
+        proc = getattr(engine, "proc", None)
+        if proc is None:
+            return
+        try:
+            pgid = os.getpgid(proc.pid)
+        except Exception:
+            try:
+                proc.send_signal(sig)
+            except Exception:
+                pass
+            return
+        # Children = group members other than the engine's own process.
+        children = []
+        try:
+            for name in os.listdir("/proc"):
+                if not name.isdigit():
+                    continue
+                cpid = int(name)
+                if cpid == proc.pid:
+                    continue
+                try:
+                    if os.getpgid(cpid) == pgid:
+                        children.append(cpid)
+                except Exception:
+                    continue
+        except Exception:
+            children = []
+        if children:
+            for cpid in children:
+                try:
+                    os.kill(cpid, sig)
+                except Exception:
+                    pass
+            return
+        # No child compute process (in-process engine): stop the whole group.
+        try:
+            os.killpg(pgid, sig)
+        except Exception:
+            try:
+                proc.send_signal(sig)
+            except Exception:
+                pass
+
+    def _thermal_signal_group(self, engine, sig) -> None:
+        """Signal the engine's WHOLE process group (legacy behaviour) — used by
+        shutdown/restart where we do want to reach every child in one shot."""
         proc = engine.proc
         if proc is None:
             return
@@ -768,6 +900,13 @@ class EngineSupervisor:
                     cpu_warm = (cpu_t is not None and cpu_t > settings.cpu_resume)
                     escalate_n = int(getattr(self.config.thermal,
                                              "stop_escalate_checks", 3) or 3)
+                    # CPU is a SHARED resource, but only the engines actually loading it
+                    # can be cooled by pausing them. Sample per-engine CPU once per poll
+                    # so the shared-CPU thermal term applies only to real CPU-heat
+                    # sources — a GPU-bound engine (embeddings) is never paused/stranded
+                    # by another engine's (e.g. colibri's) CPU heat.
+                    pg_ticks = self._scan_pgroup_cpu() if cpu_t is not None else {}
+                    _now = time.monotonic()
                     for engine in self.registry.all():
                         if getattr(engine, "role", "engine") == "system":
                             continue   # the cache/downloads worker isn't on the GPU
@@ -802,11 +941,19 @@ class EngineSupervisor:
                             if t > resume:
                                 gpu_warm = True
                         engine.therm_temp = max(temps) if temps else None
-                        # Pause when this engine's GPU is over high OR the (global)
-                        # CPU is over high. Resume only when BOTH are back to resume.
-                        want_pause = (settings.gpu_enabled and gpu_hot) or cpu_hot
+                        # Apply the (global) CPU term only to engines that actually heat
+                        # the CPU — pausing a GPU-bound engine can't cool it, and the
+                        # hysteresis would strand it in the resume<CPU<high dead-band a
+                        # CPU-heavy engine keeps the shared CPU parked in.
+                        cpu_rel = (self._engine_cpu_relevant(engine, pg_ticks, _now)
+                                   if cpu_t is not None else False)
+                        e_cpu_hot = cpu_hot and cpu_rel
+                        e_cpu_warm = cpu_warm and cpu_rel
+                        # Pause when this engine's GPU is over high OR (it's a CPU-heat
+                        # source AND) the CPU is over high. Resume only when BOTH clear.
+                        want_pause = (settings.gpu_enabled and gpu_hot) or e_cpu_hot
                         want_resume_ok = (not (settings.gpu_enabled and gpu_warm)
-                                          and not cpu_warm)
+                                          and not e_cpu_warm)
                         self._thermal_apply(client, engine, settings, want_pause,
                                             want_resume_ok, engine.therm_temp, cpu_t,
                                             escalate_n)

docs/deepseek-ds4.md

@@ -72,3 +72,10 @@ coderai shallow-clones the repo at first use to obtain `download_model.sh`.
   `ModelBackend` interface.
 - `codai/models/manager.py` — `ds4_should_handle()` routes matching models to
   `Ds4Backend`; `is_allowed_model()` accepts the ds4 model id.
+
+## Credits
+
+DeepSeek-V4 support is made possible entirely by **[ds4 / DwarfStar](https://github.com/antirez/ds4)**,
+a from-scratch native inference engine by **Salvatore Sanfilippo ([antirez](https://github.com/antirez))**.
+CoderAI merely owns its lifecycle and proxies to it — all of the hard, brilliant
+inference work is theirs. Huge thanks.

docs/glm-colibri.md

+# GLM-5.2 via colibri
+
+CoderAI can serve **GLM-5.2** through **[colibri](https://github.com/JustVugg/colibri)**
+by **JustVugg** — a brilliant, pure-C Mixture-of-Experts inference engine that treats
+VRAM, RAM and disk as one memory hierarchy and streams expert weights on demand, so a
+**744B-parameter** model runs on a single consumer GPU.
+
+Unlike ds4 (which ships its own HTTP server), colibri's Python side is only a thin
+gateway around the C engine. So coderai drives the **C engine binary directly** over
+its stdin/stdout *mux* wire protocol (`docs/serve_protocol.md` in the colibri repo) —
+we own the build, the process, the GLM-5.2 chat template and the protocol client; no
+colibri Python runs at request time.
+
+> **Hardware reality:** GLM-5.2 is huge (~429 GB int4 container). colibri runs the
+> **dense layers on CPU** and streams **routed experts** (a small hot tier pinned in
+> VRAM, the rest from NVMe). It is **CPU- and streaming-bound**, not GPU-bound — the
+> GPU is a small accelerator. Keep the model on fast local storage (NVMe/ext4).
+
+## The model
+
+The model is a **directory** (int4 g64 shards + config + tokenizer + int8 MTP head),
+not a single file — the gs64 build with the int8 MTP head from
+[`mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp`](https://huggingface.co/mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp).
+Download it from the Models page like any other HF repo (whole-repo snapshot → a
+directory); coderai registers it as a `colibri`-backed model.
+
+## Pre-compiled, not built at request time
+
+The engine is a **pre-compiled** C binary (like ds4), never compiled inside the CUDA
+*runtime* container (which has no `nvcc`). Build it on a host with the CUDA toolkit —
+`build.sh --colibri` — with a **portable** GPU arch (SASS for Ampere→Blackwell). The
+binary links `libcudart.so.13`, resolved in-container from `/opt/coderai/local-libs`
+(the CUDA-13 runtime coderai already ships for PyTorch), so it adds **no** portability
+constraint beyond what the existing stack requires.
+
+## Tuning (env, via the per-model / global `extra_env`)
+
+- **`CTX`** — context window. Sized from the model's `n_ctx`. The DSA KV reserve is
+  **host RAM** (~330 KB/token), so max context is RAM-bound: ~64k is comfortable on a
+  54 GB box; the model itself supports up to 1,048,576 positions given enough RAM.
+- **`CUDA_DENSE=1`** — run the dense layers on the GPU (uses ~11 GB VRAM) instead of
+  the CPU. The big lever for offloading CPU heat and actually using the 3090.
+- **`CUDA_EXPERT_GB`** — VRAM budget for the hot expert tier.
+- **`COLI_NO_OMP_TUNE=1`** / **`OMP_NUM_THREADS`** — tame CPU spin/heat when the CPU is
+  mostly waiting on the GPU.
+
+## Graceful thermal pause (coderai patch)
+
+colibri decodes autonomously once a request is in flight, so coderai adds a small,
+idempotent **`PAUSE`/`RESUME` serve-mux patch** (`packaging/patch-colibri.py`, applied
+by `build.sh`): the engine idles the decode loop *between tokens* — keeping all KV
+state — on `PAUSE`, and continues on `RESUME`. This lets the front's cooperative
+thermal throttle cool the box without `SIGSTOP`-freezing the process. The patch is a
+candidate to upstream.
+
+## Files
+
+- `codai/config.py` — `ColibriConfig`.
+- `codai/api/colibri_worker.py` — clone/build, the `MuxEngine` protocol client, and the
+  per-container engine registry.
+- `codai/backends/colibri.py` — `ColibriBackend` + the GLM-5.2 chat template renderer.
+- `codai/models/manager.py` — `colibri_should_handle()` routes matching models to
+  `ColibriBackend`.
+- `packaging/patch-colibri.py` — the `PAUSE`/`RESUME` serve-mux patch.
+
+## Credits
+
+GLM-5.2 support exists entirely thanks to **[colibri](https://github.com/JustVugg/colibri)**
+by **[JustVugg](https://github.com/JustVugg)** — an ingenious piece of systems
+engineering that makes a 744B model runnable on hardware it has no business running on.
+CoderAI only drives it; the brilliance is theirs. Deep thanks.

packaging/patch-colibri.py

+#!/usr/bin/env python3
+# CoderAI - colibri serve-mux PAUSE/RESUME patch
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net> — GPLv3 (see the main LICENSE).
+"""Idempotently patch colibri's serve-mux loop (``c/colibri.c``) to add PAUSE/RESUME
+control frames on stdin.
+
+Rationale: colibri decodes autonomously once a request is in flight, so the only ways
+coderai could throttle it for thermal protection were "let it finish" (CPU/GPU stays
+hot) or SIGSTOP (freezes the process, and via killpg the parent engine too). A PAUSE
+frame lets colibri idle the decode loop *between tokens* — keeping all KV/slot state —
+and RESUME continues exactly where it left off, with the process staying alive and
+responsive. This is what coderai's cooperative thermal pause drives instead of SIGSTOP.
+
+Wire protocol additions (line-oriented, alongside SUBMIT/CANCEL):
+    PAUSE\n   -> engine stops issuing forward passes, replies  PAUSED\n
+    RESUME\n  -> engine resumes decoding,                      RESUMED\n
+
+Safe to run repeatedly (each edit is guarded / self-idempotent).
+"""
+import sys
+
+path = sys.argv[1] if len(sys.argv) > 1 else "c/colibri.c"
+src = open(path, encoding="utf-8", errors="surrogateescape").read()
+orig = src
+applied = []
+
+# 1) global pause flag, next to the existing soft-interrupt flag
+if "g_paused" not in src:
+    src = src.replace(
+        "static volatile sig_atomic_t g_intr=0;",
+        "static volatile sig_atomic_t g_intr=0;\n"
+        "static volatile sig_atomic_t g_paused=0;   /* PAUSE/RESUME mux control (coderai thermal throttle) */",
+        1)
+    applied.append("g_paused flag")
+
+# 2) handle PAUSE / RESUME command lines in mux_submit (line is already NUL-terminated,
+#    trailing newline stripped) — right after the CANCEL handler's NOT_FOUND return.
+_anchor = ('        printf("ERROR %llu NOT_FOUND\\n",id); fflush(stdout); free(line); return 0;\n'
+           '    }\n')
+if 'strcmp(line,"PAUSE")' not in src:
+    if _anchor not in src:
+        print("[patch-colibri] ERROR: CANCEL/NOT_FOUND anchor not found — upstream changed; "
+              "patch NOT applied", file=sys.stderr)
+        sys.exit(2)
+    src = src.replace(
+        _anchor,
+        _anchor +
+        '    if(!strcmp(line,"PAUSE")){  g_paused=1; printf("PAUSED\\n");  fflush(stdout); free(line); return 0; }\n'
+        '    if(!strcmp(line,"RESUME")){ g_paused=0; printf("RESUMED\\n"); fflush(stdout); free(line); return 0; }\n',
+        1)
+    applied.append("PAUSE/RESUME handler")
+
+# 3) while paused, block in select() awaiting the next command instead of busy-polling
+_old_ptv = "struct timeval tv={0,0}, *ptv=active?&tv:NULL;"
+if _old_ptv in src:
+    src = src.replace(_old_ptv,
+                      "struct timeval tv={0,0}, *ptv=(active && !g_paused)?&tv:NULL;", 1)
+    applied.append("select-block-when-paused")
+
+# 4) skip the forward pass while paused (state preserved; loop re-enters and blocks)
+_old_dec = ("        active=0; for(int i=0;i<nctx;i++) active+=req[i].active;\n"
+            "        if(!active){ if(eof) break; continue; }")
+if _old_dec in src:
+    src = src.replace(
+        _old_dec,
+        "        active=0; for(int i=0;i<nctx;i++) active+=req[i].active;\n"
+        "        if(g_paused) continue;   /* paused: no forward pass; loop blocks in select() awaiting RESUME */\n"
+        "        if(!active){ if(eof) break; continue; }", 1)
+    applied.append("skip-decode-when-paused")
+
+if src != orig:
+    open(path, "w", encoding="utf-8", errors="surrogateescape").write(src)
+    print("[patch-colibri] applied to %s: %s" % (path, ", ".join(applied)))
+else:
+    print("[patch-colibri] already patched (no change): %s" % path)