-
Stefy Lanza (nextime / spora ) authored
Add JustVugg/colibri as a managed engine, mirroring the ds4 integration but driving the pure-C `colibri` binary DIRECTLY over its stdin/stdout mux protocol (docs/serve_protocol.md) instead of proxying to a Python gateway — coderai reproduces openai_server.py's engine client and GLM-5.2 chat template itself. - config: ColibriConfig (disabled by default; model is a directory container, not a GGUF — routes by model_id/alias/name, no arch sniff) - codai/api/colibri_worker.py: clone+build (make colibri CUDA=1), MuxEngine protocol client (READY handshake, SUBMIT/DATA/DONE, KV-slot pool, CANCEL, stderr log pump), per-container engine registry - codai/backends/colibri.py: ColibriBackend + render_chat (byte-exact port of colibri's GLM-5.2 template; the engine tokenizes what we send) - front routing: `colibri` capability on nvidia/cuda/auto; router/assignment/ app/engine_supervisor thread config.colibri - manager: colibri_should_handle, backend selection, /v1/models surfacing, exclusive-VRAM eviction (wants the whole GPU like ds4) - admin: config get/set + per-model overrides; settings.html card + models.html row - packaging: build.sh --colibri, OCI bundle (repo+binary, no 372GB model), entrypoint seed, CODERAI_COLIBRI_DIR, smoke test Ships OFF: no routing changes until colibri.enabled + colibri.model_path are set. Verified offline: config round-trip, routing predicate, render_chat byte-match vs colibri's own, MuxEngine end-to-end against a fake engine, CPU engine builds. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
50f54eb3