• Stefy Lanza (nextime / spora )'s avatar
    colibri: integrate GLM-5.2 native C engine (driven directly, no colibri Python) · 50f54eb3
    Stefy Lanza (nextime / spora ) authored
    Add JustVugg/colibri as a managed engine, mirroring the ds4 integration but
    driving the pure-C `colibri` binary DIRECTLY over its stdin/stdout mux protocol
    (docs/serve_protocol.md) instead of proxying to a Python gateway — coderai
    reproduces openai_server.py's engine client and GLM-5.2 chat template itself.
    
    - config: ColibriConfig (disabled by default; model is a directory container,
      not a GGUF — routes by model_id/alias/name, no arch sniff)
    - codai/api/colibri_worker.py: clone+build (make colibri CUDA=1), MuxEngine
      protocol client (READY handshake, SUBMIT/DATA/DONE, KV-slot pool, CANCEL,
      stderr log pump), per-container engine registry
    - codai/backends/colibri.py: ColibriBackend + render_chat (byte-exact port of
      colibri's GLM-5.2 template; the engine tokenizes what we send)
    - front routing: `colibri` capability on nvidia/cuda/auto; router/assignment/
      app/engine_supervisor thread config.colibri
    - manager: colibri_should_handle, backend selection, /v1/models surfacing,
      exclusive-VRAM eviction (wants the whole GPU like ds4)
    - admin: config get/set + per-model overrides; settings.html card + models.html row
    - packaging: build.sh --colibri, OCI bundle (repo+binary, no 372GB model),
      entrypoint seed, CODERAI_COLIBRI_DIR, smoke test
    
    Ships OFF: no routing changes until colibri.enabled + colibri.model_path are set.
    Verified offline: config round-trip, routing predicate, render_chat byte-match
    vs colibri's own, MuxEngine end-to-end against a fake engine, CPU engine builds.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
    50f54eb3
build_oci_image.sh 23 KB