Add dedicated OCR subsystem + finalize multi-engine backends

OCR (v0.1.85): new codai/ocr/ subsystem — dedicated OCR engines (NOT a VLM):
- docTR (in-process, GPU) + PaddleOCR & Surya (isolated-venv subprocess workers
  via codai/ocr/workers/ocr_worker.py to avoid opencv-contrib/pillow<11 clashes;
  paddle-gpu bundles its own CUDA so a cu12x wheel runs on newer-CUDA hosts).
- /v1/ocr, /v1/ocr/batch, /v1/ocr/schemas CRUD; `ocr` pipeline step.
- Data-driven, user-extensible extraction schemas (named files + inline + auto),
  built-in seeds (italian_sentenza/generic_document/invoice); optional JSON Schema
  validation. Structured extraction via an existing coderai text model.
- Configurable stamp/signature detection (off|layout|detector|both); bundled
  signature YOLO default; layout markers EN+IT (timbro/sigillo/firma).
- Admin OCR card: engines/instances/detect/extraction + schema picker/raw-JSON/
  field-builder + isolated-venv build trigger (background build + status).
- Isolated venvs resolve baked(/opt) > /cache mount (persistent) > ~/.coderai.
- Deps: pypdfium2 in base requirements; requirements-ocr(.txt/-paddle.txt);
  requirements-surya.txt; build.sh --ocr.

Also lands the multi-engine backend work: colibri multi-family (GLM/DeepSeek/Kimi),
kimi-k3-in-c (k3), ktransformers (kt) — per-model selectable via resolve_engine_backend;
plus a deferred vLLM backend design note (docs/vllm.md).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW

README.md

@@ -699,13 +699,23 @@ thanks to the developers of the native inference engines that let it punch far a
 its hardware — running models that otherwise simply couldn't run on a single GPU:
 
 - **[colibri](https://github.com/JustVugg/colibri)** by **JustVugg** — a brilliant
-  pure-C MoE engine that streams **GLM-5.2 (744B params)** across VRAM/RAM/disk to run
-  it on a single consumer GPU.
+  pure-C multi-family MoE engine that streams **GLM-5.2, DeepSeek-V4 and Kimi-K3**
+  across VRAM/RAM/disk to run them on a single consumer GPU (or CPU).
 - **[ds4 / DwarfStar](https://github.com/antirez/ds4)** by **Salvatore Sanfilippo
   (antirez)** — a superb from-scratch **DeepSeek-V4** inference engine.
+- **[kimi-k3-in-c](https://github.com/FareedKhan-dev/kimi-k3-in-c)** by
+  **Fareed Khan** — a portable-C engine that runs **Kimi-K3 (2.78T params)** on CPU by
+  streaming the dense trunk + routed experts from disk in as little as ~8 GB RAM.
+- **[ktransformers](https://github.com/kvcache-ai/ktransformers)** by **KVCache.AI** —
+  a CPU+GPU heterogeneous engine (served via SGLang) for large MoE models
+  (DeepSeek / Kimi / Qwen / GLM / MiniMax).
 - **[llama.cpp](https://github.com/ggml-org/llama.cpp)** &
   **[whisper.cpp](https://github.com/ggml-org/whisper.cpp)** by **Georgi Gerganov**
   and contributors — the foundational GGUF LLM inference and Whisper STT engines.
+- **[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)** (Baidu),
+  **[docTR](https://github.com/mindee/doctr)** (Mindee) and
+  **[Surya](https://github.com/VikParuchuri/surya)** — the dedicated OCR engines behind
+  the `/v1/ocr` document-transcription subsystem (text + layout + structured extraction).
 
 And the libraries, models and research CoderAI builds on:
 
@@ -717,6 +727,7 @@ And the libraries, models and research CoderAI builds on:
 - [F5-TTS](https://github.com/SWivid/F5-TTS) — voice cloning
 - [Seed-VC](https://github.com/Plachta/Seed-VC) — singing voice conversion
 - [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) — image/video upscaling
+- [pypdfium2](https://github.com/pypdfium2-team/pypdfium2) — PDF rasterisation for OCR; [Ultralytics YOLO](https://github.com/ultralytics/ultralytics) — stamp/signature detection
 - [Wav2Lip](https://github.com/Rudrabha/Wav2Lip) — audio-driven lip sync
 - [SadTalker](https://github.com/OpenTalker/SadTalker) — talking head / lip sync generation
 - Visual place recognition & geolocation research — [EigenPlaces](https://github.com/gmberton/EigenPlaces) (Gabriele Berton et al.), [DINOv2-SALAD](https://github.com/serizba/salad) (Sergio Izquierdo, Javier Civera), [GeoCLIP](https://github.com/VicenteVivan/geo-clip) (Vicente Vivanco et al.), [DINOv2](https://github.com/facebookresearch/dinov2) (Meta AI)

build.sh

@@ -37,6 +37,8 @@ CUSTOM_VENV=""
 PACKAGE=false
 DS4=false
 COLIBRI=false
+K3=false
+OCR=false
 
 # Parse arguments
 i=1
@@ -58,6 +60,12 @@ for arg in "$@"; do
         --colibri)
             COLIBRI=true
             ;;
+        --k3)
+            K3=true
+            ;;
+        --ocr)
+            OCR=true
+            ;;
     esac
     i=$((i + 1))
 done
@@ -77,7 +85,9 @@ if [[ "$BACKEND" != "nvidia" && "$BACKEND" != "vulkan" && "$BACKEND" != "vulkan-
     echo "Options:"
     echo "  --flash     - Install Flash Attention 2 for faster inference (NVIDIA only)"
     echo "  --ds4       - Clone + build the ds4 (DeepSeek V4) native engine"
-    echo "  --colibri   - Clone + build the colibri (GLM-5.2) native engine"
+    echo "  --colibri   - Clone + build the colibri (GLM-5.2 / DeepSeek-V4 / Kimi-K3) engines"
+    echo "  --k3        - Clone + patch + build the kimi-k3-in-c (Kimi-K3) native engine"
+    echo "  --ocr       - Install the dedicated OCR engine stack (requirements-ocr.txt: PaddleOCR/docTR/YOLO)"
     exit 1
 fi
 
@@ -827,15 +837,81 @@ build_colibri() {
     ( cd "$COLIBRI_DIR/c" && make -s $MAKE_ARGS ) || {
         echo -e "${YELLOW}Warning: colibri build failed; it can still be built at runtime.${NC}"; return 0; }
     if [ -x "$COLIBRI_DIR/c/colibri" ]; then
-        echo -e "${GREEN}✓ colibri built at $COLIBRI_DIR/c/colibri${NC}"
+        echo -e "${GREEN}✓ colibri (GLM-5.2) built at $COLIBRI_DIR/c/colibri${NC}"
         echo -e "${YELLOW}Note: the GLM-5.2 int4 container (~372 GB dir) is NOT downloaded; point colibri.model_path at it.${NC}"
     fi
+
+    # colibri v1.5.0 ships sibling engines for other families, each its own binary +
+    # make target. Build them too (non-fatal) so a single install serves DeepSeek-V4 and
+    # Kimi-K3 as well as GLM-5.2. Both are CPU engines that stream experts from disk:
+    # DeepSeek-V4 is AVX2 (no CUDA path, distinct from the separate ds4 CUDA engine) and
+    # Kimi-K3 is CPU + an opt-in Vulkan tier (COLI_K3_VK=1 → make VK=1, needs
+    # glslc/shaderc). The multi-GB model containers are NOT downloaded here.
+    ( cd "$COLIBRI_DIR/c" && make -s deepseek-v4 ) \
+        && [ -x "$COLIBRI_DIR/c/deepseek_v4" ] \
+        && echo -e "${GREEN}✓ colibri deepseek_v4 (CPU/AVX2) built at $COLIBRI_DIR/c/deepseek_v4${NC}" \
+        || echo -e "${YELLOW}Warning: colibri deepseek_v4 build skipped/failed (built on demand at runtime).${NC}"
+
+    local K3_ARGS="kimi_k3"
+    if [ "${COLI_K3_VK:-0}" = "1" ]; then K3_ARGS="kimi_k3 VK=1"; fi
+    ( cd "$COLIBRI_DIR/c" && make -s $K3_ARGS ) \
+        && [ -x "$COLIBRI_DIR/c/kimi_k3" ] \
+        && echo -e "${GREEN}✓ colibri kimi_k3 built at $COLIBRI_DIR/c/kimi_k3${NC}" \
+        || echo -e "${YELLOW}Warning: colibri kimi_k3 build skipped/failed (built on demand at runtime).${NC}"
 }
 
 if [ "$COLIBRI" = true ]; then
     build_colibri
 fi
 
+# Optionally clone + patch + build kimi-k3-in-c (Kimi-K3 native engine). Opt-in via
+# --k3. coderai patches in a resident serve loop (packaging/patch-k3.py) that speaks the
+# colibri mux protocol, then drives it via MuxEngine. CPU-only (AVX2+FMA); the ~1.56 TB
+# checkpoint + ~109 GB packed trunk are NOT downloaded here.
+build_k3() {
+    local K3_DIR="${CODERAI_K3_DIR:-$HOME/.coderai/kimi-k3-in-c}"
+    echo -e "${YELLOW}Building kimi-k3-in-c (Kimi-K3 engine) → $K3_DIR ...${NC}"
+    if [ ! -e "$K3_DIR/Makefile" ]; then
+        mkdir -p "$(dirname "$K3_DIR")"
+        git clone --depth 1 https://github.com/FareedKhan-dev/kimi-k3-in-c "$K3_DIR" || {
+            echo -e "${YELLOW}Warning: could not clone kimi-k3-in-c; skipping.${NC}"; return 0; }
+    fi
+    # Apply coderai's resident serve-loop patch (idempotent) so the engine stays warm.
+    if [ -f "$(dirname "$0")/packaging/patch-k3.py" ] && [ -f "$K3_DIR/src/cli/k3_run.c" ]; then
+        python3 "$(dirname "$0")/packaging/patch-k3.py" "$K3_DIR/src/cli/k3_run.c" || \
+            echo -e "${YELLOW}Warning: k3 serve-loop patch failed (serve mode will not work).${NC}"
+    fi
+    ( cd "$K3_DIR" && make -j ) || {
+        echo -e "${YELLOW}Warning: kimi-k3-in-c build failed; it can still be built at runtime.${NC}"; return 0; }
+    if [ -x "$K3_DIR/bin/k3" ]; then
+        echo -e "${GREEN}✓ kimi-k3-in-c built at $K3_DIR/bin/k3${NC}"
+        echo -e "${YELLOW}Note: the Kimi-K3 checkpoint (~1.56 TB) + packed trunk are NOT downloaded; point k3.model_path/k3.trunk_dir at them.${NC}"
+    fi
+}
+
+if [ "$K3" = true ]; then
+    build_k3
+fi
+
+# Optionally install the dedicated OCR engine stack (PaddleOCR/docTR/YOLO). Opt-in via
+# --ocr. Kept out of the base requirements so the default install/image stay lean; the
+# /v1/ocr subsystem returns HTTP 503 for any engine whose dependency is absent.
+install_ocr() {
+    echo -e "${YELLOW}Installing OCR engine stack (requirements-ocr.txt)...${NC}"
+    if [ -f "$(dirname "$0")/requirements-ocr.txt" ]; then
+        pip install -r "$(dirname "$0")/requirements-ocr.txt" || {
+            echo -e "${YELLOW}Warning: some OCR deps failed (PaddlePaddle is CUDA-version-specific).${NC}"
+            echo -e "${YELLOW}         See requirements-ocr.txt for the correct paddlepaddle-gpu wheel.${NC}"
+        }
+    else
+        echo -e "${YELLOW}Warning: requirements-ocr.txt not found; skipping OCR stack.${NC}"
+    fi
+}
+
+if [ "$OCR" = true ]; then
+    install_ocr
+fi
+
 # Create .backend file to track which backend was used
 echo "$BACKEND" > .backend
 

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.1.83"
+__version__ = "0.1.85"
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
 # expandable_segments lets the allocator return freed pages to the driver even

codai/admin/routes.py

@@ -835,6 +835,80 @@ _DS4_DEFAULT_MODELS = [
 ]
 
 
+@router.get("/admin/api/ocr/schemas", summary="List OCR extraction schemas")
+async def api_ocr_schemas(username: str = Depends(require_admin)):
+    from codai.ocr.schemas import schema_store
+    return {"schemas": schema_store.list()}
+
+
+@router.get("/admin/api/ocr/schemas/{name}", summary="Get one OCR extraction schema")
+async def api_ocr_schema_get(name: str, username: str = Depends(require_admin)):
+    from codai.ocr.schemas import schema_store
+    from codai.ocr.base import OcrError
+    try:
+        return schema_store.get(name)
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+@router.post("/admin/api/ocr/schemas", summary="Create/update a user OCR schema")
+async def api_ocr_schema_save(request: Request, username: str = Depends(require_admin)):
+    from codai.ocr.schemas import schema_store
+    from codai.ocr.base import OcrError
+    body = await request.json()
+    name = (body.get("name") or "").strip()
+    if not name:
+        raise HTTPException(status_code=400, detail="schema 'name' is required")
+    data = {
+        "name": name,
+        "description": body.get("description", "") or "",
+        "language": body.get("language", "") or "",
+        "type": body.get("type", "template") or "template",
+        "prompt_hint": body.get("prompt_hint", "") or "",
+        "schema": body.get("schema") or {},
+    }
+    try:
+        return schema_store.save(name, data)
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+@router.delete("/admin/api/ocr/schemas/{name}", summary="Delete a user OCR schema")
+async def api_ocr_schema_delete(name: str, username: str = Depends(require_admin)):
+    from codai.ocr.schemas import schema_store
+    from codai.ocr.base import OcrError
+    try:
+        schema_store.delete(name)
+        return {"deleted": name}
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+@router.post("/admin/api/ocr/build-venv", summary="Build a Paddle/Surya isolated venv")
+async def api_ocr_build_venv(request: Request, username: str = Depends(require_admin)):
+    """Kick off a background build of an OCR engine's isolated venv (paddle|surya)."""
+    body = await request.json()
+    engine = (body.get("engine") or "").strip().lower()
+    if engine not in ("paddle", "surya"):
+        raise HTTPException(status_code=400, detail="engine must be 'paddle' or 'surya'")
+    if config_manager is None or config_manager.config is None:
+        raise HTTPException(status_code=503, detail="config unavailable")
+    from codai.ocr.venv_build import build_async
+    try:
+        return build_async(config_manager.config.ocr, engine)
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@router.get("/admin/api/ocr/build-venv/status", summary="OCR isolated-venv build status")
+async def api_ocr_build_venv_status(engine: str, username: str = Depends(require_admin)):
+    engine = (engine or "").strip().lower()
+    if engine not in ("paddle", "surya"):
+        raise HTTPException(status_code=400, detail="engine must be 'paddle' or 'surya'")
+    from codai.ocr.venv_build import status
+    return status(engine)
+
+
 @router.get("/admin/api/ds4/default-models", summary="List downloadable ds4 default models")
 async def api_ds4_default_models(username: str = Depends(require_admin)):
     """Catalog of official DeepSeek V4 GGUF variants ds4 can serve. The UI offers
@@ -3401,6 +3475,71 @@ def build_settings_dict(c, gpu_cards):
             "extra_env": c.colibri.extra_env,
             "auto_build": c.colibri.auto_build,
         },
+        "k3": {
+            "enabled": c.k3.enabled,
+            "repo_url": c.k3.repo_url,
+            "install_dir": c.k3.install_dir,
+            "model_path": c.k3.model_path,
+            "trunk_dir": c.k3.trunk_dir,
+            "tok_dir": c.k3.tok_dir,
+            "model_id": c.k3.model_id,
+            "preset": c.k3.preset,
+            "trunk_gb": c.k3.trunk_gb,
+            "cache_gb": c.k3.cache_gb,
+            "ctx": c.k3.ctx,
+            "extra_args": c.k3.extra_args,
+            "extra_env": c.k3.extra_env,
+            "auto_build": c.k3.auto_build,
+        },
+        "ktransformers": {
+            "enabled": c.ktransformers.enabled,
+            "repo_url": c.ktransformers.repo_url,
+            "install_dir": c.ktransformers.install_dir,
+            "model_path": c.ktransformers.model_path,
+            "kt_weight_path": c.ktransformers.kt_weight_path,
+            "model_id": c.ktransformers.model_id,
+            "host": c.ktransformers.host,
+            "port": c.ktransformers.port,
+            "ctx": c.ktransformers.ctx,
+            "extra_args": c.ktransformers.extra_args,
+            "extra_env": c.ktransformers.extra_env,
+            "auto_build": c.ktransformers.auto_build,
+        },
+        "ocr": {
+            "enabled": c.ocr.enabled,
+            "default_engine": c.ocr.default_engine,
+            "dpi": c.ocr.dpi,
+            "max_concurrency": c.ocr.max_concurrency,
+            "lang": c.ocr.lang,
+            "paddle_enabled": c.ocr.paddle_enabled,
+            "paddle_use_gpu": c.ocr.paddle_use_gpu,
+            "paddle_instances": c.ocr.paddle_instances,
+            "paddle_structure": c.ocr.paddle_structure,
+            "paddle_lang": c.ocr.paddle_lang,
+            "paddle_det_model_dir": c.ocr.paddle_det_model_dir,
+            "paddle_rec_model_dir": c.ocr.paddle_rec_model_dir,
+            "paddle_venv": c.ocr.paddle_venv,
+            "paddle_auto_build": c.ocr.paddle_auto_build,
+            "doctr_enabled": c.ocr.doctr_enabled,
+            "doctr_use_gpu": c.ocr.doctr_use_gpu,
+            "doctr_instances": c.ocr.doctr_instances,
+            "doctr_det_arch": c.ocr.doctr_det_arch,
+            "doctr_reco_arch": c.ocr.doctr_reco_arch,
+            "surya_enabled": c.ocr.surya_enabled,
+            "surya_accept_license": c.ocr.surya_accept_license,
+            "surya_instances": c.ocr.surya_instances,
+            "surya_langs": c.ocr.surya_langs,
+            "surya_venv": c.ocr.surya_venv,
+            "surya_auto_build": c.ocr.surya_auto_build,
+            "detect_mode": c.ocr.detect_mode,
+            "detect_model_path": c.ocr.detect_model_path,
+            "detect_conf": c.ocr.detect_conf,
+            "extract_enabled": c.ocr.extract_enabled,
+            "extract_model_id": c.ocr.extract_model_id,
+            "extract_max_tokens": c.ocr.extract_max_tokens,
+            "extract_schema": c.ocr.extract_schema,
+            "extract_validate": c.ocr.extract_validate,
+        },
         "compaction": {
             "enabled": c.compaction.enabled,
             "pct": c.compaction.pct,
@@ -3777,6 +3916,134 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
         if "auto_build" in d:
             c.colibri.auto_build = bool(d["auto_build"])
 
+    if "k3" in data:
+        d = data["k3"]
+        c.k3.enabled = bool(d.get("enabled", c.k3.enabled))
+        if "repo_url" in d:
+            c.k3.repo_url = (d.get("repo_url") or c.k3.repo_url or "").strip()
+        if "install_dir" in d:
+            c.k3.install_dir = (d.get("install_dir") or "").strip() or None
+        if "model_path" in d:
+            c.k3.model_path = (d.get("model_path") or "").strip()
+        if "trunk_dir" in d:
+            c.k3.trunk_dir = (d.get("trunk_dir") or "").strip()
+        if "tok_dir" in d:
+            c.k3.tok_dir = (d.get("tok_dir") or "").strip()
+        if "model_id" in d:
+            c.k3.model_id = (d.get("model_id") or c.k3.model_id or "kimi-k3").strip()
+        if "preset" in d:
+            c.k3.preset = (d.get("preset") or "").strip()
+        if "trunk_gb" in d:
+            try:
+                c.k3.trunk_gb = float(d.get("trunk_gb") or c.k3.trunk_gb)
+            except (TypeError, ValueError):
+                pass
+        if "cache_gb" in d:
+            try:
+                c.k3.cache_gb = float(d.get("cache_gb") or c.k3.cache_gb)
+            except (TypeError, ValueError):
+                pass
+        if "ctx" in d:
+            try:
+                c.k3.ctx = max(32, int(d.get("ctx") or c.k3.ctx))
+            except (TypeError, ValueError):
+                pass
+        if "extra_args" in d:
+            c.k3.extra_args = (d.get("extra_args") or "").strip()
+        if "extra_env" in d:
+            c.k3.extra_env = (d.get("extra_env") or "").strip()
+        if "auto_build" in d:
+            c.k3.auto_build = bool(d["auto_build"])
+
+    if "ktransformers" in data:
+        d = data["ktransformers"]
+        c.ktransformers.enabled = bool(d.get("enabled", c.ktransformers.enabled))
+        if "repo_url" in d:
+            c.ktransformers.repo_url = (d.get("repo_url") or c.ktransformers.repo_url or "").strip()
+        if "install_dir" in d:
+            c.ktransformers.install_dir = (d.get("install_dir") or "").strip() or None
+        if "model_path" in d:
+            c.ktransformers.model_path = (d.get("model_path") or "").strip()
+        if "kt_weight_path" in d:
+            c.ktransformers.kt_weight_path = (d.get("kt_weight_path") or "").strip()
+        if "model_id" in d:
+            c.ktransformers.model_id = (d.get("model_id") or c.ktransformers.model_id or "ktransformers").strip()
+        if "host" in d:
+            c.ktransformers.host = (d.get("host") or "127.0.0.1").strip()
+        if "port" in d:
+            try:
+                c.ktransformers.port = int(d.get("port") or 0)
+            except (TypeError, ValueError):
+                pass
+        if "ctx" in d:
+            try:
+                c.ktransformers.ctx = max(512, int(d.get("ctx") or c.ktransformers.ctx))
+            except (TypeError, ValueError):
+                pass
+        if "extra_args" in d:
+            c.ktransformers.extra_args = (d.get("extra_args") or "").strip()
+        if "extra_env" in d:
+            c.ktransformers.extra_env = (d.get("extra_env") or "").strip()
+        if "auto_build" in d:
+            c.ktransformers.auto_build = bool(d["auto_build"])
+
+    if "ocr" in data:
+        d = data["ocr"]
+        o = c.ocr
+
+        def _s(key, default=""):
+            return (d.get(key) or default).strip() if isinstance(d.get(key), str) else d.get(key, default)
+
+        def _i(key, cur, lo=None):
+            try:
+                v = int(d.get(key, cur))
+                return max(lo, v) if lo is not None else v
+            except (TypeError, ValueError):
+                return cur
+
+        if "enabled" in d: o.enabled = bool(d["enabled"])
+        if "default_engine" in d: o.default_engine = (d.get("default_engine") or "paddle").strip()
+        if "dpi" in d: o.dpi = _i("dpi", o.dpi, 36)
+        if "max_concurrency" in d: o.max_concurrency = _i("max_concurrency", o.max_concurrency, 1)
+        if "lang" in d: o.lang = (d.get("lang") or "it").strip()
+        # PaddleOCR
+        if "paddle_enabled" in d: o.paddle_enabled = bool(d["paddle_enabled"])
+        if "paddle_use_gpu" in d: o.paddle_use_gpu = bool(d["paddle_use_gpu"])
+        if "paddle_instances" in d: o.paddle_instances = _i("paddle_instances", o.paddle_instances, 1)
+        if "paddle_structure" in d: o.paddle_structure = bool(d["paddle_structure"])
+        if "paddle_lang" in d: o.paddle_lang = (d.get("paddle_lang") or "it").strip()
+        if "paddle_det_model_dir" in d: o.paddle_det_model_dir = (d.get("paddle_det_model_dir") or "").strip()
+        if "paddle_rec_model_dir" in d: o.paddle_rec_model_dir = (d.get("paddle_rec_model_dir") or "").strip()
+        if "paddle_venv" in d: o.paddle_venv = (d.get("paddle_venv") or "").strip()
+        if "paddle_auto_build" in d: o.paddle_auto_build = bool(d["paddle_auto_build"])
+        # docTR
+        if "doctr_enabled" in d: o.doctr_enabled = bool(d["doctr_enabled"])
+        if "doctr_use_gpu" in d: o.doctr_use_gpu = bool(d["doctr_use_gpu"])
+        if "doctr_instances" in d: o.doctr_instances = _i("doctr_instances", o.doctr_instances, 1)
+        if "doctr_det_arch" in d: o.doctr_det_arch = (d.get("doctr_det_arch") or "db_resnet50").strip()
+        if "doctr_reco_arch" in d: o.doctr_reco_arch = (d.get("doctr_reco_arch") or "crnn_vgg16_bn").strip()
+        # Surya
+        if "surya_enabled" in d: o.surya_enabled = bool(d["surya_enabled"])
+        if "surya_accept_license" in d: o.surya_accept_license = bool(d["surya_accept_license"])
+        if "surya_instances" in d: o.surya_instances = _i("surya_instances", o.surya_instances, 1)
+        if "surya_langs" in d: o.surya_langs = (d.get("surya_langs") or "it").strip()
+        if "surya_venv" in d: o.surya_venv = (d.get("surya_venv") or "").strip()
+        if "surya_auto_build" in d: o.surya_auto_build = bool(d["surya_auto_build"])
+        # detection
+        if "detect_mode" in d: o.detect_mode = (d.get("detect_mode") or "off").strip()
+        if "detect_model_path" in d: o.detect_model_path = (d.get("detect_model_path") or "").strip()
+        if "detect_conf" in d:
+            try:
+                o.detect_conf = float(d.get("detect_conf") or o.detect_conf)
+            except (TypeError, ValueError):
+                pass
+        # extraction
+        if "extract_enabled" in d: o.extract_enabled = bool(d["extract_enabled"])
+        if "extract_model_id" in d: o.extract_model_id = (d.get("extract_model_id") or "").strip()
+        if "extract_max_tokens" in d: o.extract_max_tokens = _i("extract_max_tokens", o.extract_max_tokens, 64)
+        if "extract_schema" in d: o.extract_schema = (d.get("extract_schema") or "").strip()
+        if "extract_validate" in d: o.extract_validate = bool(d["extract_validate"])
+
     if "compaction" in data:
         cp = data["compaction"] or {}
         if "enabled" in cp:

codai/admin/templates/models.html

@@ -551,7 +551,10 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
             <option value="vulkan">Vulkan</option>
             <option value="opencl">OpenCL</option>
             <option value="cpu">CPU only</option>
-            <option value="colibri">colibri (GLM-5.2 C engine)</option>
+            <option value="colibri">colibri (GLM-5.2 / DeepSeek-V4 / Kimi-K3 C engine)</option>
+            <option value="ds4">ds4 (DeepSeek-V4 CUDA engine)</option>
+            <option value="k3">k3 (Kimi-K3 / kimi-k3-in-c)</option>
+            <option value="kt">ktransformers (SGLang, CPU+GPU)</option>
           </select>
         </div>
         <div class="form-row" style="margin:0">

codai/admin/templates/settings.html

@@ -631,6 +631,275 @@
       <input type="text" id="s-colibri-extra-env" class="form-input" placeholder="COLI_MODEL_MIRROR=/nvme2/glm52_i4 DIRECT=1 PIPE=2">
       <span class="form-hint">Whitespace-separated <code>KEY=VALUE</code> pairs added to the engine's environment — colibri exposes its tuning only via env (<code>COLI_MODEL_MIRROR</code>, <code>COLI_DISK_WEIGHTS</code>, <code>COLI_NUMA</code>, <code>COLI_CUDA_PIPE</code>, <code>DIRECT</code>, <code>PIPE</code>, <code>PILOT</code>, <code>DRAFT</code>, …). Context window is set per-model via <b>n_ctx</b>.</span>
     </div>
+    <p class="form-hint" style="margin-top:.6rem">colibri v1.5.0 also serves <b>DeepSeek-V4</b> and <b>Kimi-K3</b> (CPU engines) from the same install — add a model whose <code>backend</code> is <code>colibri</code> pointing at that family's container; the engine auto-detects the family from the container's <code>config.json</code>.</p>
+  </div>
+</div>
+
+<!-- Kimi-K3 (kimi-k3-in-c) -->
+<div class="card">
+  <div class="card-title">Kimi-K3 (kimi-k3-in-c)</div>
+  <p class="form-hint" style="margin-bottom:.6rem">Run Kimi-K3 through FareedKhan-dev's portable-C <a href="https://github.com/FareedKhan-dev/kimi-k3-in-c" target="_blank" rel="noopener">kimi-k3-in-c</a> engine. coderai patches in a resident serve loop and drives it over the same mux protocol as colibri. <b>CPU-only</b> (AVX2+FMA) and needs ~1.7&nbsp;TB fast local storage (the ~1.56&nbsp;TB checkpoint + a ~109&nbsp;GB packed trunk from the repo's <code>scripts/pack-trunk.sh</code>). Nothing is downloaded automatically.</p>
+  <div class="form-group">
+    <label class="toggle-row" style="display:flex;align-items:center;gap:.5rem;cursor:pointer">
+      <input type="checkbox" id="s-k3-enabled" onchange="toggleK3Fields()">
+      <span style="font-size:13px;font-weight:500">Enable k3 (Kimi-K3)</span>
+    </label>
+  </div>
+  <div id="k3-fields" style="display:none">
+    <div class="form-group">
+      <label class="form-label">Model id / alias</label>
+      <input type="text" id="s-k3-model-id" class="form-input" placeholder="kimi-k3">
+      <span class="form-hint">When enabled, this id (and names containing <b>kimi-k3</b>, or any model whose config <code>backend</code> is <code>k3</code>) routes to the k3 engine. If colibri is also enabled, a bare Kimi name is ambiguous — set the model's <code>backend</code> to choose.</span>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Checkpoint path</label>
+      <input type="text" id="s-k3-model-path" class="form-input" placeholder="/nvme/Kimi-K3">
+      <span class="form-hint">The Kimi-K3 checkpoint <b>directory</b> (config.json + tokenizer + safetensors shards).</span>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Packed trunk dir</label>
+      <input type="text" id="s-k3-trunk-dir" class="form-input" placeholder="/nvme/Kimi-K3-trunk">
+      <span class="form-hint">The packed dense trunk (<code>scripts/pack-trunk.sh</code>). Blank = fully resident (needs ~114&nbsp;GB RAM).</span>
+    </div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap">
+      <div class="form-group" style="flex:1;min-width:150px">
+        <label class="form-label">Preset</label>
+        <select id="s-k3-preset" class="form-input">
+          <option value="">(explicit budgets)</option>
+          <option value="laptop">laptop</option>
+          <option value="desktop">desktop</option>
+          <option value="workstation">workstation</option>
+          <option value="server">server</option>
+          <option value="max">max</option>
+        </select>
+      </div>
+      <div class="form-group" style="flex:1;min-width:120px">
+        <label class="form-label">Trunk GB</label>
+        <input type="number" id="s-k3-trunk-gb" class="form-input" min="0" step="1" placeholder="16">
+      </div>
+      <div class="form-group" style="flex:1;min-width:120px">
+        <label class="form-label">Cache GB</label>
+        <input type="number" id="s-k3-cache-gb" class="form-input" min="0" step="1" placeholder="64">
+      </div>
+      <div class="form-group" style="flex:1;min-width:120px">
+        <label class="form-label">Context (K3_MAXT)</label>
+        <input type="number" id="s-k3-ctx" class="form-input" min="32" placeholder="4096">
+      </div>
+    </div>
+    <div class="form-group">
+      <label class="toggle-row" style="display:flex;align-items:center;gap:.5rem;cursor:pointer">
+        <input type="checkbox" id="s-k3-auto-build">
+        <span>Auto clone + patch + build the k3 engine binary if missing</span>
+      </label>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Extra engine env</label>
+      <input type="text" id="s-k3-extra-env" class="form-input" placeholder="K3_EXPERT_GB=8 K3_BITS=4 K3_DIRECT=1">
+      <span class="form-hint">Whitespace-separated <code>KEY=VALUE</code> pairs (<code>K3_EXPERT_GB</code>, <code>K3_BITS</code>, <code>K3_DIRECT</code>, <code>K3_PIPE</code>, …).</span>
+    </div>
+  </div>
+</div>
+
+<!-- ktransformers (SGLang) -->
+<div class="card">
+  <div class="card-title">ktransformers (SGLang)</div>
+  <p class="form-hint" style="margin-bottom:.6rem">Run many large MoE families (DeepSeek, Kimi, Qwen, GLM, MiniMax) through <a href="https://github.com/kvcache-ai/ktransformers" target="_blank" rel="noopener">ktransformers</a>' CPU+GPU heterogeneous engine, served over SGLang's OpenAI API. coderai launches <code>python -m sglang.launch_server</code> as a managed subprocess and proxies to it. Requires <b>SGLang + kt-kernel</b> installed out of band (best on AMX/AVX-512 CPUs). Selected per model via an explicit <code>backend: kt</code> pin or the model id below — never by a broad name marker.</p>
+  <div class="form-group">
+    <label class="toggle-row" style="display:flex;align-items:center;gap:.5rem;cursor:pointer">
+      <input type="checkbox" id="s-kt-enabled" onchange="toggleKtFields()">
+      <span style="font-size:13px;font-weight:500">Enable ktransformers</span>
+    </label>
+  </div>
+  <div id="kt-fields" style="display:none">
+    <div class="form-group">
+      <label class="form-label">Model id / alias</label>
+      <input type="text" id="s-kt-model-id" class="form-input" placeholder="ktransformers">
+      <span class="form-hint">This id routes to ktransformers and is passed to SGLang as <code>--served-model-name</code>. Also matched by any model whose config <code>backend</code> is <code>kt</code>.</span>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Model path (HF dir)</label>
+      <input type="text" id="s-kt-model-path" class="form-input" placeholder="/nvme/DeepSeek-V4-Flash">
+    </div>
+    <div class="form-group">
+      <label class="form-label">KT weight path</label>
+      <input type="text" id="s-kt-weight-path" class="form-input" placeholder="/nvme/DeepSeek-V4-Flash-kt">
+      <span class="form-hint">The KT quantized weights directory (<code>--kt-weight-path</code>).</span>
+    </div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap">
+      <div class="form-group" style="flex:2;min-width:160px">
+        <label class="form-label">Host</label>
+        <input type="text" id="s-kt-host" class="form-input" placeholder="127.0.0.1">
+      </div>
+      <div class="form-group" style="flex:1;min-width:110px">
+        <label class="form-label">Port</label>
+        <input type="number" id="s-kt-port" class="form-input" min="0" placeholder="0 = auto">
+      </div>
+      <div class="form-group" style="flex:1;min-width:140px">
+        <label class="form-label">Context length</label>
+        <input type="number" id="s-kt-ctx" class="form-input" min="512" placeholder="32768">
+      </div>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Extra SGLang args</label>
+      <input type="text" id="s-kt-extra-args" class="form-input" placeholder="--tp-size 1 --mem-fraction-static 0.8">
+    </div>
+    <div class="form-group">
+      <label class="toggle-row" style="display:flex;align-items:center;gap:.5rem;cursor:pointer">
+        <input type="checkbox" id="s-kt-auto-build">
+        <span>Attempt <code>pip install sglang</code> if missing (heavy; kt-kernel still installed manually)</span>
+      </label>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Extra subprocess env</label>
+      <input type="text" id="s-kt-extra-env" class="form-input" placeholder="CUDA_VISIBLE_DEVICES=0">
+    </div>
+  </div>
+</div>
+
+<!-- OCR (dedicated document OCR engines) -->
+<div class="card">
+  <div class="card-title">OCR (document → text)</div>
+  <p class="form-hint" style="margin-bottom:.6rem">Dedicated OCR engines (detection + recognition), <b>not</b> a vision LLM — faithful text with bounding boxes and layout, batched on GPU. Serve <a href="https://github.com/PaddlePaddle/PaddleOCR" target="_blank" rel="noopener">PaddleOCR</a>, <a href="https://github.com/mindee/doctr" target="_blank" rel="noopener">docTR</a> or <a href="https://github.com/VikParuchuri/surya" target="_blank" rel="noopener">Surya</a>, selectable per request. Endpoints: <code>POST /v1/ocr</code>, <code>POST /v1/ocr/batch</code>. Heavy deps install out of band (<code>pip install -r requirements-ocr.txt</code> or <code>./build.sh … --ocr</code>); a missing engine returns HTTP 503.</p>
+  <div class="form-row">
+    <label style="display:flex;align-items:center;gap:.5rem;cursor:pointer">
+      <input type="checkbox" id="s-ocr-enabled" onchange="toggleOcrFields()">
+      <span style="font-size:13px;font-weight:500">Enable OCR subsystem</span>
+    </label>
+  </div>
+  <div id="ocr-fields" style="display:none">
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap">
+      <div class="form-group" style="flex:1;min-width:150px">
+        <label class="form-label">Default engine</label>
+        <select id="s-ocr-default-engine" class="form-input">
+          <option value="paddle">PaddleOCR</option>
+          <option value="doctr">docTR</option>
+          <option value="surya">Surya</option>
+        </select>
+      </div>
+      <div class="form-group" style="flex:1;min-width:110px">
+        <label class="form-label">PDF DPI</label>
+        <input type="number" id="s-ocr-dpi" class="form-input" min="72" placeholder="200">
+      </div>
+      <div class="form-group" style="flex:1;min-width:110px">
+        <label class="form-label">Max concurrency</label>
+        <input type="number" id="s-ocr-max-concurrency" class="form-input" min="1" placeholder="4">
+      </div>
+      <div class="form-group" style="flex:1;min-width:90px">
+        <label class="form-label">Language</label>
+        <input type="text" id="s-ocr-lang" class="form-input" placeholder="it">
+      </div>
+    </div>
+
+    <div class="card-title" style="font-size:13px;margin-top:.4rem">PaddleOCR</div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-paddle-enabled"><span>Enable</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-paddle-use-gpu"><span>Use GPU</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-paddle-structure"><span>PP-Structure (layout + tables)</span></label>
+      <div class="form-group" style="flex:1;min-width:110px"><label class="form-label">Instances</label><input type="number" id="s-ocr-paddle-instances" class="form-input" min="1" placeholder="2"></div>
+      <div class="form-group" style="flex:1;min-width:90px"><label class="form-label">Lang</label><input type="text" id="s-ocr-paddle-lang" class="form-input" placeholder="it"></div>
+    </div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap">
+      <div class="form-group" style="flex:1;min-width:180px"><label class="form-label">Det model dir <span class="muted">(opt)</span></label><input type="text" id="s-ocr-paddle-det-dir" class="form-input" placeholder="(default)"></div>
+      <div class="form-group" style="flex:1;min-width:180px"><label class="form-label">Rec model dir <span class="muted">(opt)</span></label><input type="text" id="s-ocr-paddle-rec-dir" class="form-input" placeholder="(default)"></div>
+    </div>
+    <p class="form-hint" style="margin:0 0 .3rem">Runs in an <b>isolated venv</b> (opencv-contrib clash + bundled CUDA — a cu12x wheel runs on GPU even on newer host CUDA).</p>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">Isolated venv dir</label><input type="text" id="s-ocr-paddle-venv" class="form-input" placeholder="/opt/coderai/paddle_venv"></div>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-paddle-auto-build"><span>Auto-build on first use</span></label>
+    </div>
+    <div style="display:flex;gap:.5rem;align-items:center;flex-wrap:wrap"><button type="button" class="btn btn-sm" onclick="ocrBuildVenv('paddle')">Build Paddle venv now</button><span id="s-ocr-paddle-build-status" class="form-hint"></span></div>
+
+    <div class="card-title" style="font-size:13px;margin-top:.4rem">docTR</div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-doctr-enabled"><span>Enable</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-doctr-use-gpu"><span>Use GPU</span></label>
+      <div class="form-group" style="flex:1;min-width:110px"><label class="form-label">Instances</label><input type="number" id="s-ocr-doctr-instances" class="form-input" min="1" placeholder="1"></div>
+      <div class="form-group" style="flex:1;min-width:150px"><label class="form-label">Det arch</label><input type="text" id="s-ocr-doctr-det" class="form-input" placeholder="db_resnet50"></div>
+      <div class="form-group" style="flex:1;min-width:150px"><label class="form-label">Reco arch</label><input type="text" id="s-ocr-doctr-reco" class="form-input" placeholder="crnn_vgg16_bn"></div>
+    </div>
+
+    <div class="card-title" style="font-size:13px;margin-top:.4rem">Surya</div>
+    <p class="form-hint" style="margin:0 0 .4rem">GPL-licensed (compatible with coderai's GPLv3). Enable + accept to load it. Some releases add a revenue-capped commercial clause — check the version you install.</p>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-surya-enabled"><span>Enable</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-surya-license"><span>Accept license</span></label>
+      <div class="form-group" style="flex:1;min-width:110px"><label class="form-label">Instances</label><input type="number" id="s-ocr-surya-instances" class="form-input" min="1" placeholder="1"></div>
+      <div class="form-group" style="flex:1;min-width:120px"><label class="form-label">Langs</label><input type="text" id="s-ocr-surya-langs" class="form-input" placeholder="it"></div>
+    </div>
+    <p class="form-hint" style="margin:0 0 .3rem">Runs in an <b>isolated venv</b> (Surya caps pillow&lt;11, incompatible with the main venv).</p>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">Isolated venv dir</label><input type="text" id="s-ocr-surya-venv" class="form-input" placeholder="/opt/coderai/surya_venv"></div>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-surya-auto-build"><span>Auto-build on first use</span></label>
+    </div>
+    <div style="display:flex;gap:.5rem;align-items:center;flex-wrap:wrap"><button type="button" class="btn btn-sm" onclick="ocrBuildVenv('surya')">Build Surya venv now</button><span id="s-ocr-surya-build-status" class="form-hint"></span></div>
+
+    <div class="card-title" style="font-size:13px;margin-top:.4rem">Stamp / signature detection</div>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap">
+      <div class="form-group" style="flex:1;min-width:150px">
+        <label class="form-label">Mode</label>
+        <select id="s-ocr-detect-mode" class="form-input">
+          <option value="off">Off</option>
+          <option value="layout">Layout regions (coarse, no extra model)</option>
+          <option value="detector">YOLO detector (accurate)</option>
+          <option value="both">Both</option>
+        </select>
+      </div>
+      <div class="form-group" style="flex:2;min-width:200px"><label class="form-label">YOLO weights (.pt)</label><input type="text" id="s-ocr-detect-model" class="form-input" placeholder="/models/signature-stamp-yolo.pt"></div>
+      <div class="form-group" style="flex:1;min-width:100px"><label class="form-label">Confidence</label><input type="number" id="s-ocr-detect-conf" class="form-input" min="0" max="1" step="0.05" placeholder="0.35"></div>
+    </div>
+
+    <div class="card-title" style="font-size:13px;margin-top:.4rem">Structured field extraction</div>
+    <p class="form-hint" style="margin:0 0 .4rem">Extract JSON fields from the OCR text using an existing coderai text model. Choose the default schema below; override per request with the <code>schema</code> field.</p>
+    <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-extract-enabled"><span>Enable extraction</span></label>
+      <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer"><input type="checkbox" id="s-ocr-extract-validate"><span>Validate (JSON Schema)</span></label>
+      <div class="form-group" style="flex:1;min-width:110px"><label class="form-label">Max tokens</label><input type="number" id="s-ocr-extract-max-tokens" class="form-input" min="64" placeholder="2048"></div>
+    </div>
+    <div class="form-group">
+      <label class="form-label">Extraction model id</label>
+      <input type="text" id="s-ocr-extract-model" class="form-input" placeholder="a coderai text model id/alias">
+    </div>
+    <div class="form-group">
+      <label class="form-label">Default schema</label>
+      <div style="display:flex;gap:.5rem;flex-wrap:wrap;align-items:center">
+        <select id="s-ocr-extract-schema" class="form-input" style="flex:1;min-width:180px" onchange="ocrSchemaPicked()"></select>
+        <button type="button" class="btn btn-sm" onclick="ocrLoadSchemas()">Refresh</button>
+      </div>
+      <span class="form-hint">The default schema for extraction. Pick a built-in or one of your saved schemas, or <b>auto</b> for generic key/value extraction.</span>
+    </div>
+
+    <!-- Schema editor: picker (above) + raw JSON + form builder, kept in sync -->
+    <div class="card mb-0" style="margin-top:.4rem;background:rgba(127,127,127,.06)">
+      <div class="card-title" style="font-size:13px">Schema editor</div>
+      <p class="form-hint" style="margin:0 0 .5rem">Create or edit a <b>user</b> schema (built-ins are read-only; saving a built-in's name creates an override). The field builder and raw JSON stay in sync.</p>
+      <div class="form-row" style="display:flex;gap:1rem;flex-wrap:wrap;align-items:center">
+        <div class="form-group" style="flex:2;min-width:160px"><label class="form-label">Name</label><input type="text" id="s-ocr-sch-name" class="form-input" placeholder="my_schema"></div>
+        <div class="form-group" style="flex:1;min-width:130px"><label class="form-label">Type</label>
+          <select id="s-ocr-sch-type" class="form-input"><option value="template">template</option><option value="jsonschema">jsonschema</option></select>
+        </div>
+        <div class="form-group" style="flex:1;min-width:90px"><label class="form-label">Lang</label><input type="text" id="s-ocr-sch-lang" class="form-input" placeholder=""></div>
+      </div>
+      <div class="form-group"><label class="form-label">Description</label><input type="text" id="s-ocr-sch-desc" class="form-input" placeholder=""></div>
+      <div class="form-group"><label class="form-label">Prompt hint <span class="muted">(opt)</span></label><input type="text" id="s-ocr-sch-hint" class="form-input" placeholder=""></div>
+
+      <label class="form-label" style="margin-top:.3rem">Fields <span class="muted">(builder — template type)</span></label>
+      <div id="s-ocr-sch-builder"></div>
+      <div style="margin:.35rem 0 .6rem"><button type="button" class="btn btn-sm" onclick="ocrBuilderAddRow()">+ Add field</button>
+        <button type="button" class="btn btn-sm" onclick="ocrBuilderToJson()">Builder → JSON ↓</button>
+        <button type="button" class="btn btn-sm" onclick="ocrJsonToBuilder()">JSON → Builder ↑</button></div>
+
+      <div class="form-group">
+        <label class="form-label">Raw schema JSON</label>
+        <textarea id="s-ocr-sch-json" class="form-input" rows="8" spellcheck="false" style="font-family:monospace;font-size:12px" placeholder='{"numero": "string", "data": "YYYY-MM-DD"}'></textarea>
+        <span class="form-hint">The value of the schema's <code>schema</code> field (a fill-in template or a JSON Schema object).</span>
+      </div>
+      <div style="display:flex;gap:.5rem;flex-wrap:wrap">
+        <button type="button" class="btn btn-sm" onclick="ocrSchemaLoadIntoEditor()">Load selected</button>
+        <button type="button" class="btn btn-sm btn-primary" onclick="ocrSchemaSave()">Save schema</button>
+        <button type="button" class="btn btn-sm btn-danger" onclick="ocrSchemaDelete()">Delete (user only)</button>
+        <span id="s-ocr-sch-status" class="form-hint" style="align-self:center"></span>
+      </div>
+    </div>
   </div>
 </div>
 
@@ -791,6 +1060,157 @@ function toggleColibriFields(){
     document.getElementById('s-colibri-enabled').checked ? 'block' : 'none';
 }
 
+function toggleK3Fields(){
+  document.getElementById('k3-fields').style.display =
+    document.getElementById('s-k3-enabled').checked ? 'block' : 'none';
+}
+
+function toggleKtFields(){
+  document.getElementById('kt-fields').style.display =
+    document.getElementById('s-kt-enabled').checked ? 'block' : 'none';
+}
+function toggleOcrFields(){
+  document.getElementById('ocr-fields').style.display =
+    document.getElementById('s-ocr-enabled').checked ? 'block' : 'none';
+}
+
+// ---- OCR isolated-venv build (paddle/surya) ----
+const _ocrBuildPollers = {};
+async function ocrBuildVenv(engine){
+  const statusEl = document.getElementById('s-ocr-'+engine+'-build-status');
+  statusEl.textContent = 'starting build…';
+  try{
+    const r = await fetch(ROOT_PATH + '/admin/api/ocr/build-venv', {
+      method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify({engine})});
+    if(!r.ok){ const e=await r.json(); throw new Error(e.detail||'failed'); }
+    ocrPollBuild(engine);
+  }catch(e){ statusEl.textContent = 'error: '+e.message; }
+}
+function ocrPollBuild(engine){
+  if(_ocrBuildPollers[engine]) clearInterval(_ocrBuildPollers[engine]);
+  const statusEl = document.getElementById('s-ocr-'+engine+'-build-status');
+  _ocrBuildPollers[engine] = setInterval(async () => {
+    try{
+      const r = await fetch(ROOT_PATH + '/admin/api/ocr/build-venv/status?engine='+engine);
+      const d = await r.json();
+      const tail = (d.log||[]).slice(-1)[0] || '';
+      statusEl.textContent = d.state + (tail ? ' — '+tail : '');
+      if(d.state === 'done' || d.state === 'error'){
+        clearInterval(_ocrBuildPollers[engine]); _ocrBuildPollers[engine] = null;
+      }
+    }catch(e){ /* keep polling */ }
+  }, 3000);
+}
+
+// ---- OCR extraction schema management (picker + raw JSON + field builder) ----
+let _ocrDefaultSchema = '';
+function _ocrSchStatus(msg, isErr){
+  const el = document.getElementById('s-ocr-sch-status');
+  el.textContent = msg || '';
+  el.style.color = isErr ? '#c0392b' : '';
+}
+async function ocrLoadSchemas(){
+  const sel = document.getElementById('s-ocr-extract-schema');
+  if(!sel) return;
+  try{
+    const r = await fetch(ROOT_PATH + '/admin/api/ocr/schemas');
+    const d = await r.json();
+    const list = (d.schemas || []);
+    sel.innerHTML = '<option value="">auto (generic key/value)</option>' +
+      list.map(s => `<option value="${s.name}">${s.name} — ${s.description||s.type} [${s.source}]</option>`).join('');
+    sel.value = _ocrDefaultSchema || '';
+  }catch(e){ _ocrSchStatus('failed to list schemas: '+e.message, true); }
+}
+function ocrSchemaPicked(){ /* selection is read on save; nothing else needed */ }
+
+function ocrBuilderRow(key, val){
+  const wrap = document.createElement('div');
+  wrap.className = 'ocr-sch-row';
+  wrap.style = 'display:flex;gap:.4rem;margin-bottom:.3rem';
+  wrap.innerHTML =
+    `<input type="text" class="form-input ocr-sch-key" placeholder="field name" style="flex:1" value="${(key||'').replace(/"/g,'&quot;')}">`+
+    `<input type="text" class="form-input ocr-sch-val" placeholder="type/hint e.g. string|null" style="flex:1" value="${(val||'').replace(/"/g,'&quot;')}">`+
+    `<button type="button" class="btn btn-sm btn-danger" onclick="this.parentNode.remove()">×</button>`;
+  document.getElementById('s-ocr-sch-builder').appendChild(wrap);
+}
+function ocrBuilderAddRow(){ ocrBuilderRow('', ''); }
+function ocrBuilderClear(){ document.getElementById('s-ocr-sch-builder').innerHTML = ''; }
+function ocrBuilderToJson(){
+  const rows = document.querySelectorAll('#s-ocr-sch-builder .ocr-sch-row');
+  const obj = {};
+  rows.forEach(r => {
+    const k = r.querySelector('.ocr-sch-key').value.trim();
+    const v = r.querySelector('.ocr-sch-val').value.trim();
+    if(k) obj[k] = v || 'string';
+  });
+  document.getElementById('s-ocr-sch-json').value = JSON.stringify(obj, null, 2);
+}
+function ocrJsonToBuilder(){
+  let obj;
+  try{ obj = JSON.parse(document.getElementById('s-ocr-sch-json').value || '{}'); }
+  catch(e){ _ocrSchStatus('raw JSON is invalid: '+e.message, true); return; }
+  ocrBuilderClear();
+  if(obj && typeof obj === 'object' && !Array.isArray(obj)){
+    Object.keys(obj).forEach(k => {
+      const v = obj[k];
+      ocrBuilderRow(k, (typeof v === 'string') ? v : JSON.stringify(v));
+    });
+  }
+  _ocrSchStatus('builder rebuilt from JSON (nested values shown as JSON text)');
+}
+async function ocrSchemaLoadIntoEditor(){
+  const name = document.getElementById('s-ocr-extract-schema').value;
+  if(!name){ _ocrSchStatus('pick a named schema first (auto has no body)', true); return; }
+  try{
+    const r = await fetch(ROOT_PATH + '/admin/api/ocr/schemas/' + encodeURIComponent(name));
+    if(!r.ok){ const e=await r.json(); throw new Error(e.detail||'not found'); }
+    const s = await r.json();
+    document.getElementById('s-ocr-sch-name').value = s.name || name;
+    document.getElementById('s-ocr-sch-type').value = s.type || 'template';
+    document.getElementById('s-ocr-sch-lang').value = s.language || '';
+    document.getElementById('s-ocr-sch-desc').value = s.description || '';
+    document.getElementById('s-ocr-sch-hint').value = s.prompt_hint || '';
+    document.getElementById('s-ocr-sch-json').value = JSON.stringify(s.schema || {}, null, 2);
+    ocrJsonToBuilder();
+    _ocrSchStatus('loaded "'+(s.name||name)+'" ('+(s.source||'')+')');
+  }catch(e){ _ocrSchStatus('load failed: '+e.message, true); }
+}
+async function ocrSchemaSave(){
+  const name = document.getElementById('s-ocr-sch-name').value.trim();
+  if(!name){ _ocrSchStatus('name is required', true); return; }
+  let schema;
+  try{ schema = JSON.parse(document.getElementById('s-ocr-sch-json').value || '{}'); }
+  catch(e){ _ocrSchStatus('raw JSON invalid: '+e.message, true); return; }
+  const payload = {
+    name,
+    type: document.getElementById('s-ocr-sch-type').value || 'template',
+    language: document.getElementById('s-ocr-sch-lang').value.trim(),
+    description: document.getElementById('s-ocr-sch-desc').value.trim(),
+    prompt_hint: document.getElementById('s-ocr-sch-hint').value.trim(),
+    schema,
+  };
+  try{
+    const r = await fetch(ROOT_PATH + '/admin/api/ocr/schemas', {
+      method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify(payload)});
+    if(!r.ok){ const e=await r.json(); throw new Error(e.detail||'save failed'); }
+    _ocrSchStatus('saved "'+name+'"');
+    await ocrLoadSchemas();
+    document.getElementById('s-ocr-extract-schema').value = name;
+  }catch(e){ _ocrSchStatus('save failed: '+e.message, true); }
+}
+async function ocrSchemaDelete(){
+  const name = document.getElementById('s-ocr-sch-name').value.trim() ||
+               document.getElementById('s-ocr-extract-schema').value;
+  if(!name){ _ocrSchStatus('nothing to delete', true); return; }
+  if(!confirm('Delete user schema "'+name+'"? (built-ins cannot be deleted)')) return;
+  try{
+    const r = await fetch(ROOT_PATH + '/admin/api/ocr/schemas/' + encodeURIComponent(name), {method:'DELETE'});
+    if(!r.ok){ const e=await r.json(); throw new Error(e.detail||'delete failed'); }
+    _ocrSchStatus('deleted "'+name+'"');
+    await ocrLoadSchemas();
+  }catch(e){ _ocrSchStatus('delete failed: '+e.message, true); }
+}
+
 let _ds4DefaultModels = [];
 async function loadDs4DefaultModels(){
   const sel = document.getElementById('s-ds4-dl-select');
@@ -1135,6 +1555,70 @@ async function loadSettings(){
     document.getElementById('s-colibri-extra-env').value = colibri.extra_env ?? '';
     toggleColibriFields();
     loadColibriDefaultModels();
+    // Kimi-K3 (kimi-k3-in-c)
+    const k3 = d.k3 || {};
+    document.getElementById('s-k3-enabled').checked = !!k3.enabled;
+    document.getElementById('s-k3-model-id').value = k3.model_id ?? 'kimi-k3';
+    document.getElementById('s-k3-model-path').value = k3.model_path ?? '';
+    document.getElementById('s-k3-trunk-dir').value = k3.trunk_dir ?? '';
+    document.getElementById('s-k3-preset').value = k3.preset ?? '';
+    document.getElementById('s-k3-trunk-gb').value = k3.trunk_gb ?? 16;
+    document.getElementById('s-k3-cache-gb').value = k3.cache_gb ?? 64;
+    document.getElementById('s-k3-ctx').value = k3.ctx ?? 4096;
+    document.getElementById('s-k3-auto-build').checked = k3.auto_build !== false;
+    document.getElementById('s-k3-extra-env').value = k3.extra_env ?? '';
+    toggleK3Fields();
+    // ktransformers (SGLang)
+    const kt = d.ktransformers || {};
+    document.getElementById('s-kt-enabled').checked = !!kt.enabled;
+    document.getElementById('s-kt-model-id').value = kt.model_id ?? 'ktransformers';
+    document.getElementById('s-kt-model-path').value = kt.model_path ?? '';
+    document.getElementById('s-kt-weight-path').value = kt.kt_weight_path ?? '';
+    document.getElementById('s-kt-host').value = kt.host ?? '127.0.0.1';
+    document.getElementById('s-kt-port').value = kt.port ?? 0;
+    document.getElementById('s-kt-ctx').value = kt.ctx ?? 32768;
+    document.getElementById('s-kt-extra-args').value = kt.extra_args ?? '';
+    document.getElementById('s-kt-auto-build').checked = !!kt.auto_build;
+    document.getElementById('s-kt-extra-env').value = kt.extra_env ?? '';
+    toggleKtFields();
+
+    // OCR subsystem
+    const ocr = d.ocr || {};
+    document.getElementById('s-ocr-enabled').checked = !!ocr.enabled;
+    document.getElementById('s-ocr-default-engine').value = ocr.default_engine ?? 'paddle';
+    document.getElementById('s-ocr-dpi').value = ocr.dpi ?? 200;
+    document.getElementById('s-ocr-max-concurrency').value = ocr.max_concurrency ?? 4;
+    document.getElementById('s-ocr-lang').value = ocr.lang ?? 'it';
+    document.getElementById('s-ocr-paddle-enabled').checked = ocr.paddle_enabled !== false;
+    document.getElementById('s-ocr-paddle-use-gpu').checked = ocr.paddle_use_gpu !== false;
+    document.getElementById('s-ocr-paddle-structure').checked = ocr.paddle_structure !== false;
+    document.getElementById('s-ocr-paddle-instances').value = ocr.paddle_instances ?? 2;
+    document.getElementById('s-ocr-paddle-lang').value = ocr.paddle_lang ?? 'it';
+    document.getElementById('s-ocr-paddle-det-dir').value = ocr.paddle_det_model_dir ?? '';
+    document.getElementById('s-ocr-paddle-rec-dir').value = ocr.paddle_rec_model_dir ?? '';
+    document.getElementById('s-ocr-paddle-venv').value = ocr.paddle_venv ?? '';
+    document.getElementById('s-ocr-paddle-auto-build').checked = !!ocr.paddle_auto_build;
+    document.getElementById('s-ocr-doctr-enabled').checked = !!ocr.doctr_enabled;
+    document.getElementById('s-ocr-doctr-use-gpu').checked = ocr.doctr_use_gpu !== false;
+    document.getElementById('s-ocr-doctr-instances').value = ocr.doctr_instances ?? 1;
+    document.getElementById('s-ocr-doctr-det').value = ocr.doctr_det_arch ?? 'db_resnet50';
+    document.getElementById('s-ocr-doctr-reco').value = ocr.doctr_reco_arch ?? 'crnn_vgg16_bn';
+    document.getElementById('s-ocr-surya-enabled').checked = !!ocr.surya_enabled;
+    document.getElementById('s-ocr-surya-license').checked = !!ocr.surya_accept_license;
+    document.getElementById('s-ocr-surya-instances').value = ocr.surya_instances ?? 1;
+    document.getElementById('s-ocr-surya-langs').value = ocr.surya_langs ?? 'it';
+    document.getElementById('s-ocr-surya-venv').value = ocr.surya_venv ?? '';
+    document.getElementById('s-ocr-surya-auto-build').checked = !!ocr.surya_auto_build;
+    document.getElementById('s-ocr-detect-mode').value = ocr.detect_mode ?? 'off';
+    document.getElementById('s-ocr-detect-model').value = ocr.detect_model_path ?? '';
+    document.getElementById('s-ocr-detect-conf').value = ocr.detect_conf ?? 0.35;
+    document.getElementById('s-ocr-extract-enabled').checked = !!ocr.extract_enabled;
+    document.getElementById('s-ocr-extract-validate').checked = ocr.extract_validate !== false;
+    document.getElementById('s-ocr-extract-max-tokens').value = ocr.extract_max_tokens ?? 2048;
+    document.getElementById('s-ocr-extract-model').value = ocr.extract_model_id ?? '';
+    _ocrDefaultSchema = ocr.extract_schema ?? '';
+    toggleOcrFields();
+    ocrLoadSchemas();
   }catch(e){ showAlert('error','Failed to load settings: '+e.message); }
 }
 
@@ -1235,6 +1719,65 @@ async function saveSettings(){
       repo_url: document.getElementById('s-colibri-repo-url').value.trim() || 'https://github.com/JustVugg/colibri',
       extra_env: document.getElementById('s-colibri-extra-env').value.trim(),
     },
+    k3:{
+      enabled: document.getElementById('s-k3-enabled').checked,
+      model_id: document.getElementById('s-k3-model-id').value.trim() || 'kimi-k3',
+      model_path: document.getElementById('s-k3-model-path').value.trim(),
+      trunk_dir: document.getElementById('s-k3-trunk-dir').value.trim(),
+      preset: document.getElementById('s-k3-preset').value,
+      trunk_gb: parseFloat(document.getElementById('s-k3-trunk-gb').value) || 16,
+      cache_gb: parseFloat(document.getElementById('s-k3-cache-gb').value) || 64,
+      ctx: parseInt(document.getElementById('s-k3-ctx').value) || 4096,
+      auto_build: document.getElementById('s-k3-auto-build').checked,
+      extra_env: document.getElementById('s-k3-extra-env').value.trim(),
+    },
+    ktransformers:{
+      enabled: document.getElementById('s-kt-enabled').checked,
+      model_id: document.getElementById('s-kt-model-id').value.trim() || 'ktransformers',
+      model_path: document.getElementById('s-kt-model-path').value.trim(),
+      kt_weight_path: document.getElementById('s-kt-weight-path').value.trim(),
+      host: document.getElementById('s-kt-host').value.trim() || '127.0.0.1',
+      port: parseInt(document.getElementById('s-kt-port').value) || 0,
+      ctx: parseInt(document.getElementById('s-kt-ctx').value) || 32768,
+      extra_args: document.getElementById('s-kt-extra-args').value.trim(),
+      auto_build: document.getElementById('s-kt-auto-build').checked,
+      extra_env: document.getElementById('s-kt-extra-env').value.trim(),
+    },
+    ocr:{
+      enabled: document.getElementById('s-ocr-enabled').checked,
+      default_engine: document.getElementById('s-ocr-default-engine').value || 'paddle',
+      dpi: parseInt(document.getElementById('s-ocr-dpi').value) || 200,
+      max_concurrency: parseInt(document.getElementById('s-ocr-max-concurrency').value) || 4,
+      lang: document.getElementById('s-ocr-lang').value.trim() || 'it',
+      paddle_enabled: document.getElementById('s-ocr-paddle-enabled').checked,
+      paddle_use_gpu: document.getElementById('s-ocr-paddle-use-gpu').checked,
+      paddle_structure: document.getElementById('s-ocr-paddle-structure').checked,
+      paddle_instances: parseInt(document.getElementById('s-ocr-paddle-instances').value) || 2,
+      paddle_lang: document.getElementById('s-ocr-paddle-lang').value.trim() || 'it',
+      paddle_det_model_dir: document.getElementById('s-ocr-paddle-det-dir').value.trim(),
+      paddle_rec_model_dir: document.getElementById('s-ocr-paddle-rec-dir').value.trim(),
+      paddle_venv: document.getElementById('s-ocr-paddle-venv').value.trim(),
+      paddle_auto_build: document.getElementById('s-ocr-paddle-auto-build').checked,
+      doctr_enabled: document.getElementById('s-ocr-doctr-enabled').checked,
+      doctr_use_gpu: document.getElementById('s-ocr-doctr-use-gpu').checked,
+      doctr_instances: parseInt(document.getElementById('s-ocr-doctr-instances').value) || 1,
+      doctr_det_arch: document.getElementById('s-ocr-doctr-det').value.trim() || 'db_resnet50',
+      doctr_reco_arch: document.getElementById('s-ocr-doctr-reco').value.trim() || 'crnn_vgg16_bn',
+      surya_enabled: document.getElementById('s-ocr-surya-enabled').checked,
+      surya_accept_license: document.getElementById('s-ocr-surya-license').checked,
+      surya_instances: parseInt(document.getElementById('s-ocr-surya-instances').value) || 1,
+      surya_langs: document.getElementById('s-ocr-surya-langs').value.trim() || 'it',
+      surya_venv: document.getElementById('s-ocr-surya-venv').value.trim(),
+      surya_auto_build: document.getElementById('s-ocr-surya-auto-build').checked,
+      detect_mode: document.getElementById('s-ocr-detect-mode').value || 'off',
+      detect_model_path: document.getElementById('s-ocr-detect-model').value.trim(),
+      detect_conf: parseFloat(document.getElementById('s-ocr-detect-conf').value) || 0.35,
+      extract_enabled: document.getElementById('s-ocr-extract-enabled').checked,
+      extract_validate: document.getElementById('s-ocr-extract-validate').checked,
+      extract_max_tokens: parseInt(document.getElementById('s-ocr-extract-max-tokens').value) || 2048,
+      extract_model_id: document.getElementById('s-ocr-extract-model').value.trim(),
+      extract_schema: document.getElementById('s-ocr-extract-schema').value || '',
+    },
     compaction:{
       enabled: document.getElementById('s-compact-enabled').checked,
       pct: parseInt(document.getElementById('s-compact-pct').value) || 85,

codai/api/app.py

@@ -142,6 +142,7 @@ from codai.api.characters import router as characters_router
 from codai.api.loras import router as loras_router
 from codai.api.spatial import router as spatial_router
 from codai.api.environments import router as environments_router
+from codai.api.ocr import router as ocr_router
 from codai.admin.routes import router as admin_router
 
 # Import and add middleware
@@ -574,6 +575,7 @@ app.include_router(characters_router, tags=["Characters"])
 app.include_router(loras_router, tags=["LoRAs"])
 app.include_router(environments_router, tags=["Environments"])
 app.include_router(spatial_router, tags=["Spatial / 3D"])
+app.include_router(ocr_router, tags=["OCR"])
 app.include_router(admin_router, tags=["Admin"])
 
 

codai/api/colibri_worker.py

@@ -69,15 +69,54 @@ def _install_dir(cfg) -> Path:
         else default_install_dir()
 
 
-def _engine_bin(install_dir: Path) -> Path:
-    """The C engine binary. colibri builds it as ``colibri`` inside the ``c/`` dir
-    (``glm`` is the pre-#391 name, kept as a fallback for old trees)."""
+# colibri v1.5.0 ships one C engine per model family. Each is a distinct binary in the
+# ``c/`` dir, built by its own make target, and speaks the shared serve framing (see
+# project memory ``project_colibri_v15_families``). ``glm`` is the default family.
+_FAMILIES = ("glm", "deepseek_v4", "kimi_k3")
+_FAMILY_BIN = {"glm": "colibri", "deepseek_v4": "deepseek_v4", "kimi_k3": "kimi_k3"}
+# The make target name (Makefile) differs from the binary name for DeepSeek (hyphen).
+_FAMILY_TARGET_NAME = {"glm": "colibri", "deepseek_v4": "deepseek-v4", "kimi_k3": "kimi_k3"}
+
+
+def _detect_family(container_dir: Optional[str]) -> str:
+    """Infer the colibri model family from a container's ``config.json`` (falling back
+    to the directory name). Returns one of :data:`_FAMILIES` (default ``"glm"``)."""
+    import json as _json
+    blob = ""
+    try:
+        if container_dir:
+            with open(os.path.join(container_dir, "config.json"),
+                      encoding="utf-8", errors="ignore") as f:
+                cfg = _json.load(f)
+            mt = str(cfg.get("model_type") or "")
+            archs = " ".join(str(a) for a in (cfg.get("architectures") or []))
+            blob = (mt + " " + archs).lower()
+    except Exception:
+        blob = ""
+    hint = (os.path.basename(os.path.expanduser(container_dir or "").rstrip("/"))).lower()
+    text = blob + " " + hint
+    if "kimi" in text or "k3" in text:
+        return "kimi_k3"
+    if "deepseek" in text:
+        return "deepseek_v4"
+    if "glm" in text:
+        return "glm"
+    return "glm"
+
+
+def _engine_bin(install_dir: Path, family: str = "glm") -> Path:
+    """The C engine binary for ``family`` inside the ``c/`` dir. colibri builds GLM as
+    ``colibri`` (``glm`` is the pre-#391 name, kept as a fallback); DeepSeek-V4 as
+    ``deepseek_v4`` and Kimi-K3 as ``kimi_k3``."""
     cdir = install_dir / "c"
-    for name in ("colibri", "colibri.exe", "glm", "glm.exe"):
+    base = _FAMILY_BIN.get(family, "colibri")
+    names = ((base, base + ".exe", "glm", "glm.exe") if family == "glm"
+             else (base, base + ".exe"))
+    for name in names:
         cand = cdir / name
         if cand.exists():
             return cand
-    return cdir / "colibri"
+    return cdir / base
 
 
 def _find_nvcc() -> Optional[str]:
@@ -108,26 +147,43 @@ def _detect_build_target() -> str:
     return "cpu"
 
 
-def _make_args(cfg) -> list:
-    """``make`` arguments for the resolved build target."""
+def _make_args(cfg, family: str = "glm") -> list:
+    """``make`` arguments for ``family``'s engine on the resolved build target."""
+    tgt = _FAMILY_TARGET_NAME.get(family, "colibri")
     target = (getattr(cfg, "build_target", "auto") or "auto").strip().lower()
     if target in ("", "auto"):
         target = _detect_build_target()
+    # Kimi-K3 has no CUDA path: it is CPU + an opt-in Vulkan compute tier (make VK=1,
+    # needs glslc/shaderc). Build CPU by default; enable VK via COLI_K3_VK or a
+    # ``vulkan`` build_target.
+    if family == "kimi_k3":
+        args = [tgt]
+        want_vk = (str(os.environ.get("COLI_K3_VK", "")).strip().lower() in ("1", "true", "on")
+                   or target == "vulkan"
+                   or (getattr(cfg, "build_target", "") or "").strip().lower() == "vulkan")
+        if want_vk:
+            args.append("VK=1")
+        return args
+    # DeepSeek-V4 (colibri's engine) is CPU/AVX2 — Makefile.deepseek-v4 has no CUDA path
+    # (distinct from the separate ds4 CUDA engine). Stream experts from disk; tune via
+    # RAM_GB. Just build the target on the host's default ARCH.
+    if family == "deepseek_v4":
+        return [tgt]
     if target == "cuda":
         # Default to a PORTABLE arch (SASS for Ampere..Blackwell + a PTX fallback) so
         # a pre-compiled/bundled binary runs on any modern card; override with
         # COLI_CUDA_ARCH (e.g. sm_86 for a single 3090 build). Point the Makefile at
         # a real nvcc/CUDA_HOME so a non-PATH toolkit still builds.
-        args = ["colibri", "CUDA=1", f"CUDA_ARCH={os.environ.get('COLI_CUDA_ARCH', 'portable')}"]
+        args = [tgt, "CUDA=1", f"CUDA_ARCH={os.environ.get('COLI_CUDA_ARCH', 'portable')}"]
         nvcc = _find_nvcc()
         if nvcc:
             args += [f"NVCC={nvcc}", f"CUDA_HOME={os.path.dirname(os.path.dirname(nvcc))}"]
         return args
     if target == "hip":
-        return ["colibri", "HIP=1", f"HIP_ARCH={os.environ.get('COLI_HIP_ARCH', 'native')}"]
+        return [tgt, "HIP=1", f"HIP_ARCH={os.environ.get('COLI_HIP_ARCH', 'native')}"]
     if target == "metal":
-        return ["colibri", "METAL=1"]
-    return ["colibri"]
+        return [tgt, "METAL=1"]
+    return [tgt]
 
 
 def _run_logged(cmd, cwd, label, tail, **kw):
@@ -146,36 +202,38 @@ def _run_logged(cmd, cwd, label, tail, **kw):
         raise RuntimeError(f"{label} failed (exit {proc.returncode}). {joined}")
 
 
-def ensure_built(cfg) -> Path:
-    """Clone + build colibri if the engine binary is missing. Returns its path."""
+def ensure_built(cfg, family: str = "glm") -> Path:
+    """Clone + build the ``family`` colibri engine if its binary is missing. Returns
+    its path. Each family (glm/deepseek_v4/kimi_k3) is a separate binary + make target."""
     global _built
     install_dir = _install_dir(cfg)
-    binary = _engine_bin(install_dir)
+    binary = _engine_bin(install_dir, family)
     if binary.exists():
         _built = True
         return binary
+    tgt = _FAMILY_TARGET_NAME.get(family, "colibri")
     if not getattr(cfg, "auto_build", True):
         raise RuntimeError(
-            f"colibri engine not found at {binary} and auto_build is disabled. Build it "
-            f"manually (git clone {cfg.repo_url}; cd c; make colibri [CUDA=1]) or enable "
-            "auto_build.")
-
-    # A CUDA build needs the toolkit (nvcc), which a CUDA *runtime* container does
-    # NOT ship. Rather than clone + fail deep in `make` (the confusing
-    # "nvcc not found … backend_cuda.o Error" the user hit), fail early with the fix:
-    # pre-compile the binary. The engine is meant to be pre-built (bundled in the
-    # image or built once on a box with the toolkit), never compiled per-request.
+            f"colibri {family} engine not found at {binary} and auto_build is disabled. "
+            f"Build it manually (git clone {cfg.repo_url}; cd c; make {tgt} [CUDA=1|VK=1]) "
+            "or enable auto_build.")
+
+    # The CUDA families (glm/deepseek_v4) need the toolkit (nvcc), which a CUDA *runtime*
+    # container does NOT ship. Rather than clone + fail deep in `make` (the confusing
+    # "nvcc not found … Error"), fail early with the fix: pre-compile the binary. The
+    # engine is meant to be pre-built (bundled in the image or built once on a box with
+    # the toolkit), never compiled per-request. Kimi-K3 is CPU/Vulkan, so it is exempt.
     _target = (getattr(cfg, "build_target", "auto") or "auto").strip().lower()
     if _target in ("", "auto"):
         _target = _detect_build_target()
-    if _target == "cuda" and _find_nvcc() is None:
+    if family != "kimi_k3" and _target == "cuda" and _find_nvcc() is None:
         raise RuntimeError(
-            f"colibri needs a CUDA build but no CUDA toolkit (nvcc) is available here "
-            f"— this looks like a CUDA *runtime* environment. Pre-compile the engine "
-            f"and place it at {binary}: build once on a box/image with the matching "
-            f"CUDA toolkit (e.g. `build.sh --colibri`, or a cuda:*-devel container) "
-            f"targeting the same CUDA runtime as this container. Runtime auto-build "
-            f"cannot compile CUDA without nvcc.")
+            f"colibri {family} needs a CUDA build but no CUDA toolkit (nvcc) is available "
+            f"here — this looks like a CUDA *runtime* environment. Pre-compile the engine "
+            f"and place it at {binary}: build once on a box/image with the matching CUDA "
+            f"toolkit (e.g. `build.sh --colibri`, or a cuda:*-devel container) targeting "
+            f"the same CUDA runtime as this container. Runtime auto-build cannot compile "
+            f"CUDA without nvcc.")
 
     tail = collections.deque(maxlen=40)
     install_dir.parent.mkdir(parents=True, exist_ok=True)
@@ -185,15 +243,15 @@ def ensure_built(cfg) -> Path:
                     cwd=install_dir.parent, label="git clone", tail=tail)
 
     cdir = install_dir / "c"
-    make_args = _make_args(cfg)
-    print(f"[colibri] building engine (make {' '.join(make_args)}) — this can take a while …",
-          flush=True)
+    make_args = _make_args(cfg, family)
+    print(f"[colibri] building {family} engine (make {' '.join(make_args)}) — this can "
+          "take a while …", flush=True)
     _run_logged(["make", "-s"] + make_args, cwd=cdir, label="make", tail=tail)
 
-    binary = _engine_bin(install_dir)
+    binary = _engine_bin(install_dir, family)
     if not binary.exists():
         raise RuntimeError(
-            f"colibri build completed but {binary} is missing. Last output: "
+            f"colibri {family} build completed but {binary} is missing. Last output: "
             + " | ".join(list(tail)[-5:]))
     _built = True
     print(f"[colibri] built {binary}", flush=True)
@@ -219,6 +277,15 @@ def _read_engine_turn(stream, sentinel: bytes) -> dict:
     return _parse_stat(fields)
 
 
+def _to_payload(prompt) -> bytes:
+    """Serve payload bytes for a prompt. GLM/DeepSeek send a rendered UTF-8 string;
+    Kimi-K3 sends a pre-framed ``K3CHAT1`` blob (already ``bytes``) that must go on the
+    wire verbatim — so accept ``bytes``/``bytearray`` as-is and only encode ``str``."""
+    if isinstance(prompt, (bytes, bytearray)):
+        return bytes(prompt)
+    return prompt.encode("utf-8")
+
+
 def _parse_stat(fields) -> dict:
     return {
         "completion_tokens": int(fields[1]),
@@ -241,7 +308,9 @@ class MuxEngine:
     """
 
     def __init__(self, binary: Path, model_dir: str, *, cap: int = 8,
-                 max_tokens: int = 1024, kv_slots: int = 1, env: Optional[dict] = None):
+                 max_tokens: int = 1024, kv_slots: int = 1, env: Optional[dict] = None,
+                 force_mux: bool = False, family: str = "glm",
+                 argv: Optional[list] = None, inject_colibri_env: bool = True):
         kv_slots = max(1, min(16, int(kv_slots or 1)))
         # colibri's context window is the CTX env (default 4096) — DISTINCT from NGEN
         # (max new tokens). Without CTX the engine rejects any prompt over ~4095 tokens
@@ -257,15 +326,31 @@ class MuxEngine:
         # a real decode win (accepted drafts = fewer forward passes = fewer expert
         # streams). Multi-slot (kv_slots>1) keeps the batched mux path (concurrent
         # conversations, no MTP). See docs/serve_protocol.md + colibri.c run_serve.
-        self.single_client = (int(kv_slots) == 1)
-        child_env = dict(env or os.environ, SNAP=str(model_dir), SERVE="1",
-                         NGEN=str(max_tokens), KV_SLOTS=str(kv_slots))
-        child_env["SERVE_BATCH"] = "0" if self.single_client else "1"
-        child_env.setdefault("CTX", str(ctx))
+        #
+        # ``force_mux`` overrides this for the non-GLM families: colibri's ``deepseek_v4``
+        # and ``kimi_k3`` engines implement ONLY the mux ``SUBMIT`` serve loop — they have
+        # no ``\x02PROMPT`` (run_serve) path — so they must speak the mux protocol even at
+        # a single KV slot. (They are single-slot: slot 0 only / slot ignored.) See
+        # project memory ``project_colibri_v15_families``.
+        self.family = family or "glm"
+        self.single_client = (int(kv_slots) == 1) and not force_mux
+        # colibri's engines take the container via SNAP + serve knobs via env. Other
+        # engines driven over the SAME mux protocol (e.g. the patched kimi-k3-in-c) take
+        # the model dir + flags as ARGV and their own env — pass argv + a full env and
+        # set inject_colibri_env=False so we don't force colibri's SNAP/NGEN/KV_SLOTS.
+        if inject_colibri_env:
+            child_env = dict(env or os.environ, SNAP=str(model_dir), SERVE="1",
+                             NGEN=str(max_tokens), KV_SLOTS=str(kv_slots))
+            child_env["SERVE_BATCH"] = "0" if self.single_client else "1"
+            child_env.setdefault("CTX", str(ctx))
+        else:
+            child_env = dict(env or os.environ)
+            child_env.setdefault("SERVE", "1")
         self.model_dir = str(model_dir)
         self.kv_slots = kv_slots
+        cmd = [str(a) for a in argv] if argv else [str(binary), str(cap)]
         self.process = subprocess.Popen(
-            [str(binary), str(cap)], env=child_env, stdin=subprocess.PIPE,
+            cmd, env=child_env, stdin=subprocess.PIPE,
             stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=0,
         )
         self.write_lock = threading.Lock()
@@ -455,7 +540,7 @@ class MuxEngine:
         normal END+STAT."""
         if self.process.poll() is not None:
             raise RuntimeError("colibri engine is not running. " + self.log_tail())
-        payload = prompt.encode("utf-8")
+        payload = _to_payload(prompt)
         if b"\0" in payload:
             raise ValueError("NUL bytes are not supported in prompts.")
         # colibri validates: temp∈[0,2], top_p∈(0,1], ngen≥1, slot 0.
@@ -532,7 +617,7 @@ class MuxEngine:
                                + (self.log_tail() or str(self.dispatcher_error)))
         if self.process.poll() is not None:
             raise RuntimeError("colibri engine is not running. " + self.log_tail())
-        payload = prompt.encode("utf-8")
+        payload = _to_payload(prompt)
         if b"\0" in payload:
             raise ValueError("NUL bytes are not supported in prompts.")
         decoder = codecs.getincrementaldecoder("utf-8")("replace")
@@ -674,14 +759,18 @@ def resolve_service_key(cfg, model_dir: Optional[str] = None):
     return resolved, svc_key
 
 
-def _build_env(cfg) -> tuple:
+def _build_env(cfg, family: str = "glm") -> tuple:
     """Engine environment: enable the GPU backend + CUDA_EXPERT_GB + free-form extra_env.
 
     colibri's GPU tiering is OFF unless explicitly enabled — ``COLI_CUDA=1`` (CUDA/HIP)
     or ``COLI_METAL=1`` (Apple). It is NOT implied by shipping a CUDA build, and
     ``CUDA_EXPERT_GB`` without ``COLI_CUDA=1`` aborts startup ("CUDA_EXPERT_GB requires
     COLI_CUDA=1"). Since coderai routes colibri to the CUDA engine and ships a CUDA
-    binary, default the backend on (overridable via extra_env, e.g. COLI_CUDA=0)."""
+    binary, default the backend on (overridable via extra_env, e.g. COLI_CUDA=0).
+
+    Only the GLM/DeepSeek families use the ``COLI_CUDA``/``CUDA_EXPERT_GB`` knobs; Kimi-K3
+    has its own ``K3_*`` env (K3_VK for the Vulkan tier, K3_EXPERT_GB, K3_BITS, …), so we
+    do NOT force the COLI_CUDA vars for it — Kimi tuning goes through ``extra_env``."""
     env = os.environ.copy()
     applied = {}
 
@@ -689,7 +778,12 @@ def _build_env(cfg) -> tuple:
     if target in ("", "auto"):
         target = "cuda"    # coderai ships a CUDA build and routes colibri to the CUDA engine
     cuda_on = False
-    if target in ("cuda", "hip"):
+    if family in ("kimi_k3", "deepseek_v4"):
+        # colibri's Kimi-K3 and DeepSeek-V4 engines are CPU (Kimi + opt-in Vulkan); the
+        # COLI_CUDA/CUDA_EXPERT_GB knobs don't apply. Leave them unset for a clean env —
+        # Kimi tuning goes via K3_* and DeepSeek via RAM_GB, both through extra_env.
+        pass
+    elif target in ("cuda", "hip"):
         if "COLI_CUDA" not in env:
             env["COLI_CUDA"] = "1"
             applied["COLI_CUDA"] = "1"
@@ -729,6 +823,9 @@ def ensure_engine(cfg, model_dir: Optional[str] = None, ctx: Optional[int] = Non
     per-turn generation budget (NGEN). Returns a live :class:`MuxEngine`.
     """
     resolved, svc_key = resolve_service_key(cfg, model_dir)
+    # Family is inferred from the resolved container (config.json / dir name) and
+    # decides the binary, the make target, and the serve protocol.
+    family = _detect_family(resolved)
     with _lock:
         eng = _services.get(svc_key)
         if eng and eng.is_alive():
@@ -737,12 +834,12 @@ def ensure_engine(cfg, model_dir: Optional[str] = None, ctx: Optional[int] = Non
             eng.close()
             _services.pop(svc_key, None)
 
-        binary = ensure_built(cfg)
+        binary = ensure_built(cfg, family)
         if not resolved:
             raise RuntimeError(
-                "colibri: no GLM-5.2 container resolved for this request. Point the "
-                "model at the int4 container directory (or set colibri.model_path). "
-                "There is no auto-download of the ~372 GB container.")
+                "colibri: no model container resolved for this request. Point the model "
+                "at the container directory (GLM-5.2 int4 / DeepSeek-V4 / Kimi-K3), or set "
+                "colibri.model_path. There is no auto-download of the multi-GB container.")
 
         try:
             ngen = int(ctx) if ctx else 0
@@ -751,15 +848,20 @@ def ensure_engine(cfg, model_dir: Optional[str] = None, ctx: Optional[int] = Non
         if ngen <= 0:
             ngen = int(getattr(cfg, "ctx", 100000) or 100000)
 
-        env, applied = _build_env(cfg)
+        env, applied = _build_env(cfg, family)
         env_note = ("  (" + " ".join(f"{k}={v}" for k, v in applied.items()) + ")"
                     if applied else "")
-        kv_slots = int(getattr(cfg, "kv_slots", 1) or 1)
+        # GLM keeps its configurable KV-slot pool (and MTP at 1 slot); DeepSeek-V4 and
+        # Kimi-K3 are single-slot mux engines (no run_serve/PROMPT path), so pin them to
+        # 1 slot and force the mux SUBMIT protocol.
+        force_mux = family != "glm"
+        kv_slots = 1 if force_mux else int(getattr(cfg, "kv_slots", 1) or 1)
         cap = int(getattr(cfg, "cap", 8) or 8)
-        print(f"[colibri] launching engine {binary} on {resolved} "
-              f"(cap={cap}, kv_slots={kv_slots}, ngen={ngen}){env_note}", flush=True)
+        print(f"[colibri] launching {family} engine {binary} on {resolved} "
+              f"(cap={cap}, kv_slots={kv_slots}, ngen={ngen}, mux={force_mux}){env_note}",
+              flush=True)
         eng = MuxEngine(binary, resolved, cap=cap, max_tokens=ngen,
-                        kv_slots=kv_slots, env=env)
+                        kv_slots=kv_slots, env=env, force_mux=force_mux, family=family)
         _services[svc_key] = eng
 
     # READY already completed inside MuxEngine.__init__; a quick liveness gate here

codai/api/custom_pipelines.py

@@ -89,6 +89,7 @@ STEP_TYPES = {
     "audio_gen":      ("codai.api.audio_gen",     "audio_generate",         None),
     "voice_clone":    ("codai.api.voice_clone",   "clone_voice",            None),
     "voice_convert":  ("codai.api.voice_convert", "convert_voice",          None),
+    "ocr":            ("codai.api.ocr",           "run_ocr_step",           None),
 }
 
 # Human-readable labels for the UI
@@ -112,6 +113,7 @@ STEP_TYPE_LABELS = {
     "audio_gen":      "Audio/Music Generation",
     "voice_clone":    "Voice Clone (TTS)",
     "voice_convert":  "Voice Convert (SVC)",
+    "ocr":            "OCR (document → text)",
 }
 
 # Which params each step type accepts (for the UI form builder)
@@ -136,6 +138,7 @@ STEP_PARAMS = {
     "audio_cleanup":  [("audio","ref","Source audio"),("noise_reduction","checkbox","Reduce background noise"),("normalize","checkbox","Normalize levels"),("remove_hum","checkbox","Remove hum"),("repair_clicks","checkbox","Repair clicks")],
     "voice_clone":    [("text","textarea","Text to synthesize"),("voice_name","text","Voice profile name"),("ref_text","text","Reference transcript"),("speed","number","Speed","1.0")],
     "voice_convert":  [("source_audio","ref","Source audio"),("voice_name","text","Voice profile name"),("f0_condition","checkbox","Singing mode"),("pitch_shift","number","Pitch shift","0"),("diffusion_steps","number","Steps","10")],
+    "ocr":            [("image","ref","Document image/PDF ({{stepN.url}})"),("engine","select:paddle|doctr|surya","OCR engine","paddle"),("detect","select:off|layout|detector|both","Stamp/signature detect","off"),("structured","checkbox","Structured JSON extraction")],
 }
 
 

codai/api/k3_worker.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""Fully-managed Kimi-K3 worker — the kimi-k3-in-c C engine, driven directly.
+
+kimi-k3-in-c (https://github.com/FareedKhan-dev/kimi-k3-in-c) is a portable-C CPU
+engine for Kimi-K3 that streams the dense trunk + routed experts from disk. Upstream it
+is a one-shot batch CLI; coderai applies ``packaging/patch-k3.py`` to add a resident
+serve loop that speaks the SAME mux stdin/stdout protocol as colibri, so we reuse
+:class:`~codai.api.colibri_worker.MuxEngine` as the protocol client. This module owns the
+build + process lifecycle; the Kimi-K3 chat template lives in :mod:`codai.backends.k3`.
+
+Lifecycle (mirrors the other managed engines):
+
+* :func:`ensure_built` clones the repo, applies the serve-loop patch, and runs ``make``
+  so the ``bin/k3`` binary exists (idempotent).
+* :func:`ensure_engine` launches the engine on the configured checkpoint in serve mode
+  (``SERVE=1`` + the model dir / ``--trunk`` / ``--tok`` / budgets as argv), completes
+  the ``READY`` handshake, and returns a live :class:`MuxEngine`.
+"""
+
+import collections
+import os
+import shutil
+import subprocess
+import threading
+import time
+from pathlib import Path
+from typing import Optional
+
+from codai.api.colibri_worker import MuxEngine, _run_logged
+
+_lock = threading.RLock()
+_services: dict[str, MuxEngine] = {}
+
+
+def default_install_dir() -> Path:
+    return Path(os.environ.get("CODERAI_K3_DIR")
+               or os.path.expanduser("~/.coderai/kimi-k3-in-c"))
+
+
+def _install_dir(cfg) -> Path:
+    return Path(cfg.install_dir).expanduser() if getattr(cfg, "install_dir", None) \
+        else default_install_dir()
+
+
+def _engine_bin(install_dir: Path) -> Path:
+    """The k3 engine binary (``bin/k3`` in the repo root)."""
+    for name in ("k3", "k3.exe"):
+        cand = install_dir / "bin" / name
+        if cand.exists():
+            return cand
+    return install_dir / "bin" / "k3"
+
+
+def _patch_script() -> Optional[str]:
+    """Path to coderai's serve-loop patch (packaging/patch-k3.py), or None."""
+    here = Path(__file__).resolve()
+    for base in (here.parents[2], here.parents[1]):   # repo root, then codai/
+        cand = base / "packaging" / "patch-k3.py"
+        if cand.exists():
+            return str(cand)
+    return None
+
+
+def ensure_built(cfg) -> Path:
+    """Clone + patch + build kimi-k3-in-c if the ``bin/k3`` binary is missing."""
+    install_dir = _install_dir(cfg)
+    binary = _engine_bin(install_dir)
+    if binary.exists():
+        return binary
+    if not getattr(cfg, "auto_build", True):
+        raise RuntimeError(
+            f"k3 engine not found at {binary} and auto_build is disabled. Build it "
+            f"manually (git clone {cfg.repo_url}; python3 packaging/patch-k3.py "
+            f"src/cli/k3_run.c; make) or enable auto_build.")
+    tail = collections.deque(maxlen=40)
+    install_dir.parent.mkdir(parents=True, exist_ok=True)
+    if not (install_dir / ".git").exists() and not (install_dir / "Makefile").exists():
+        print(f"[k3] cloning {cfg.repo_url} → {install_dir} …", flush=True)
+        _run_logged(["git", "clone", "--depth", "1", cfg.repo_url, str(install_dir)],
+                    cwd=install_dir.parent, label="git clone", tail=tail)
+    # Apply the resident serve-loop patch (idempotent) so the engine can stay warm.
+    patch = _patch_script()
+    k3_run = install_dir / "src" / "cli" / "k3_run.c"
+    if patch and k3_run.exists():
+        try:
+            _run_logged(["python3", patch, str(k3_run)],
+                        cwd=install_dir, label="patch-k3", tail=tail)
+        except Exception as exc:
+            print(f"[k3] warning: serve-loop patch failed ({exc}); building unpatched "
+                  "(serve mode will not work)", flush=True)
+    print("[k3] building engine (make) — CPU/AVX2, this can take a moment …", flush=True)
+    _run_logged(["make", "-j"], cwd=install_dir, label="make", tail=tail)
+    binary = _engine_bin(install_dir)
+    if not binary.exists():
+        raise RuntimeError("k3 build completed but bin/k3 is missing. Last output: "
+                           + " | ".join(list(tail)[-5:]))
+    print(f"[k3] built {binary}", flush=True)
+    return binary
+
+
+def _resolve_checkpoint(cfg, model_dir: Optional[str]) -> str:
+    """The Kimi-K3 checkpoint directory (config.json + tokenizer + shards)."""
+    for cand in (model_dir, getattr(cfg, "model_path", "") or ""):
+        cand = os.path.expanduser((cand or "").strip())
+        if cand and os.path.isdir(cand):
+            return os.path.abspath(cand)
+    return ""
+
+
+def resolve_service_key(cfg, model_dir: Optional[str] = None):
+    ckpt = _resolve_checkpoint(cfg, model_dir)
+    return ckpt, (ckpt or (getattr(cfg, "model_id", "kimi-k3") or "kimi-k3"))
+
+
+def _build_argv(cfg, binary: Path, ckpt: str) -> list:
+    """The k3 serve command line: model dir + trunk/tok/budget flags."""
+    argv = [str(binary), ckpt]
+    trunk = os.path.expanduser((getattr(cfg, "trunk_dir", "") or "").strip())
+    if trunk:
+        argv += ["--trunk", trunk]
+    tok = os.path.expanduser((getattr(cfg, "tok_dir", "") or "").strip()) or ckpt
+    argv += ["--tok", tok]
+    preset = (getattr(cfg, "preset", "") or "").strip()
+    if preset:
+        argv += ["--preset", preset]
+    else:
+        argv += ["--trunk-gb", str(getattr(cfg, "trunk_gb", 16.0) or 16.0),
+                 "--cache-gb", str(getattr(cfg, "cache_gb", 64.0) or 64.0)]
+    extra = (getattr(cfg, "extra_args", "") or "").strip()
+    if extra:
+        import shlex
+        argv += shlex.split(extra)
+    return argv
+
+
+def _build_env(cfg, ctx: int) -> dict:
+    env = os.environ.copy()
+    env["SERVE"] = "1"
+    env["K3_MAXT"] = str(int(ctx) if ctx else int(getattr(cfg, "ctx", 4096) or 4096))
+    extra_env = (getattr(cfg, "extra_env", "") or "").strip()
+    if extra_env:
+        import shlex
+        for tok in shlex.split(extra_env):
+            if "=" in tok:
+                k, v = tok.split("=", 1)
+                if k.strip():
+                    env[k.strip()] = v
+    return env
+
+
+def ensure_engine(cfg, model_dir: Optional[str] = None, ctx: Optional[int] = None,
+                  ready_timeout: float = 3600.0) -> MuxEngine:
+    """Build (as needed), then start (or reuse) the k3 engine for a checkpoint."""
+    ckpt, svc_key = resolve_service_key(cfg, model_dir)
+    with _lock:
+        eng = _services.get(svc_key)
+        if eng and eng.is_alive():
+            return eng
+        if eng and not eng.is_alive():
+            eng.close()
+            _services.pop(svc_key, None)
+
+        binary = ensure_built(cfg)
+        if not ckpt:
+            raise RuntimeError(
+                "k3: no Kimi-K3 checkpoint resolved for this request. Point the model at "
+                "the checkpoint directory (or set k3.model_path). There is no auto-"
+                "download of the ~1.56 TB checkpoint.")
+        try:
+            maxt = int(ctx) if ctx else 0
+        except (TypeError, ValueError):
+            maxt = 0
+        if maxt <= 0:
+            maxt = int(getattr(cfg, "ctx", 4096) or 4096)
+
+        argv = _build_argv(cfg, binary, ckpt)
+        env = _build_env(cfg, maxt)
+        print(f"[k3] launching engine {' '.join(argv)} (K3_MAXT={maxt})", flush=True)
+        eng = MuxEngine(binary, ckpt, max_tokens=maxt, kv_slots=1, env=env,
+                        force_mux=True, family="kimi_k3", argv=argv,
+                        inject_colibri_env=False)
+        _services[svc_key] = eng
+
+    deadline = time.time() + ready_timeout
+    while time.time() < deadline:
+        if not eng.is_alive():
+            tail = eng.log_tail()
+            stop_service(svc_key)
+            raise RuntimeError("k3 engine exited before serving"
+                               + (f". Last output: {tail}" if tail else ""))
+        print(f"[k3] engine ready for {svc_key}", flush=True)
+        return eng
+    stop_service(svc_key)
+    raise RuntimeError(f"k3 engine for {svc_key} did not become ready in time")
+
+
+def stop_service(model_id: str) -> None:
+    with _lock:
+        eng = _services.pop(model_id, None)
+    if not eng:
+        return
+    try:
+        eng.close()
+    except Exception:
+        pass
+    print(f"[k3] engine for {model_id} stopped", flush=True)
+
+
+def stop_all() -> None:
+    for mid in list(_services.keys()):
+        stop_service(mid)
+
+
+import atexit as _atexit
+_atexit.register(stop_all)

codai/api/kt_worker.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""Fully-managed ktransformers worker — the SGLang OpenAI server, driven as a subprocess.
+
+ktransformers (https://github.com/kvcache-ai/ktransformers) serves through SGLang's
+OpenAI-compatible HTTP server, so — like :mod:`codai.api.ds4_worker` — coderai launches
+``python -m sglang.launch_server`` as a managed subprocess, health-checks its
+``/v1/models`` endpoint, and the backend (:mod:`codai.backends.ktransformers`) proxies
+chat/completions to it. This module owns the process lifecycle only.
+
+Heavy dependencies (SGLang + kt-kernel) are expected to be installed out of band; with
+``auto_build`` a best-effort ``pip install`` is attempted, else a clear error is raised.
+"""
+
+import collections
+import os
+import shlex
+import socket
+import subprocess
+import sys
+import threading
+import time
+from typing import Optional
+
+_lock = threading.RLock()
+_services: dict[str, dict] = {}   # svc_key -> {"proc","port","url"}
+
+
+def _free_port() -> int:
+    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    s.bind(("127.0.0.1", 0))
+    port = s.getsockname()[1]
+    s.close()
+    return port
+
+
+def _pump_logs(proc, tail):
+    for line in proc.stdout:
+        line = line.rstrip()
+        if line:
+            tail.append(line)
+            print(f"[kt] {line}", flush=True)
+
+
+def _health_ok(url: str) -> bool:
+    import requests
+    try:
+        r = requests.get(url + "/v1/models", timeout=3)
+        return r.status_code == 200
+    except Exception:
+        return False
+
+
+def _sglang_available() -> bool:
+    import importlib.util
+    return importlib.util.find_spec("sglang") is not None
+
+
+def ensure_built(cfg) -> None:
+    """Ensure SGLang (the ktransformers serving frontend) is importable.
+
+    ktransformers' kt-kernel is a native build; coderai does not compile it per-request.
+    With ``auto_build`` we attempt a best-effort ``pip install sglang``, but the kt-kernel
+    wheel/build must be provided out of band — otherwise raise a clear, actionable error.
+    """
+    if _sglang_available():
+        return
+    if not getattr(cfg, "auto_build", False):
+        raise RuntimeError(
+            "ktransformers needs SGLang (python -m sglang.launch_server) which is not "
+            "installed, and auto_build is disabled. Install SGLang + kt-kernel out of "
+            "band (see the ktransformers docs), or enable ktransformers.auto_build.")
+    print("[kt] SGLang not found; attempting pip install (heavy) …", flush=True)
+    tail = collections.deque(maxlen=40)
+    try:
+        proc = subprocess.Popen([sys.executable, "-m", "pip", "install", "sglang"],
+                                stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,
+                                bufsize=1)
+        for line in proc.stdout:
+            line = line.rstrip()
+            if line:
+                tail.append(line)
+                print(f"[kt] {line}", flush=True)
+        proc.wait()
+    except Exception as exc:
+        raise RuntimeError(f"ktransformers: pip install sglang failed: {exc}")
+    if not _sglang_available():
+        raise RuntimeError(
+            "ktransformers: SGLang still not importable after pip install. Install "
+            "SGLang + kt-kernel manually per the ktransformers docs. Last output: "
+            + " | ".join(list(tail)[-5:]))
+
+
+def resolve_service_key(cfg, model_path: Optional[str] = None):
+    mp = os.path.expanduser((model_path or getattr(cfg, "model_path", "") or "").strip())
+    key = mp or (getattr(cfg, "model_id", "ktransformers") or "ktransformers")
+    return mp, key
+
+
+def _launch_cmd(cfg, host: str, port: int, model_path: str) -> list:
+    mid = (getattr(cfg, "model_id", "ktransformers") or "ktransformers")
+    cmd = [sys.executable, "-m", "sglang.launch_server",
+           "--host", host, "--port", str(port),
+           "--model", model_path,
+           "--served-model-name", mid]
+    ktw = os.path.expanduser((getattr(cfg, "kt_weight_path", "") or "").strip())
+    if ktw:
+        cmd += ["--kt-weight-path", ktw]
+    ctx = int(getattr(cfg, "ctx", 0) or 0)
+    if ctx > 0:
+        cmd += ["--context-length", str(ctx)]
+    extra = (getattr(cfg, "extra_args", "") or "").strip()
+    if extra:
+        cmd += shlex.split(extra)
+    return cmd
+
+
+def ensure_service(cfg, model_path: Optional[str] = None,
+                   ready_timeout: float = 3600.0) -> str:
+    """Launch (or reuse) the SGLang server for a ktransformers model; return its base URL."""
+    resolved, svc_key = resolve_service_key(cfg, model_path)
+    with _lock:
+        svc = _services.get(svc_key)
+        if svc and svc["proc"].poll() is None and _health_ok(svc["url"]):
+            return svc["url"]
+        if svc:
+            _services.pop(svc_key, None)   # died — restart below
+
+        ensure_built(cfg)
+        if not resolved or not os.path.isdir(resolved):
+            raise RuntimeError(
+                "ktransformers: no model directory resolved for this request. Point the "
+                "model at the HF model dir (or set ktransformers.model_path); also set "
+                "ktransformers.kt_weight_path. There is no auto-download.")
+
+        host = (getattr(cfg, "host", "127.0.0.1") or "127.0.0.1").strip()
+        port = int(getattr(cfg, "port", 0) or 0) or _free_port()
+        # Health/proxy over loopback even when SGLang is bound to 0.0.0.0.
+        url_host = "127.0.0.1" if host in ("0.0.0.0", "") else host
+        url = f"http://{url_host}:{port}"
+        cmd = _launch_cmd(cfg, host, port, resolved)
+
+        env = os.environ.copy()
+        applied_env = {}
+        extra_env = (getattr(cfg, "extra_env", "") or "").strip()
+        if extra_env:
+            for tok in shlex.split(extra_env):
+                if "=" in tok:
+                    k, v = tok.split("=", 1)
+                    if k.strip():
+                        env[k.strip()] = v
+                        applied_env[k.strip()] = v
+        env_note = ("  (" + " ".join(f"{k}={v}" for k, v in applied_env.items()) + ")"
+                    if applied_env else "")
+        print(f"[kt] launching SGLang: {' '.join(cmd)}{env_note}", flush=True)
+        tail = collections.deque(maxlen=60)
+        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+                                text=True, bufsize=1, env=env)
+        threading.Thread(target=_pump_logs, args=(proc, tail), daemon=True).start()
+        _services[svc_key] = {"proc": proc, "port": port, "url": url}
+
+    def _tail_msg():
+        joined = " | ".join(list(tail)[-5:]).strip()
+        return f". Last output: {joined}" if joined else ""
+
+    deadline = time.time() + ready_timeout
+    while time.time() < deadline:
+        if proc.poll() is not None:
+            stop_service(svc_key)
+            raise RuntimeError(
+                f"SGLang exited (code {proc.returncode}) before becoming ready" + _tail_msg())
+        if _health_ok(url):
+            print(f"[kt] service ready for {svc_key} at {url}", flush=True)
+            return url
+        time.sleep(2)
+    stop_service(svc_key)
+    raise RuntimeError(f"SGLang for {svc_key} did not become ready in time" + _tail_msg())
+
+
+def stop_service(model_id: str) -> None:
+    with _lock:
+        svc = _services.pop(model_id, None)
+    if not svc:
+        return
+    proc = svc["proc"]
+    if proc.poll() is None:
+        try:
+            proc.terminate()
+            proc.wait(timeout=10)
+        except Exception:
+            pass
+    if proc.poll() is None:
+        try:
+            proc.kill()
+        except Exception:
+            pass
+    print(f"[kt] service for {model_id} stopped", flush=True)
+
+
+def stop_all() -> None:
+    for mid in list(_services.keys()):
+        stop_service(mid)
+
+
+import atexit as _atexit
+_atexit.register(stop_all)

codai/api/ocr.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""Dedicated OCR endpoint.
+
+POST /v1/ocr — OCR an uploaded image or PDF with a purpose-built OCR engine
+(PaddleOCR / docTR / Surya), returning faithful text plus per-line boxes and layout.
+NOT a vision LLM. Structured-JSON extraction and stamp/signature detection are layered
+on in later milestones (fields already accepted, no-op until then).
+"""
+
+import asyncio
+from typing import List, Optional
+
+from fastapi import APIRouter, HTTPException, UploadFile, File, Form
+from pydantic import BaseModel, ConfigDict, Field
+
+from codai.ocr.base import OcrError
+from codai.ocr.manager import ocr_manager
+from codai.ocr.schemas import schema_store
+
+router = APIRouter()
+
+# Maximum upload size: 100 MB (multi-page scanned PDFs can be large).
+_MAX_OCR_BYTES = 100 * 1024 * 1024
+
+global_args = None
+
+
+def set_global_args(args):
+    """Set global args from coderai."""
+    global global_args
+    global_args = args
+
+
+def _ocr_config():
+    """Return the live OcrConfig from the admin config manager (or None)."""
+    try:
+        from codai.admin.routes import config_manager
+        if config_manager is not None and getattr(config_manager, "config", None) is not None:
+            return config_manager.config.ocr
+    except Exception:
+        pass
+    return None
+
+
+@router.post("/v1/ocr", summary="OCR an image or PDF with a dedicated OCR engine")
+async def create_ocr(
+    file: UploadFile = File(...),
+    engine: Optional[str] = Form(None),        # paddle|doctr|surya (default from config)
+    dpi: Optional[int] = Form(None),           # PDF rasterisation DPI
+    lang: Optional[str] = Form(None),          # reserved (per-engine language override)
+    structured: Optional[bool] = Form(False),  # structured JSON extraction via text model
+    schema_: Optional[str] = Form(None, alias="schema"),  # override extraction schema (JSON string)
+    detect: Optional[str] = Form(None),        # off|layout|detector|both
+):
+    cfg = _ocr_config()
+    if cfg is None:
+        raise HTTPException(status_code=503, detail="OCR subsystem is not available")
+    ocr_manager.configure(cfg)
+
+    data = await file.read()
+    if not data:
+        raise HTTPException(status_code=400, detail="empty upload")
+    if len(data) > _MAX_OCR_BYTES:
+        raise HTTPException(status_code=413, detail="file too large (max 100 MB)")
+
+    try:
+        return await ocr_manager.ocr_document(
+            data,
+            filename=file.filename or "",
+            content_type=file.content_type or "",
+            engine=engine,
+            dpi=dpi,
+            detect=detect,
+            structured=bool(structured),
+            schema=schema_,
+        )
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"OCR failed: {e}")
+
+
+@router.post("/v1/ocr/batch", summary="OCR many images/PDFs concurrently")
+async def create_ocr_batch(
+    files: List[UploadFile] = File(...),
+    engine: Optional[str] = Form(None),
+    dpi: Optional[int] = Form(None),
+    structured: Optional[bool] = Form(False),
+    schema_: Optional[str] = Form(None, alias="schema"),
+    detect: Optional[str] = Form(None),
+):
+    """Fan a batch of documents across the engine's instance pool. Per-file failures are
+    reported inline (not fatal) so one bad scan doesn't sink the batch."""
+    cfg = _ocr_config()
+    if cfg is None:
+        raise HTTPException(status_code=503, detail="OCR subsystem is not available")
+    ocr_manager.configure(cfg)
+
+    payloads = []
+    for f in files:
+        data = await f.read()
+        payloads.append((f.filename or "", f.content_type or "", data))
+
+    async def _one(name, ctype, data):
+        if not data:
+            return {"filename": name, "error": "empty upload"}
+        if len(data) > _MAX_OCR_BYTES:
+            return {"filename": name, "error": "file too large (max 100 MB)"}
+        try:
+            res = await ocr_manager.ocr_document(
+                data, filename=name, content_type=ctype, engine=engine, dpi=dpi,
+                detect=detect, structured=bool(structured), schema=schema_,
+            )
+            res["filename"] = name
+            return res
+        except OcrError as e:
+            return {"filename": name, "error": str(e), "status": e.status}
+        except Exception as e:
+            return {"filename": name, "error": f"OCR failed: {e}"}
+
+    results = await asyncio.gather(*[_one(n, c, d) for (n, c, d) in payloads])
+    ok = sum(1 for r in results if "error" not in r)
+    return {"num_files": len(results), "num_ok": ok, "results": list(results)}
+
+
+# ---------------------------------------------------------------------------
+# Extraction schema management (schemas are data, user-extensible)
+# ---------------------------------------------------------------------------
+
+class SchemaPayload(BaseModel):
+    name: Optional[str] = None
+    description: Optional[str] = ""
+    language: Optional[str] = ""
+    type: Optional[str] = "template"          # template | jsonschema
+    prompt_hint: Optional[str] = ""
+    schema_: dict = Field(default_factory=dict, alias="schema")
+    model_config = ConfigDict(extra="allow", populate_by_name=True)
+
+
+@router.get("/v1/ocr/schemas", summary="List available extraction schemas")
+async def list_schemas():
+    return {"schemas": schema_store.list()}
+
+
+@router.get("/v1/ocr/schemas/{name}", summary="Get one extraction schema")
+async def get_schema(name: str):
+    try:
+        return schema_store.get(name)
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+@router.post("/v1/ocr/schemas", summary="Create or update a user extraction schema")
+async def save_schema(payload: SchemaPayload):
+    name = payload.name
+    if not name:
+        raise HTTPException(status_code=400, detail="schema 'name' is required")
+    data = {
+        "name": name,
+        "description": payload.description or "",
+        "language": payload.language or "",
+        "type": payload.type or "template",
+        "prompt_hint": payload.prompt_hint or "",
+        "schema": getattr(payload, "schema_", None) or getattr(payload, "schema", None) or {},
+    }
+    try:
+        return schema_store.save(name, data)
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+@router.delete("/v1/ocr/schemas/{name}", summary="Delete a user extraction schema")
+async def delete_schema(name: str):
+    try:
+        schema_store.delete(name)
+        return {"deleted": name}
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))
+
+
+# ---------------------------------------------------------------------------
+# Pipeline step handler (registered as step type "ocr" in custom_pipelines)
+# ---------------------------------------------------------------------------
+
+class OcrStepRequest(BaseModel):
+    image: str = ""                       # data: URI, base64, or {{stepN.url}} reference
+    engine: Optional[str] = None
+    dpi: Optional[int] = None
+    detect: Optional[str] = None
+    structured: Optional[bool] = False
+    schema_: Optional[str] = Field(default=None, alias="schema")
+    model_config = ConfigDict(extra="allow", populate_by_name=True)
+
+
+async def run_ocr_step(request: OcrStepRequest, http_request=None):
+    """Pipeline step: OCR an image reference and return {text, structured, ...}."""
+    import base64
+
+    cfg = _ocr_config()
+    if cfg is None:
+        raise HTTPException(status_code=503, detail="OCR subsystem is not available")
+    ocr_manager.configure(cfg)
+
+    ref = (getattr(request, "image", "") or "").strip()
+    filename = "input.png"
+    if ref.startswith("data:"):
+        header, encoded = ref.split(",", 1)
+        data = base64.b64decode(encoded)
+        if "application/pdf" in header:
+            filename = "input.pdf"
+    elif ref.startswith("http://") or ref.startswith("https://"):
+        import urllib.request
+        with urllib.request.urlopen(ref) as resp:      # nosec - user-provided pipeline input
+            data = resp.read()
+        filename = ref.rsplit("/", 1)[-1] or filename
+    else:
+        data = base64.b64decode(ref) if ref else b""
+
+    if not data:
+        raise HTTPException(status_code=400, detail="ocr step: no image provided")
+
+    schema = getattr(request, "schema_", None) or getattr(request, "schema", None)
+    try:
+        return await ocr_manager.ocr_document(
+            data, filename=filename, engine=request.engine, dpi=request.dpi,
+            detect=request.detect, structured=bool(request.structured), schema=schema,
+        )
+    except OcrError as e:
+        raise HTTPException(status_code=e.status, detail=str(e))

codai/backends/colibri.py

@@ -26,6 +26,7 @@ import threading
 from typing import AsyncGenerator, Dict, List, Optional
 
 from codai.backends.base import ModelBackend
+from codai.backends.colibri_families import render_deepseek_chat, build_kimi_wire
 
 # GLM-5.2 chat-template markers (from colibri openai_server.py — the model expresses
 # tool calls as ordinary text, so we render them into the prompt and let the parser
@@ -62,6 +63,70 @@ def clean_glm_output(text: str) -> str:
     return text
 
 
+# --- DeepSeek-V4 / Kimi-K3 output trimming (multi-family colibri, v1.5.0) -------- #
+# DeepSeek uses the official markers (｜=U+FF5C, ▁=U+2581); Kimi emits XTML specials
+# that its serve loop already suppresses, but we strip any residual defensively.
+_DS_STOP_MARKERS = ("<｜end▁of▁sentence｜>", "<｜User｜>", "<｜Assistant｜>",
+                    "<｜begin▁of▁sentence｜>")
+_KIMI_STRIP_TOKENS = ("<|open|>", "<|close|>", "<|sep|>", "<|end_of_msg|>")
+
+
+def _ds_cut_index(text: str) -> int:
+    cut = len(text)
+    for m in _DS_STOP_MARKERS:
+        i = text.find(m)
+        if i != -1 and i < cut:
+            cut = i
+    return cut
+
+
+def clean_deepseek_output(text: str) -> str:
+    """Trim a DeepSeek-V4 completion at its turn boundary."""
+    if not text:
+        return text
+    text = text[:_ds_cut_index(text)]
+    return text
+
+
+def clean_kimi_output(text: str) -> str:
+    """Strip any residual Kimi-K3 XTML control tokens from the visible content (the
+    engine's serve loop already suppresses structural runs, so this is defensive)."""
+    if not text:
+        return text
+    for t in _KIMI_STRIP_TOKENS:
+        text = text.replace(t, "")
+    return text
+
+
+# Family → (payload builder, output cleaner, turn-boundary cut). The payload builder
+# takes (messages, enable_thinking, tools) and returns str (GLM/DeepSeek rendered
+# prompt) or bytes (Kimi K3CHAT1 wire). See project_colibri_v15_families.
+def _payload_for(family: str, messages, enable_thinking: bool, tools):
+    if family == "deepseek_v4":
+        return render_deepseek_chat(messages, enable_thinking=enable_thinking)
+    if family == "kimi_k3":
+        return build_kimi_wire(messages, enable_thinking=enable_thinking)
+    return render_chat(messages, enable_thinking=enable_thinking, tools=tools)
+
+
+def _clean_for(family: str, text: str) -> str:
+    if family == "deepseek_v4":
+        return clean_deepseek_output(text)
+    if family == "kimi_k3":
+        return clean_kimi_output(text)
+    return clean_glm_output(text)
+
+
+def _cut_index_for(family: str, text: str) -> int:
+    if family == "deepseek_v4":
+        return _ds_cut_index(text)
+    if family == "kimi_k3":
+        # The Kimi engine emits already-trimmed visible text and stops on its own EOS;
+        # no extra server-side turn cut is needed.
+        return len(text)
+    return _glm_cut_index(text)
+
+
 def _content_text(content) -> str:
     """Flatten OpenAI message content (string or list of text parts) to a string."""
     if content is None:
@@ -401,7 +466,8 @@ class ColibriBackend(ModelBackend):
         self._enter_request()
         try:
             engine = self._need_engine()
-            prompt = render_chat(messages, enable_thinking=self._enable_thinking, tools=tools)
+            family = getattr(engine, "family", "glm")
+            prompt = _payload_for(family, messages, self._enable_thinking, tools)
             chunks: List[str] = []
 
             def _hit_turn_boundary():
@@ -409,13 +475,13 @@ class ColibriBackend(ModelBackend):
                 # ramble into a new turn. Stop it the moment a turn marker appears —
                 # a big saving at streaming-bound decode speeds.
                 joined = "".join(chunks)
-                return _glm_cut_index(joined) < len(joined)
+                return _cut_index_for(family, joined) < len(joined)
 
             stats = engine.run(prompt, int(max_tokens or 1024), float(temperature),
                                float(top_p), on_text=chunks.append,
                                cancelled=_hit_turn_boundary)
             self._store_usage(stats)
-            return clean_glm_output("".join(chunks))
+            return _clean_for(family, "".join(chunks))
         finally:
             self._exit_request()
 
@@ -425,9 +491,10 @@ class ColibriBackend(ModelBackend):
         self._enter_request()
         try:
             engine = self._need_engine()
-            prompt = render_chat(messages, enable_thinking=self._enable_thinking, tools=tools)
+            family = getattr(engine, "family", "glm")
+            prompt = _payload_for(family, messages, self._enable_thinking, tools)
             async for chunk in self._stream(engine, prompt, int(max_tokens or 1024),
-                                            float(temperature), float(top_p)):
+                                            float(temperature), float(top_p), family=family):
                 yield chunk
         finally:
             self._exit_request()
@@ -453,8 +520,8 @@ class ColibriBackend(ModelBackend):
     # SSE streaming: the engine's blocking run() streams tokens to a callback on a
     # worker thread; bridge them to the event loop through an asyncio.Queue.
     # ------------------------------------------------------------------ #
-    async def _stream(self, engine, prompt: str, max_tokens: int, temperature: float,
-                      top_p: float) -> AsyncGenerator[str, None]:
+    async def _stream(self, engine, prompt, max_tokens: int, temperature: float,
+                      top_p: float, family: str = "glm") -> AsyncGenerator[str, None]:
         loop = asyncio.get_event_loop()
         out_queue: asyncio.Queue = asyncio.Queue()
         _SENTINEL = object()
@@ -462,7 +529,7 @@ class ColibriBackend(ModelBackend):
 
         def _hit_turn_boundary():
             joined = "".join(raw)
-            return _glm_cut_index(joined) < len(joined)
+            return _cut_index_for(family, joined) < len(joined)
 
         def _on_text(text: str):
             if text:
@@ -480,9 +547,9 @@ class ColibriBackend(ModelBackend):
                 loop.call_soon_threadsafe(out_queue.put_nowait, _SENTINEL)
 
         threading.Thread(target=_worker, daemon=True).start()
-        # Emit the CLEANED text incrementally. Hold back the last few chars so a GLM
-        # control token split across chunks (e.g. "<th"+"ink>") is never emitted raw;
-        # clean_glm_output also cuts at the turn boundary, so the stream ends there.
+        # Emit the CLEANED text incrementally. Hold back the last few chars so a control
+        # token split across chunks (e.g. "<th"+"ink>") is never emitted raw; the
+        # per-family cleaner also cuts at the turn boundary, so the stream ends there.
         _HOLD = 16
         yielded = 0
         while True:
@@ -491,11 +558,11 @@ class ColibriBackend(ModelBackend):
                 break
             if isinstance(item, Exception):
                 raise item
-            cleaned = clean_glm_output("".join(raw))
+            cleaned = _clean_for(family, "".join(raw))
             safe = max(yielded, len(cleaned) - _HOLD)
             if safe > yielded:
                 yield cleaned[yielded:safe]
                 yielded = safe
-        cleaned = clean_glm_output("".join(raw))
+        cleaned = _clean_for(family, "".join(raw))
         if len(cleaned) > yielded:
             yield cleaned[yielded:]

codai/backends/colibri_families.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""Per-family prompt builders for the multi-family colibri engine (v1.5.0).
+
+colibri v1.5.0 ships one C engine per model family, and each expects a DIFFERENT
+serve payload even though they share the ``DATA``/``DONE`` mux framing (see the
+project memory ``project_colibri_v15_families``):
+
+* **GLM-5.2** (``colibri.c``) — a fully rendered XTML string. Lives in
+  :mod:`codai.backends.colibri` (:func:`render_chat`), unchanged.
+* **DeepSeek-V4** (``deepseek_v4.c``) — a fully rendered prompt string using the
+  official DeepSeek markers. Built here by :func:`render_deepseek_chat`, byte-matching
+  the engine's ``coli_v4_prompt_build`` for the system+user case.
+* **Kimi-K3** (``kimi_k3.c``) — NOT a rendered string: a length-framed ``K3CHAT1``
+  wire blob that the engine turns into its XTML template itself. Built here by
+  :func:`build_kimi_wire`, byte-matching the engine's ``chat_build_wire`` parser
+  (fixture ``c/tests/fixtures/kimi_chat_wire.txt``).
+
+The family is chosen by the backend from the container's ``config.json``; this module
+only owns the payload byte-format.
+"""
+
+import json
+from typing import Dict, List, Optional
+
+# --- DeepSeek-V4 markers (deepseek_v4.c: v4_bos/v4_user/v4_assistant) ----------- #
+# NB: the bars are U+FF5C (｜) and U+2581 (▁), NOT ASCII '|' / '_'.
+DS_BOS = "<｜begin▁of▁sentence｜>"
+DS_EOS = "<｜end▁of▁sentence｜>"
+DS_USER = "<｜User｜>"
+DS_ASSISTANT = "<｜Assistant｜>"
+
+
+def _content_text(content) -> str:
+    """Flatten OpenAI message content (string or list of text parts) to a string.
+
+    Kept in sync with :func:`codai.backends.colibri._content_text`.
+    """
+    if content is None:
+        return ""
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts = []
+        for part in content:
+            if isinstance(part, dict) and part.get("type") in ("text", "input_text"):
+                t = part.get("text")
+                if isinstance(t, str):
+                    parts.append(t)
+        return "".join(parts)
+    return str(content)
+
+
+def _assistant_reasoning(message: Dict) -> str:
+    """Return an assistant turn's separate reasoning text, if the client sent one.
+
+    OpenAI-compatible clients surface chain-of-thought under a few keys; we accept the
+    common ones. Empty string when the turn carries only visible content.
+    """
+    for key in ("reasoning_content", "reasoning"):
+        v = message.get(key)
+        if isinstance(v, str) and v:
+            return v
+    return ""
+
+
+# --------------------------------------------------------------------------- #
+# DeepSeek-V4 — rendered prompt string
+# --------------------------------------------------------------------------- #
+def render_deepseek_chat(messages: List[Dict], enable_thinking: bool = False) -> str:
+    """Render the DeepSeek-V4 chat prompt colibri's ``deepseek_v4`` engine expects.
+
+    For a single system+user turn this byte-matches the engine's own
+    ``coli_v4_prompt_build`` (``{bos}{system}{User}{user}{Assistant}`` + ``</think>``
+    for non-thinking or ``<think>`` for thinking). Multi-turn history follows the
+    official DeepSeek template: assistant turns are closed with the end-of-sentence
+    marker, and the trailing ``<｜Assistant｜>`` opens the turn to generate.
+    """
+    if not isinstance(messages, list) or not messages:
+        raise ValueError("`messages` must be a non-empty array.")
+    # System text is concatenated and placed right after BOS (matches the C helper's
+    # single leading-system layout).
+    system = "".join(_content_text(m.get("content")) for m in messages
+                     if isinstance(m, dict) and m.get("role") in ("system", "developer"))
+    out = [DS_BOS, system]
+    for m in messages:
+        if not isinstance(m, dict):
+            raise ValueError("Each message must be an object.")
+        role = m.get("role")
+        if role in ("system", "developer"):
+            continue
+        if role == "user":
+            out.append(DS_USER + _content_text(m.get("content")))
+        elif role == "assistant":
+            out.append(DS_ASSISTANT + _content_text(m.get("content")) + DS_EOS)
+        elif role == "tool":
+            # DeepSeek has no distinct tool turn in this template; fold tool output
+            # into the user channel so the model still sees it.
+            out.append(DS_USER + _content_text(m.get("content")))
+        else:
+            raise ValueError(f"Unsupported message role: {role!r}.")
+    out.append(DS_ASSISTANT + ("<think>" if enable_thinking else "</think>"))
+    return "".join(out)
+
+
+# --------------------------------------------------------------------------- #
+# Kimi-K3 — K3CHAT1 length-framed wire blob
+# --------------------------------------------------------------------------- #
+def _wire_msg(role: str, text: str) -> bytes:
+    b = text.encode("utf-8")
+    return (f"M {role} {len(b)}\n").encode("utf-8") + b
+
+
+def _wire_assistant(reason: str, text: str) -> bytes:
+    rb = reason.encode("utf-8")
+    tb = text.encode("utf-8")
+    return (f"A {len(rb)} {len(tb)}\n").encode("utf-8") + rb + tb
+
+
+# --------------------------------------------------------------------------- #
+# Kimi-K3 — rendered XTML string (for engines that tokenize a prompt string, e.g.
+# the patched FareedKhan-dev/kimi-k3-in-c, as opposed to colibri's K3CHAT1 wire).
+# Byte-reproduces colibri kimi_k3.c chat_build/chat_message/chat_assistant output.
+# --------------------------------------------------------------------------- #
+KIMI_OPEN, KIMI_CLOSE, KIMI_SEP, KIMI_EOM = "<|open|>", "<|close|>", "<|sep|>", "<|end_of_msg|>"
+
+
+def _k_open(tag: str, role: Optional[str] = None) -> str:
+    s = KIMI_OPEN + tag
+    if role is not None:
+        s += ' role="' + role + '"'
+    return s + KIMI_SEP
+
+
+def _k_close(tag: str) -> str:
+    return KIMI_CLOSE + tag + KIMI_SEP
+
+
+def render_kimi_xtml(messages: List[Dict], enable_thinking: bool = True) -> str:
+    """Render the Kimi-K3 XTML chat prompt as a STRING (the engine tokenizes it).
+
+    Mirrors colibri's ``kimi_k3.c`` builders: system/user/tool turns render as
+    ``<|open|>message role="…"<|sep|>TEXT<|close|>message<|sep|><|end_of_msg|>``;
+    assistant turns wrap the reply in ``<|open|>response<|sep|>…<|close|>response<|sep|>``
+    (with a leading ``<|open|>think<|sep|>…<|close|>think<|sep|>`` when the turn carried
+    reasoning); and the prompt ends by opening the assistant message plus its structural
+    think/response channel to generate. The four XTML control tokens must exist in the
+    engine's tokenizer.
+    """
+    if not isinstance(messages, list) or not messages:
+        raise ValueError("`messages` must be a non-empty array.")
+    out: List[str] = []
+    for m in messages:
+        if not isinstance(m, dict):
+            raise ValueError("Each message must be an object.")
+        role = m.get("role")
+        text = _content_text(m.get("content"))
+        if role in ("system", "developer"):
+            out.append(_k_open("message", "system") + text + _k_close("message") + KIMI_EOM)
+        elif role == "user":
+            out.append(_k_open("message", "user") + text + _k_close("message") + KIMI_EOM)
+        elif role == "tool":
+            out.append(_k_open("message", "user") + text + _k_close("message") + KIMI_EOM)
+        elif role == "assistant":
+            reason = _assistant_reasoning(m)
+            s = _k_open("message", "assistant")
+            if reason:
+                s += _k_open("think") + reason + _k_close("think")
+            s += _k_open("response") + text + _k_close("response")
+            s += _k_close("message") + KIMI_EOM
+            out.append(s)
+        else:
+            raise ValueError(f"Unsupported message role: {role!r}.")
+    out.append(_k_open("message", "assistant")
+               + _k_open("think" if enable_thinking else "response"))
+    return "".join(out)
+
+
+def build_kimi_wire(messages: List[Dict], enable_thinking: bool = True) -> bytes:
+    """Build the Kimi-K3 ``K3CHAT1`` wire payload for colibri's ``kimi_k3`` engine.
+
+    Byte-matches the engine's ``chat_build_wire`` parser: a ``K3CHAT1`` header, then
+    one framed directive per history message —
+
+      * ``M <role> <nbytes>\\n<content>``   (system / user / assistant-without-reasoning;
+        ``developer`` is normalised to ``system`` by the engine, but we send it as-is
+        which the engine maps)
+      * ``A <nreason> <ntext>\\n<reason><text>``  (assistant turn carrying reasoning)
+
+    and a terminal ``G <0|1>\\n`` that sets the thinking channel and opens the
+    assistant turn to generate. ``nbytes`` are exact UTF-8 byte counts.
+
+    Returns raw ``bytes`` (the payload is length-framed, so the caller must send these
+    bytes verbatim — do not re-encode).
+    """
+    if not isinstance(messages, list) or not messages:
+        raise ValueError("`messages` must be a non-empty array.")
+    out = [b"K3CHAT1\n"]
+    for m in messages:
+        if not isinstance(m, dict):
+            raise ValueError("Each message must be an object.")
+        role = m.get("role")
+        if role in ("system", "developer", "user"):
+            out.append(_wire_msg("system" if role == "developer" else role,
+                                 _content_text(m.get("content"))))
+        elif role == "assistant":
+            reason = _assistant_reasoning(m)
+            text = _content_text(m.get("content"))
+            if reason:
+                out.append(_wire_assistant(reason, text))
+            else:
+                out.append(_wire_msg("assistant", text))
+        elif role == "tool":
+            # No Kimi tool turn in the wire format; surface tool output as a user msg.
+            out.append(_wire_msg("user", _content_text(m.get("content"))))
+        else:
+            raise ValueError(f"Unsupported message role: {role!r}.")
+    out.append((f"G {1 if enable_thinking else 0}\n").encode("utf-8"))
+    return b"".join(out)

codai/backends/k3.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""Kimi-K3 backend — the kimi-k3-in-c C engine, driven in-process.
+
+Mirrors :class:`~codai.backends.colibri.ColibriBackend` but for a single family
+(Kimi-K3) served by the patched ``kimi-k3-in-c`` engine. The engine tokenizes a
+rendered XTML prompt string (unlike colibri's Kimi ``K3CHAT1`` wire), so this backend
+renders with :func:`~codai.backends.colibri_families.render_kimi_xtml` and drives the
+engine through :class:`~codai.api.colibri_worker.MuxEngine` (managed by
+:mod:`codai.api.k3_worker`). Tool/think parsing is handled the usual way over the
+returned text.
+"""
+
+import asyncio
+import threading
+from typing import AsyncGenerator, Dict, List, Optional
+
+from codai.backends.base import ModelBackend
+from codai.backends.colibri_families import render_kimi_xtml
+
+# End-of-turn / structural markers to trim from the visible reply.
+_K3_STOP_MARKERS = ("<|close|>response", "<|close|>message", "<|close|>",
+                    "<|end_of_msg|>", "<|open|>")
+_K3_STRIP_TOKENS = ("<|open|>", "<|close|>", "<|sep|>", "<|end_of_msg|>")
+
+
+def _k3_cut_index(text: str) -> int:
+    cut = len(text)
+    for m in _K3_STOP_MARKERS:
+        i = text.find(m)
+        if i != -1 and i < cut:
+            cut = i
+    return cut
+
+
+def clean_k3_output(text: str) -> str:
+    """Trim a Kimi-K3 completion at its response-channel boundary and strip XTML tokens.
+
+    With the backend's default non-thinking prompt the engine generates the response
+    directly, so the visible stream is the answer up to ``<|close|>response`` /
+    ``<|end_of_msg|>``; any residual XTML control tokens are removed defensively.
+    """
+    if not text:
+        return text
+    text = text[:_k3_cut_index(text)]
+    for t in _K3_STRIP_TOKENS:
+        text = text.replace(t, "")
+    return text
+
+
+class K3Backend(ModelBackend):
+    """In-process backend that drives a managed kimi-k3-in-c engine (Kimi-K3)."""
+
+    _inflight = 0
+    _inflight_lock = threading.Lock()
+
+    @classmethod
+    def _enter_request(cls):
+        with cls._inflight_lock:
+            cls._inflight += 1
+
+    @classmethod
+    def _exit_request(cls):
+        with cls._inflight_lock:
+            cls._inflight = max(0, cls._inflight - 1)
+
+    @classmethod
+    def any_request_active(cls) -> bool:
+        with cls._inflight_lock:
+            return cls._inflight > 0
+
+    def __init__(self, cfg=None):
+        if cfg is None:
+            from codai.config import K3Config
+            cfg = K3Config()
+        self._cfg = cfg
+        self._model_id = getattr(cfg, "model_id", "kimi-k3") or "kimi-k3"
+        self._svc_key: Optional[str] = None
+        self._engine = None
+        self._ctx = int(getattr(cfg, "ctx", 4096) or 4096)
+        # Default to the response-only channel so the visible stream is the answer, not
+        # the structural think channel (kimi-k3-in-c serve emits raw token text).
+        self._enable_thinking = False
+        self._last_usage: Dict = {}
+
+    # ------------------------------------------------------------------ #
+    # lifecycle
+    # ------------------------------------------------------------------ #
+    def load_model(self, model_name: str, **kwargs) -> None:
+        from codai.api import k3_worker
+        if model_name:
+            self._model_id = model_name
+        _ctx = kwargs.get("n_ctx", kwargs.get("ctx"))
+        if isinstance(_ctx, (list, tuple)):
+            _ctx = _ctx[0] if _ctx else None
+        try:
+            _ctx = int(_ctx) if _ctx else 0
+        except (TypeError, ValueError):
+            _ctx = 0
+        if _ctx > 0:
+            self._ctx = _ctx
+        model_dir = self._resolve_checkpoint(model_name)
+        _resolved, self._svc_key = k3_worker.resolve_service_key(self._cfg, model_dir)
+        self._engine = k3_worker.ensure_engine(
+            self._cfg, model_dir=model_dir, ctx=(self._ctx or None))
+
+    def _resolve_checkpoint(self, model_name: str) -> Optional[str]:
+        """Map the requested model name/alias/path to its Kimi-K3 checkpoint directory."""
+        import os
+        cand = os.path.expanduser(model_name or "")
+        if cand and os.path.isdir(cand):
+            return os.path.abspath(cand)
+        mp = os.path.expanduser((getattr(self._cfg, "model_path", "") or "").strip())
+        if mp and os.path.isdir(mp):
+            return os.path.abspath(mp)
+        # A models.json entry's path for this name/alias.
+        try:
+            from codai.admin.routes import config_manager
+            md = getattr(config_manager, "models_data", {}) or {}
+            name_l = (model_name or "").strip().lower()
+            for lst in md.values():
+                if not isinstance(lst, list):
+                    continue
+                for m in lst:
+                    if not isinstance(m, dict):
+                        continue
+                    path = str(m.get("path") or "")
+                    base = os.path.basename(path.rstrip("/")).lower()
+                    cands = {path.lower(), base, str(m.get("alias") or "").lower(),
+                             str(m.get("id") or "").lower()}
+                    if name_l in cands and path and os.path.isdir(os.path.expanduser(path)):
+                        return os.path.abspath(os.path.expanduser(path))
+        except Exception:
+            pass
+        return None
+
+    def get_model_name(self) -> str:
+        return self._model_id
+
+    def get_context_size(self) -> int:
+        return self._ctx
+
+    def get_last_usage(self) -> dict:
+        return dict(self._last_usage)
+
+    def cleanup(self) -> None:
+        from codai.api import k3_worker
+        key = getattr(self, "_svc_key", None) or getattr(self._cfg, "model_id", self._model_id)
+        k3_worker.stop_service(key)
+        self._engine = None
+
+    # ------------------------------------------------------------------ #
+    # helpers
+    # ------------------------------------------------------------------ #
+    def _need_engine(self):
+        if self._engine is None or not self._engine.is_alive():
+            self.load_model(self._model_id)
+        return self._engine
+
+    def _store_usage(self, stats: dict) -> None:
+        if stats:
+            pt = int(stats.get("prompt_tokens", 0) or 0)
+            ct = int(stats.get("completion_tokens", 0) or 0)
+            self._last_usage = {"prompt_tokens": pt, "completion_tokens": ct,
+                                "total_tokens": pt + ct}
+
+    def format_messages(self, messages) -> str:
+        return render_kimi_xtml(messages, enable_thinking=self._enable_thinking)
+
+    # ------------------------------------------------------------------ #
+    # chat-level generation
+    # ------------------------------------------------------------------ #
+    def generate_chat(self, messages: List[Dict], max_tokens=None, temperature=0.7,
+                      top_p=1.0, stop=None, tools=None, response_format=None):
+        self._enter_request()
+        try:
+            engine = self._need_engine()
+            prompt = render_kimi_xtml(messages, enable_thinking=self._enable_thinking)
+            chunks: List[str] = []
+
+            def _hit_turn_boundary():
+                joined = "".join(chunks)
+                return _k3_cut_index(joined) < len(joined)
+
+            stats = engine.run(prompt, int(max_tokens or 1024), float(temperature),
+                               float(top_p), on_text=chunks.append,
+                               cancelled=_hit_turn_boundary)
+            self._store_usage(stats)
+            return clean_k3_output("".join(chunks))
+        finally:
+            self._exit_request()
+
+    async def generate_chat_stream(self, messages: List[Dict], max_tokens=None,
+                                   temperature=0.7, top_p=1.0, stop=None, tools=None,
+                                   response_format=None) -> AsyncGenerator[str, None]:
+        self._enter_request()
+        try:
+            engine = self._need_engine()
+            prompt = render_kimi_xtml(messages, enable_thinking=self._enable_thinking)
+            async for chunk in self._stream(engine, prompt, int(max_tokens or 1024),
+                                            float(temperature), float(top_p)):
+                yield chunk
+        finally:
+            self._exit_request()
+
+    # ------------------------------------------------------------------ #
+    # plain completion (fallback path)
+    # ------------------------------------------------------------------ #
+    def generate(self, prompt: str, max_tokens=None, temperature: float = 0.7,
+                 top_p: float = 1.0, stop=None, repeat_penalty: float = 1.0,
+                 presence_penalty: float = 0.0, frequency_penalty: float = 0.0) -> str:
+        return self.generate_chat([{"role": "user", "content": prompt}],
+                                  max_tokens, temperature, top_p, stop)
+
+    async def generate_stream(self, prompt: str, max_tokens=None, temperature: float = 0.7,
+                              top_p: float = 1.0, stop=None, repeat_penalty: float = 1.0,
+                              presence_penalty: float = 0.0,
+                              frequency_penalty: float = 0.0) -> AsyncGenerator[str, None]:
+        async for chunk in self.generate_chat_stream(
+                [{"role": "user", "content": prompt}], max_tokens, temperature, top_p, stop):
+            yield chunk
+
+    # ------------------------------------------------------------------ #
+    # SSE streaming (mirrors ColibriBackend._stream)
+    # ------------------------------------------------------------------ #
+    async def _stream(self, engine, prompt: str, max_tokens: int, temperature: float,
+                      top_p: float) -> AsyncGenerator[str, None]:
+        loop = asyncio.get_event_loop()
+        out_queue: asyncio.Queue = asyncio.Queue()
+        _SENTINEL = object()
+        raw: List[str] = []
+
+        def _hit_turn_boundary():
+            joined = "".join(raw)
+            return _k3_cut_index(joined) < len(joined)
+
+        def _on_text(text: str):
+            if text:
+                raw.append(text)
+                loop.call_soon_threadsafe(out_queue.put_nowait, True)
+
+        def _worker():
+            try:
+                stats = engine.run(prompt, max_tokens, temperature, top_p,
+                                   on_text=_on_text, cancelled=_hit_turn_boundary)
+                self._store_usage(stats)
+            except Exception as exc:
+                loop.call_soon_threadsafe(out_queue.put_nowait, exc)
+            finally:
+                loop.call_soon_threadsafe(out_queue.put_nowait, _SENTINEL)
+
+        threading.Thread(target=_worker, daemon=True).start()
+        _HOLD = 16
+        yielded = 0
+        while True:
+            item = await out_queue.get()
+            if item is _SENTINEL:
+                break
+            if isinstance(item, Exception):
+                raise item
+            cleaned = clean_k3_output("".join(raw))
+            safe = max(yielded, len(cleaned) - _HOLD)
+            if safe > yielded:
+                yield cleaned[yielded:safe]
+                yielded = safe
+        cleaned = clean_k3_output("".join(raw))
+        if len(cleaned) > yielded:
+            yield cleaned[yielded:]

codai/backends/ktransformers.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+"""ktransformers proxy backend.
+
+ktransformers serves through SGLang's OpenAI HTTP API, so this backend is a thin proxy
+(mirroring :class:`~codai.backends.ds4.Ds4Backend`): it forwards chat/completion requests
+to the managed SGLang subprocess (lifecycle in :mod:`codai.api.kt_worker`), which applies
+the model's own chat template server-side.
+"""
+
+import asyncio
+import threading
+from typing import AsyncGenerator, Dict, List, Optional
+
+from codai.backends.base import ModelBackend
+
+
+class KtransformersBackend(ModelBackend):
+    """Thin OpenAI proxy to a managed ktransformers/SGLang server."""
+
+    _inflight = 0
+    _inflight_lock = threading.Lock()
+
+    @classmethod
+    def _enter_request(cls):
+        with cls._inflight_lock:
+            cls._inflight += 1
+
+    @classmethod
+    def _exit_request(cls):
+        with cls._inflight_lock:
+            cls._inflight = max(0, cls._inflight - 1)
+
+    @classmethod
+    def any_request_active(cls) -> bool:
+        with cls._inflight_lock:
+            return cls._inflight > 0
+
+    def __init__(self, cfg=None):
+        if cfg is None:
+            from codai.config import KtransformersConfig
+            cfg = KtransformersConfig()
+        self._cfg = cfg
+        self._model_id = getattr(cfg, "model_id", "ktransformers") or "ktransformers"
+        self._svc_key: Optional[str] = None
+        self._url: Optional[str] = None
+        self._ctx = int(getattr(cfg, "ctx", 32768) or 32768)
+        self._last_usage: Dict = {}
+
+    # ------------------------------------------------------------------ #
+    # lifecycle
+    # ------------------------------------------------------------------ #
+    def load_model(self, model_name: str, **kwargs) -> None:
+        from codai.api import kt_worker
+        if model_name:
+            self._model_id = model_name
+        _ctx = kwargs.get("n_ctx", kwargs.get("ctx"))
+        if isinstance(_ctx, (list, tuple)):
+            _ctx = _ctx[0] if _ctx else None
+        try:
+            _ctx = int(_ctx) if _ctx else 0
+        except (TypeError, ValueError):
+            _ctx = 0
+        if _ctx > 0:
+            self._ctx = _ctx
+        model_path = self._resolve_model_dir(model_name)
+        _resolved, self._svc_key = kt_worker.resolve_service_key(self._cfg, model_path)
+        self._url = kt_worker.ensure_service(self._cfg, model_path=model_path)
+        # SGLang serves under --served-model-name = the configured model_id; send that.
+        self._served_name = getattr(self._cfg, "model_id", "ktransformers") or "ktransformers"
+
+    def _resolve_model_dir(self, model_name: str) -> Optional[str]:
+        """Resolve the requested model to its HF model directory for SGLang."""
+        import os
+        cand = os.path.expanduser(model_name or "")
+        if cand and os.path.isdir(cand):
+            return os.path.abspath(cand)
+        mp = os.path.expanduser((getattr(self._cfg, "model_path", "") or "").strip())
+        if mp and os.path.isdir(mp):
+            return os.path.abspath(mp)
+        try:
+            from codai.admin.routes import config_manager
+            md = getattr(config_manager, "models_data", {}) or {}
+            name_l = (model_name or "").strip().lower()
+            for lst in md.values():
+                if not isinstance(lst, list):
+                    continue
+                for m in lst:
+                    if not isinstance(m, dict):
+                        continue
+                    path = str(m.get("path") or "")
+                    base = os.path.basename(path.rstrip("/")).lower()
+                    cands = {path.lower(), base, str(m.get("alias") or "").lower(),
+                             str(m.get("id") or "").lower()}
+                    if name_l in cands and path and os.path.isdir(os.path.expanduser(path)):
+                        return os.path.abspath(os.path.expanduser(path))
+        except Exception:
+            pass
+        return mp or None
+
+    def get_model_name(self) -> str:
+        return self._model_id
+
+    def get_context_size(self) -> int:
+        return self._ctx
+
+    def get_last_usage(self) -> dict:
+        return dict(self._last_usage)
+
+    def cleanup(self) -> None:
+        from codai.api import kt_worker
+        key = getattr(self, "_svc_key", None) or getattr(self._cfg, "model_id", self._model_id)
+        kt_worker.stop_service(key)
+        self._url = None
+
+    # ------------------------------------------------------------------ #
+    # helpers
+    # ------------------------------------------------------------------ #
+    def _base(self) -> str:
+        if not self._url:
+            raise RuntimeError("ktransformers service not started")
+        return self._url
+
+    def _store_usage(self, usage: dict) -> None:
+        if usage:
+            self._last_usage = {
+                "prompt_tokens": usage.get("prompt_tokens", 0),
+                "completion_tokens": usage.get("completion_tokens", 0),
+                "total_tokens": usage.get("total_tokens", 0),
+            }
+
+    def format_messages(self, messages) -> str:
+        parts = []
+        for m in messages:
+            role = m.get("role") if isinstance(m, dict) else getattr(m, "role", "")
+            content = m.get("content") if isinstance(m, dict) else getattr(m, "content", "")
+            parts.append(f"{role}: {content}")
+        return "\n".join(parts)
+
+    def _chat_payload(self, messages, max_tokens, temperature, top_p, stop, stream):
+        payload = {
+            "model": getattr(self, "_served_name", self._model_id),
+            "messages": messages,
+            "temperature": temperature,
+            "top_p": top_p,
+            "stream": stream,
+        }
+        if max_tokens:
+            payload["max_tokens"] = max_tokens
+        if stop:
+            payload["stop"] = stop
+        return payload
+
+    # ------------------------------------------------------------------ #
+    # chat-level generation
+    # ------------------------------------------------------------------ #
+    def generate_chat(self, messages: List[Dict], max_tokens=None, temperature=0.7,
+                      top_p=1.0, stop=None, tools=None, response_format=None):
+        import requests
+        self._enter_request()
+        try:
+            payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, False)
+            if response_format and response_format.get("type") == "json_object":
+                payload["response_format"] = {"type": "json_object"}
+            if tools:
+                payload["tools"] = tools
+            r = requests.post(self._base() + "/v1/chat/completions", json=payload, timeout=3600)
+            r.raise_for_status()
+            data = r.json()
+            self._store_usage(data.get("usage", {}))
+            return data["choices"][0]["message"].get("content") or ""
+        finally:
+            self._exit_request()
+
+    async def generate_chat_stream(self, messages: List[Dict], max_tokens=None,
+                                   temperature=0.7, top_p=1.0, stop=None, tools=None,
+                                   response_format=None) -> AsyncGenerator[str, None]:
+        self._enter_request()
+        try:
+            payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, True)
+            if tools:
+                payload["tools"] = tools
+            async for chunk in self._stream(self._base() + "/v1/chat/completions", payload,
+                                            delta_key="delta"):
+                yield chunk
+        finally:
+            self._exit_request()
+
+    # ------------------------------------------------------------------ #
+    # plain completion (fallback path)
+    # ------------------------------------------------------------------ #
+    def generate(self, prompt: str, max_tokens=None, temperature: float = 0.7,
+                 top_p: float = 1.0, stop=None, repeat_penalty: float = 1.0,
+                 presence_penalty: float = 0.0, frequency_penalty: float = 0.0) -> str:
+        return self.generate_chat([{"role": "user", "content": prompt}],
+                                  max_tokens, temperature, top_p, stop)
+
+    async def generate_stream(self, prompt: str, max_tokens=None, temperature: float = 0.7,
+                              top_p: float = 1.0, stop=None, repeat_penalty: float = 1.0,
+                              presence_penalty: float = 0.0,
+                              frequency_penalty: float = 0.0) -> AsyncGenerator[str, None]:
+        async for chunk in self.generate_chat_stream(
+                [{"role": "user", "content": prompt}], max_tokens, temperature, top_p, stop):
+            yield chunk
+
+    # ------------------------------------------------------------------ #
+    # SSE streaming: iterate the blocking requests stream on a worker thread
+    # and hand chunks to the event loop through an asyncio.Queue.
+    # ------------------------------------------------------------------ #
+    async def _stream(self, url: str, payload: dict, delta_key: str
+                      ) -> AsyncGenerator[str, None]:
+        import json
+        loop = asyncio.get_event_loop()
+        queue: asyncio.Queue = asyncio.Queue()
+        _SENTINEL = object()
+
+        def _handle_line(line: str) -> bool:
+            if not line or not line.startswith("data:"):
+                return False
+            data = line[len("data:"):].strip()
+            if data == "[DONE]":
+                return True
+            try:
+                obj = json.loads(data)
+            except ValueError:
+                return False
+            choice = (obj.get("choices") or [{}])[0]
+            text = (choice.get(delta_key) or {}).get("content") or ""
+            if text:
+                loop.call_soon_threadsafe(queue.put_nowait, text)
+            if obj.get("usage"):
+                self._store_usage(obj["usage"])
+            return bool(choice.get("finish_reason"))
+
+        def _worker():
+            import requests
+            try:
+                with requests.post(url, json=payload, stream=True, timeout=3600) as r:
+                    r.raise_for_status()
+                    buf = b""
+                    done = False
+                    for bchunk in r.iter_content(chunk_size=8192):
+                        if not bchunk:
+                            continue
+                        buf += bchunk
+                        while b"\n" in buf:
+                            raw, buf = buf.split(b"\n", 1)
+                            if _handle_line(raw.decode("utf-8", "replace").strip()):
+                                done = True
+                                break
+                        if done:
+                            break
+                    if not done and buf.strip():
+                        _handle_line(buf.decode("utf-8", "replace").strip())
+            except Exception as exc:
+                loop.call_soon_threadsafe(queue.put_nowait, exc)
+            finally:
+                loop.call_soon_threadsafe(queue.put_nowait, _SENTINEL)
+
+        threading.Thread(target=_worker, daemon=True).start()
+        while True:
+            item = await queue.get()
+            if item is _SENTINEL:
+                break
+            if isinstance(item, Exception):
+                raise item
+            yield item

codai/config.py

@@ -449,6 +449,145 @@ class ColibriConfig:
     auto_build: bool = True                # clone+build the binary if it's missing
 
 
+@dataclass
+class K3Config:
+    """Kimi-K3 via kimi-k3-in-c (FareedKhan-dev) native-engine configuration.
+
+    kimi-k3-in-c is a portable-C CPU inference engine for Kimi-K3 (2.78T params, 16/896
+    experts active) that streams the always-on dense trunk + routed experts from disk,
+    running the full model in as little as ~8 GB RAM. Upstream it is a one-shot batch
+    CLI; coderai applies ``packaging/patch-k3.py`` to add a resident serve loop that
+    speaks the SAME mux stdin/stdout protocol as colibri, so the existing
+    :class:`~codai.api.colibri_worker.MuxEngine` drives it. coderai owns the build, the
+    process, the Kimi-K3 chat template (rendered XTML string) and the protocol client.
+
+    The model is a *directory* (the HF checkpoint, ~1.56 TB) plus a packed dense
+    ``trunk`` produced by the repo's ``scripts/pack-trunk.sh`` (~109 GB). Routing
+    matches by ``model_id``/alias/``model_path`` or a ``backend: "k3"`` pin. There is no
+    auto-download of the multi-TB checkpoint. CPU-only: needs AVX2 + FMA and ~1.7 TB of
+    fast local storage.
+    """
+    enabled: bool = False
+    repo_url: str = "https://github.com/FareedKhan-dev/kimi-k3-in-c"
+    install_dir: Optional[str] = None      # None = ~/.coderai/kimi-k3-in-c
+    model_path: str = ""                   # the Kimi-K3 checkpoint directory (config + tokenizer + shards)
+    trunk_dir: str = ""                    # packed dense trunk dir (scripts/pack-trunk.sh); "" = resident (needs ~114 GB RAM)
+    tok_dir: str = ""                      # tokenizer dir; "" = use the checkpoint (model_path) dir
+    model_id: str = "kimi-k3"              # model id/alias that routes to k3
+    preset: str = ""                       # laptop|desktop|workstation|server|max (blank = use trunk_gb/cache_gb)
+    trunk_gb: float = 16.0                 # GiB of the packed trunk pinned resident (--trunk-gb)
+    cache_gb: float = 64.0                 # GiB expert LRU cache (--cache-gb)
+    ctx: int = 4096                        # serve context capacity (env K3_MAXT)
+    extra_args: str = ""                   # extra flags passed to the k3 binary
+    extra_env: str = ""                    # free-form KEY=VALUE env (K3_EXPERT_GB, K3_BITS, K3_DIRECT, K3_PIPE, …)
+    auto_build: bool = True                # clone+patch+build the binary if it's missing
+
+
+@dataclass
+class KtransformersConfig:
+    """ktransformers (kvcache-ai) via SGLang — CPU+GPU heterogeneous engine config.
+
+    ktransformers is a CPU+GPU heterogeneous MoE inference engine (Intel AMX/AVX512/AVX2
+    CPU kernels for quantized experts + GPU for the dense trunk/attention). It exposes an
+    OpenAI-compatible HTTP server through SGLang, so — like ds4 — coderai owns the whole
+    lifecycle: it launches ``python -m sglang.launch_server`` as a managed subprocess and
+    proxies ``/v1/chat/completions`` to it. It can serve many families (DeepSeek-V3/R1/V4,
+    Kimi-K2/K2.5, Qwen3, GLM-5/5.2, MiniMax), so it is selected PER MODEL via an explicit
+    ``backend: "kt"`` pin or the configured ``model_id`` alias — never by a broad name
+    marker (it would collide with every other engine).
+
+    Heavy dependencies (SGLang + kt-kernel, built once) — install them out of band and
+    point ``model_path`` at the HF model dir and ``kt_weight_path`` at the KT quantized
+    weights. Best throughput needs AMX/AVX-512 CPUs.
+    """
+    enabled: bool = False
+    repo_url: str = "https://github.com/kvcache-ai/ktransformers"
+    install_dir: Optional[str] = None      # None = ~/.coderai/ktransformers (kt-kernel build)
+    model_path: str = ""                   # HF model directory (--model)
+    kt_weight_path: str = ""               # KT quantized weights directory (--kt-weight-path)
+    model_id: str = "ktransformers"        # model id/alias that routes to kt (and --served-model-name)
+    host: str = "127.0.0.1"                # SGLang bind host
+    port: int = 0                          # 0 = auto-pick a free port
+    ctx: int = 32768                       # context length (--context-length)
+    extra_args: str = ""                   # extra flags for sglang.launch_server (e.g. --tp-size, KT knobs)
+    extra_env: str = ""                    # free-form KEY=VALUE env for the subprocess
+    auto_build: bool = False               # pip-install SGLang+kt-kernel if missing (heavy; off by default)
+
+
+@dataclass
+class OcrConfig:
+    """Dedicated OCR subsystem configuration.
+
+    coderai's OCR is done by PURPOSE-BUILT OCR engines (detection + recognition),
+    NOT a vision LLM — far faster, GPU-batchable, and it returns faithful text with
+    line/word bounding boxes and layout instead of hallucinating. Three engines are
+    supported and selectable per request (``engine`` field on ``/v1/ocr``), falling
+    back to ``default_engine``:
+
+    - ``paddle``  — PaddleOCR + PP-Structure (Apache-2.0; layout + tables; recommended)
+    - ``doctr``   — Mindee docTR (Apache-2.0; pure-PyTorch; easy install)
+    - ``surya``   — Surya (best layout/reading-order; LICENSE-GATED — GPL/commercial:
+      only loaded when ``surya_accept_license`` is True)
+
+    Heavy OCR dependencies are installed out of band (optional) so the base image
+    stays lean. Concurrency on a single GPU comes from loading multiple instances of
+    an engine (``*_instances``) and fanning requests across them (``max_concurrency``);
+    a continuous-batching path via vLLM is a deferred option (see docs/vllm.md).
+
+    Per document coderai can produce: (1) plain-text transcription, (2) structured
+    JSON of fields via an existing coderai text model (``extract_*``), and (3)
+    stamp/signature flags (``detect_*``).
+    """
+    enabled: bool = False
+    default_engine: str = "paddle"        # paddle|doctr|surya
+    dpi: int = 200                        # PDF rasterisation DPI
+    max_concurrency: int = 4             # global cap on in-flight OCR pages
+    lang: str = "it"                     # default document language
+
+    # --- PaddleOCR / PP-Structure ---
+    paddle_enabled: bool = True
+    paddle_use_gpu: bool = True
+    paddle_instances: int = 2            # copies loaded for concurrent OCR
+    paddle_structure: bool = True        # PP-Structure layout + tables + reading order
+    paddle_lang: str = "it"             # PaddleOCR lang code (it / latin / …)
+    paddle_det_model_dir: str = ""       # override detection model dir (blank = default)
+    paddle_rec_model_dir: str = ""       # override recognition model dir (blank = default)
+    # PaddleOCR runs in an ISOLATED venv subprocess (opencv-contrib clash + bundled CUDA).
+    paddle_venv: str = ""               # isolated venv dir; blank = ~/.coderai/paddle_venv
+    paddle_auto_build: bool = False     # create the venv + pip install requirements-ocr-paddle.txt on first use
+
+    # --- docTR (Mindee) --- runs IN-PROCESS (uses the main venv's torch)
+    doctr_enabled: bool = False
+    doctr_use_gpu: bool = True
+    doctr_instances: int = 1
+    doctr_det_arch: str = "db_resnet50"
+    doctr_reco_arch: str = "crnn_vgg16_bn"
+
+    # --- Surya (LICENSE-GATED) --- runs in an ISOLATED venv subprocess (pillow<11 clash)
+    surya_enabled: bool = False
+    surya_accept_license: bool = False   # MUST be explicitly set (GPL — compatible with coderai GPLv3)
+    surya_instances: int = 1
+    surya_langs: str = "it"
+    surya_venv: str = ""               # isolated venv dir; blank = ~/.coderai/surya_venv
+    surya_auto_build: bool = False     # create the venv + pip install requirements-surya.txt on first use
+
+    # --- stamp / signature detection ---  [O3]
+    detect_mode: str = "off"             # off|layout|detector|both
+    detect_model_path: str = ""          # YOLO signature/stamp weights (detector/both modes)
+    detect_conf: float = 0.35            # detector confidence threshold
+
+    # --- structured field extraction (via an existing coderai text model) ---  [O4]
+    extract_enabled: bool = False
+    extract_model_id: str = ""           # coderai text model id/alias used for extraction
+    extract_max_tokens: int = 2048
+    # Default extraction schema SPEC — not a hardcoded schema. Accepts a named schema
+    # (e.g. "italian_sentenza"; files in <config_dir>/ocr_schemas/ + built-in seeds),
+    # inline JSON (starts with '{'/'['), or blank/"auto" for generic key/value
+    # extraction. Overridable per request via the `schema` field. See codai/ocr/schemas.py.
+    extract_schema: str = ""
+    extract_validate: bool = True        # validate output against JSON Schema (jsonschema-typed schemas only)
+
+
 @dataclass
 class Config:
     """Main configuration class."""
@@ -466,6 +605,9 @@ class Config:
     enhance: EnhanceConfig = field(default_factory=EnhanceConfig)
     ds4: Ds4Config = field(default_factory=Ds4Config)
     colibri: ColibriConfig = field(default_factory=ColibriConfig)
+    k3: K3Config = field(default_factory=K3Config)
+    ktransformers: KtransformersConfig = field(default_factory=KtransformersConfig)
+    ocr: OcrConfig = field(default_factory=OcrConfig)
     compaction: CompactionConfig = field(default_factory=CompactionConfig)
     broker: BrokerConfig = field(default_factory=BrokerConfig)
     system_prompt: Optional[str] = None
@@ -652,6 +794,9 @@ class ConfigManager:
                 enhance=_dc(EnhanceConfig, config_data.get("enhance", {})),
                 ds4=_dc(Ds4Config, config_data.get("ds4", {})),
                 colibri=_dc(ColibriConfig, config_data.get("colibri", {})),
+                k3=_dc(K3Config, config_data.get("k3", {})),
+                ktransformers=_dc(KtransformersConfig, config_data.get("ktransformers", {})),
+                ocr=_dc(OcrConfig, config_data.get("ocr", {})),
                 compaction=_dc(CompactionConfig, config_data.get("compaction", {})),
                 broker=_dc(BrokerConfig, config_data.get("broker", {})),
                 system_prompt=config_data.get("system_prompt"),
@@ -851,6 +996,71 @@ class ConfigManager:
                 "extra_env": self.config.colibri.extra_env,
                 "auto_build": self.config.colibri.auto_build,
             },
+            "k3": {
+                "enabled": self.config.k3.enabled,
+                "repo_url": self.config.k3.repo_url,
+                "install_dir": self.config.k3.install_dir,
+                "model_path": self.config.k3.model_path,
+                "trunk_dir": self.config.k3.trunk_dir,
+                "tok_dir": self.config.k3.tok_dir,
+                "model_id": self.config.k3.model_id,
+                "preset": self.config.k3.preset,
+                "trunk_gb": self.config.k3.trunk_gb,
+                "cache_gb": self.config.k3.cache_gb,
+                "ctx": self.config.k3.ctx,
+                "extra_args": self.config.k3.extra_args,
+                "extra_env": self.config.k3.extra_env,
+                "auto_build": self.config.k3.auto_build,
+            },
+            "ktransformers": {
+                "enabled": self.config.ktransformers.enabled,
+                "repo_url": self.config.ktransformers.repo_url,
+                "install_dir": self.config.ktransformers.install_dir,
+                "model_path": self.config.ktransformers.model_path,
+                "kt_weight_path": self.config.ktransformers.kt_weight_path,
+                "model_id": self.config.ktransformers.model_id,
+                "host": self.config.ktransformers.host,
+                "port": self.config.ktransformers.port,
+                "ctx": self.config.ktransformers.ctx,
+                "extra_args": self.config.ktransformers.extra_args,
+                "extra_env": self.config.ktransformers.extra_env,
+                "auto_build": self.config.ktransformers.auto_build,
+            },
+            "ocr": {
+                "enabled": self.config.ocr.enabled,
+                "default_engine": self.config.ocr.default_engine,
+                "dpi": self.config.ocr.dpi,
+                "max_concurrency": self.config.ocr.max_concurrency,
+                "lang": self.config.ocr.lang,
+                "paddle_enabled": self.config.ocr.paddle_enabled,
+                "paddle_use_gpu": self.config.ocr.paddle_use_gpu,
+                "paddle_instances": self.config.ocr.paddle_instances,
+                "paddle_structure": self.config.ocr.paddle_structure,
+                "paddle_lang": self.config.ocr.paddle_lang,
+                "paddle_det_model_dir": self.config.ocr.paddle_det_model_dir,
+                "paddle_rec_model_dir": self.config.ocr.paddle_rec_model_dir,
+                "paddle_venv": self.config.ocr.paddle_venv,
+                "paddle_auto_build": self.config.ocr.paddle_auto_build,
+                "doctr_enabled": self.config.ocr.doctr_enabled,
+                "doctr_use_gpu": self.config.ocr.doctr_use_gpu,
+                "doctr_instances": self.config.ocr.doctr_instances,
+                "doctr_det_arch": self.config.ocr.doctr_det_arch,
+                "doctr_reco_arch": self.config.ocr.doctr_reco_arch,
+                "surya_enabled": self.config.ocr.surya_enabled,
+                "surya_accept_license": self.config.ocr.surya_accept_license,
+                "surya_instances": self.config.ocr.surya_instances,
+                "surya_langs": self.config.ocr.surya_langs,
+                "surya_venv": self.config.ocr.surya_venv,
+                "surya_auto_build": self.config.ocr.surya_auto_build,
+                "detect_mode": self.config.ocr.detect_mode,
+                "detect_model_path": self.config.ocr.detect_model_path,
+                "detect_conf": self.config.ocr.detect_conf,
+                "extract_enabled": self.config.ocr.extract_enabled,
+                "extract_model_id": self.config.ocr.extract_model_id,
+                "extract_max_tokens": self.config.ocr.extract_max_tokens,
+                "extract_schema": self.config.ocr.extract_schema,
+                "extract_validate": self.config.ocr.extract_validate,
+            },
             "compaction": {
                 "enabled": self.config.compaction.enabled,
                 "pct": self.config.compaction.pct,

codai/frontproxy/app.py

@@ -908,6 +908,7 @@ class FrontProxy:
             new = cm.config
             for f in ("server", "backend", "models", "offload", "vulkan", "image",
                       "whisper", "archive", "thermal", "jobs", "enhance", "ds4", "colibri",
+                      "k3", "ktransformers",
                       "compaction", "broker", "system_prompt", "tools_closer_prompt",
                       "grammar_guided", "parser", "tmp_dir"):
                 if hasattr(new, f):
@@ -919,6 +920,8 @@ class FrontProxy:
     def _required_cap(self, path: str, model: Optional[str]) -> Optional[str]:
         ds4 = getattr(self.config, "ds4", None)
         colibri = getattr(self.config, "colibri", None)
+        k3 = getattr(self.config, "k3", None)
+        kt = getattr(self.config, "ktransformers", None)
         info = self._model_info(model)
         cap = _router.required_capability(
             model, path=path,
@@ -926,7 +929,11 @@ class FrontProxy:
             ds4_model_id=getattr(ds4, "model_id", None) if ds4 else None,
             ds4_enabled=bool(getattr(ds4, "enabled", False)) if ds4 else False,
             colibri_model_id=getattr(colibri, "model_id", None) if colibri else None,
-            colibri_enabled=bool(getattr(colibri, "enabled", False)) if colibri else False)
+            colibri_enabled=bool(getattr(colibri, "enabled", False)) if colibri else False,
+            k3_model_id=getattr(k3, "model_id", None) if k3 else None,
+            k3_enabled=bool(getattr(k3, "enabled", False)) if k3 else False,
+            kt_model_id=getattr(kt, "model_id", None) if kt else None,
+            kt_enabled=bool(getattr(kt, "enabled", False)) if kt else False)
         # The name heuristic can't see that a bare alias (e.g. '…-q4_k_m', no
         # literal 'gguf') backs a .gguf file, so it falls through to
         # 'transformers' (CUDA-only) and the request never reaches a Vulkan/AMD

codai/frontproxy/assignment.py

@@ -54,7 +54,7 @@ def _route_key(entry):
     return None
 
 
-def _required_cap(entry, ds4_cfg, colibri_cfg=None):
+def _required_cap(entry, ds4_cfg, colibri_cfg=None, k3_cfg=None, kt_cfg=None):
     from codai.frontproxy.router import required_capability
     path = _entry_path(entry) or ""
     backend = entry.get("backend") if isinstance(entry, dict) else None
@@ -63,11 +63,15 @@ def _required_cap(entry, ds4_cfg, colibri_cfg=None):
         ds4_model_id=getattr(ds4_cfg, "model_id", None) if ds4_cfg else None,
         ds4_enabled=bool(getattr(ds4_cfg, "enabled", False)) if ds4_cfg else False,
         colibri_model_id=getattr(colibri_cfg, "model_id", None) if colibri_cfg else None,
-        colibri_enabled=bool(getattr(colibri_cfg, "enabled", False)) if colibri_cfg else False)
+        colibri_enabled=bool(getattr(colibri_cfg, "enabled", False)) if colibri_cfg else False,
+        k3_model_id=getattr(k3_cfg, "model_id", None) if k3_cfg else None,
+        k3_enabled=bool(getattr(k3_cfg, "enabled", False)) if k3_cfg else False,
+        kt_model_id=getattr(kt_cfg, "model_id", None) if kt_cfg else None,
+        kt_enabled=bool(getattr(kt_cfg, "enabled", False)) if kt_cfg else False)
 
 
 def compute_assignment(engines, models_path, default_engine=None, ds4_cfg=None,
-                       colibri_cfg=None):
+                       colibri_cfg=None, k3_cfg=None, kt_cfg=None):
     """Return {engine_name: [model_identifiers]} — each model owned by one engine."""
     assignment = {e.name: [] for e in engines}
     if not engines or not models_path:
@@ -87,7 +91,7 @@ def compute_assignment(engines, models_path, default_engine=None, ds4_cfg=None,
             ident = _route_key(entry)
             if not ident or ident in seen:
                 continue
-            cap = _required_cap(entry, ds4_cfg, colibri_cfg)
+            cap = _required_cap(entry, ds4_cfg, colibri_cfg, k3_cfg, kt_cfg)
             candidates = [e for e in engines if e.can_serve(cap)]
             if not candidates:
                 continue   # nothing can run it — leave unassigned

codai/frontproxy/engine_supervisor.py

@@ -156,8 +156,10 @@ class EngineSupervisor:
             default_engine = getattr(self.config.server, "default_engine", None)
             ds4 = getattr(self.config, "ds4", None)
             colibri = getattr(self.config, "colibri", None)
+            k3 = getattr(self.config, "k3", None)
+            kt = getattr(self.config, "ktransformers", None)
             assignment = compute_assignment(engines, self.models_path,
-                                            default_engine, ds4, colibri)
+                                            default_engine, ds4, colibri, k3, kt)
             for e in engines:
                 owned = assignment.get(e.name, [])
                 e.assigned_models = set(owned)   # the front's router enforces this
@@ -604,8 +606,10 @@ class EngineSupervisor:
                 default_engine = getattr(self.config.server, "default_engine", None)
                 ds4 = getattr(self.config, "ds4", None)
                 colibri = getattr(self.config, "colibri", None)
+                k3 = getattr(self.config, "k3", None)
+                kt = getattr(self.config, "ktransformers", None)
                 assignment = compute_assignment(real, self.models_path,
-                                                default_engine, ds4, colibri)
+                                                default_engine, ds4, colibri, k3, kt)
             except Exception as exc:
                 print(f"[front] live reassignment skipped: {exc}", flush=True)
                 assignment = {}

codai/frontproxy/registry.py

@@ -41,12 +41,14 @@ def _short_stem(key: str) -> str:
 #   colibri      — GLM-5.2 via the native colibri C engine (CUDA build here)
 # An NVIDIA engine can do all of them; a Vulkan (e.g. Radeon) engine does GGUF and
 # whisper, but not transformers, ds4 or colibri.
+# k3 (kimi-k3-in-c) is a CPU engine → any node can host it; kt (ktransformers/SGLang)
+# is CPU+GPU → the GPU-capable nodes.
 _DEFAULT_CAPS = {
-    "nvidia": {"transformers", "gguf", "whisper", "ds4", "colibri"},
-    "cuda": {"transformers", "gguf", "whisper", "ds4", "colibri"},
-    "vulkan": {"gguf", "whisper"},
-    "opencl": {"gguf", "whisper"},
-    "auto": {"transformers", "gguf", "whisper", "ds4", "colibri"},
+    "nvidia": {"transformers", "gguf", "whisper", "ds4", "colibri", "k3", "kt"},
+    "cuda": {"transformers", "gguf", "whisper", "ds4", "colibri", "k3", "kt"},
+    "vulkan": {"gguf", "whisper", "k3"},
+    "opencl": {"gguf", "whisper", "k3"},
+    "auto": {"transformers", "gguf", "whisper", "ds4", "colibri", "k3", "kt"},
 }
 
 

codai/frontproxy/router.py

@@ -76,32 +76,59 @@ def required_capability(model: Optional[str], path: Optional[str] = None,
                         ds4_model_id: Optional[str] = None,
                         ds4_enabled: bool = False,
                         colibri_model_id: Optional[str] = None,
-                        colibri_enabled: bool = False) -> Optional[str]:
+                        colibri_enabled: bool = False,
+                        k3_model_id: Optional[str] = None,
+                        k3_enabled: bool = False,
+                        kt_model_id: Optional[str] = None,
+                        kt_enabled: bool = False) -> Optional[str]:
     """The capability an engine must have to serve this request.
 
     * ``whisper``      — whisper.cpp STT (transcription endpoint or a
                          ``whisper-server`` model). Runs on CUDA or Vulkan.
     * ``ds4``          — DeepSeek V4 via the native ds4 engine. CUDA-only.
-    * ``colibri``      — GLM-5.2 via the native colibri C engine. CUDA-only here.
+    * ``colibri``      — GLM-5.2 / DeepSeek-V4 / Kimi-K3 via the native colibri C engine.
+    * ``k3``           — Kimi-K3 via kimi-k3-in-c. CPU.
+    * ``kt``           — many families via ktransformers/SGLang. CPU+GPU.
     * ``gguf``         — llama.cpp model. Runs on CUDA or Vulkan.
     * ``transformers`` — safetensors/HF model. CUDA-only.
 
-    Signals are combined: the request path and the model's configured ``backend``
-    (from models.json) take precedence over the name heuristic, so whisper works
-    even when the model id isn't in the request body (multipart upload)."""
+    This mirrors the in-process resolver (:func:`codai.models.manager.resolve_engine_backend`):
+    an explicit ``backend`` pin is authoritative, then an enabled engine's ``model_id``
+    alias, then an unambiguous name marker (a Kimi name claimed by BOTH colibri and k3
+    falls through — the model must pin a backend)."""
     p = (path or "").split("?", 1)[0].rstrip("/")
     if p == "/v1/audio/transcriptions" or (backend or "") == "whisper-server":
         return "whisper"
+    # 1. Explicit engine-backend pin wins (authoritative), like the manager resolver.
+    b = (backend or "").lower()
+    if b in ("colibri", "ds4", "k3", "kt"):
+        return b
     m = (model or "").lower()
-    if (backend or "") == "colibri" or (colibri_enabled and m and (
-            ((colibri_model_id or "").lower()
-             and (m == (colibri_model_id or "").lower()
-                  or m.split("/")[-1] == (colibri_model_id or "").lower()))
-            or "glm-5.2" in m or "glm5.2" in m or "colibri" in m)):
+
+    def _alias(mid: Optional[str]) -> bool:
+        mid = (mid or "").lower()
+        return bool(mid) and (m == mid or m.split("/")[-1] == mid)
+
+    # 2. An enabled engine's model_id alias.
+    if kt_enabled and _alias(kt_model_id):
+        return "kt"
+    if k3_enabled and _alias(k3_model_id):
+        return "k3"
+    if colibri_enabled and _alias(colibri_model_id):
         return "colibri"
-    if ds4_enabled and m:
-        mid = (ds4_model_id or "").lower()
-        if (mid and (m == mid or m.split("/")[-1] == mid)) or "deepseek-v4" in m:
+    if ds4_enabled and _alias(ds4_model_id):
+        return "ds4"
+
+    # 3. Name markers (unambiguous only). GLM → colibri; DeepSeek → ds4 (default owner);
+    #    Kimi → colibri XOR k3 (ambiguous when both enabled → fall through to a pin).
+    if m:
+        if colibri_enabled and ("glm-5.2" in m or "glm5.2" in m or "colibri" in m):
+            return "colibri"
+        if "kimi-k3" in m or "kimi_k3" in m or "kimik3" in m:
+            claimers = [c for c, en in (("colibri", colibri_enabled), ("k3", k3_enabled)) if en]
+            if len(claimers) == 1:
+                return claimers[0]
+        if ds4_enabled and "deepseek-v4" in m:
             return "ds4"
     if m.endswith(".gguf") or "gguf" in m:
         return "gguf"

codai/main.py

@@ -1271,6 +1271,10 @@ def main():
     from codai.api.characters import set_global_args as set_chars_global_args
     set_chars_global_args(global_args)
 
+    # Set OCR module global args
+    from codai.api.ocr import set_global_args as set_ocr_global_args
+    set_ocr_global_args(global_args)
+
     # Set LoRA training module global args. Resolve job-recovery first (the
     # --no-resume-jobs flag overrides the persisted config setting), then call
     # set_global_args, which runs _load_jobs_on_start and honours the flag.

codai/models/manager.py

@@ -58,6 +58,28 @@ def get_active_colibri_config():
     return None
 
 
+def get_active_k3_config():
+    """Return the active K3Config (kimi-k3-in-c) from the server config, or None."""
+    try:
+        from codai.admin.routes import config_manager
+        if config_manager is not None and config_manager.config is not None:
+            return config_manager.config.k3
+    except Exception:
+        pass
+    return None
+
+
+def get_active_ktransformers_config():
+    """Return the active KtransformersConfig from the server config, or None."""
+    try:
+        from codai.admin.routes import config_manager
+        if config_manager is not None and config_manager.config is not None:
+            return config_manager.config.ktransformers
+    except Exception:
+        pass
+    return None
+
+
 _GGUF_ARCH_CACHE: Dict[tuple, str] = {}
 
 
@@ -165,52 +187,51 @@ def _gguf_architecture(path: str):
     return arch or None
 
 
-def ds4_should_handle(model_name: str) -> bool:
-    """True when ds4 is enabled and ``model_name`` is a DeepSeek-V4 (ds4) model.
+# --------------------------------------------------------------------------- #
+# External managed-engine routing (ds4 / colibri / k3 / kt)
+#
+# Several external engines can serve overlapping model families (DeepSeek-V4 by
+# ds4/colibri/kt; Kimi-K3 by colibri/k3/kt; GLM-5.2 by colibri/kt). To keep two
+# engines from both "claiming" the same model, a SINGLE resolver arbitrates by a
+# fixed precedence and every caller routes through it (via the thin
+# ``*_should_handle`` wrappers below). Precedence:
+#   1. an explicit ``backend:`` pin on the model's models.json entry — authoritative;
+#   2. else an *enabled* engine whose configured ``model_id`` alias equals the name;
+#   3. else name/architecture auto-detect, but ONLY when exactly one enabled engine
+#      claims the name (ambiguity routes to no engine and is logged).
+# --------------------------------------------------------------------------- #
 
-    Routing is by the GGUF ARCHITECTURE, not the filename: ds4 serves only genuine
-    ``deepseek4`` GGUFs (its own format). Mainline DeepSeek GGUFs (deepseek/
-    deepseek2/deepseek3/deepseek32) are left to llama.cpp. The configured
-    ``model_id`` alias still routes (covers the variant ds4 downloads itself, which
-    has no local file yet).
-    """
-    if not model_name:
-        return False
-    cfg = get_active_ds4_config()
-    if cfg is None or not getattr(cfg, "enabled", False):
-        return False
-    name = model_name.lower()
-    short = name.split("/")[-1]
-    mid = (getattr(cfg, "model_id", "") or "").lower()
-    if mid and (name == mid or short == mid):
-        return True
-    # Definitive: read the GGUF architecture for a local file — only deepseek4.
-    path = _resolve_local_gguf(model_name)
-    if path:
-        return (_gguf_architecture(path) or "").lower() == "deepseek4"
-    # No local file (HF id / not downloaded yet): conservative name check that
-    # matches ONLY the V4 marker, so mainline deepseek GGUFs aren't grabbed.
-    return "deepseek-v4" in name or "deepseek4" in name
+# Ordered by arbitration preference for ambiguous names. New engines append here.
+_ENGINE_BACKENDS = ("ds4", "colibri", "k3", "kt")
 
 
-def colibri_should_handle(model_name: str) -> bool:
-    """True when colibri is enabled and ``model_name`` is a GLM-5.2 (colibri) model.
+def _engine_config(engine: str):
+    """The active config object for an engine backend, or None."""
+    getter = {
+        "ds4": get_active_ds4_config,
+        "colibri": get_active_colibri_config,
+        "k3": get_active_k3_config,
+        "kt": get_active_ktransformers_config,
+    }.get(engine)
+    return getter() if getter else None
 
-    colibri's model is a *directory* container (int4), not a GGUF — so there is no
-    architecture to sniff. Routing is by the configured ``model_id`` alias, a GLM-5.2
-    name marker, or an explicit ``backend: "colibri"`` on the model's config entry.
-    """
+
+def _engine_enabled(engine: str) -> bool:
+    cfg = _engine_config(engine)
+    return bool(cfg is not None and getattr(cfg, "enabled", False))
+
+
+def _engine_model_id(engine: str) -> str:
+    cfg = _engine_config(engine)
+    return (getattr(cfg, "model_id", "") or "").lower() if cfg else ""
+
+
+def _model_entry_for(model_name: str):
+    """The models.json entry (dict) matching ``model_name`` by path / basename /
+    alias / id, or None."""
     if not model_name:
-        return False
-    cfg = get_active_colibri_config()
-    if cfg is None or not getattr(cfg, "enabled", False):
-        return False
-    name = model_name.lower()
-    short = name.split("/")[-1]
-    mid = (getattr(cfg, "model_id", "") or "").lower()
-    if mid and (name == mid or short == mid):
-        return True
-    # An explicit backend pin on the model's own config entry wins.
+        return None
+    name = model_name.strip().lower()
     try:
         from codai.admin.routes import config_manager as cfg_mgr
         md = getattr(cfg_mgr, "models_data", None) if cfg_mgr else None
@@ -223,13 +244,142 @@ def colibri_should_handle(model_name: str) -> bool:
                         continue
                     path = str(m.get("path") or m.get("id") or "")
                     base = os.path.basename(path.rstrip("/")).lower()
-                    cands = {path.lower(), base, str(m.get("alias") or "").lower()}
-                    if name in cands and str(m.get("backend") or "").lower() == "colibri":
-                        return True
+                    cands = {path.lower(), base, str(m.get("alias") or "").lower(),
+                             str(m.get("id") or "").lower()}
+                    if name in cands:
+                        return m
     except Exception:
         pass
-    # Name marker for GLM-5.2 (kept narrow so unrelated GLM GGUFs aren't grabbed).
-    return "glm-5.2" in name or "glm5.2" in name or "colibri" in short
+    return None
+
+
+def _pinned_backend(model_name: str) -> str:
+    """The explicit ``backend`` pin on the model's entry, lowercased, or ''."""
+    entry = _model_entry_for(model_name)
+    return str(entry.get("backend") or "").lower() if entry else ""
+
+
+def _engine_id_match(engine: str, model_name: str) -> bool:
+    mid = _engine_model_id(engine)
+    if not mid:
+        return False
+    name = model_name.lower()
+    return name == mid or name.split("/")[-1] == mid
+
+
+def _ds4_name_claims(model_name: str) -> bool:
+    """ds4's own name/architecture rule (no pin, no enabled check).
+
+    Routing is by GGUF ARCHITECTURE, not filename: ds4 serves only genuine
+    ``deepseek4`` GGUFs (its own format); mainline DeepSeek GGUFs are left to
+    llama.cpp. With no local file (HF id / not downloaded), fall back to a
+    conservative V4-only name marker.
+    """
+    path = _resolve_local_gguf(model_name)
+    if path:
+        return (_gguf_architecture(path) or "").lower() == "deepseek4"
+    name = model_name.lower()
+    return "deepseek-v4" in name or "deepseek4" in name
+
+
+def _colibri_name_claims(model_name: str) -> bool:
+    """colibri's own name rule (no pin, no enabled check).
+
+    colibri's model is a *directory* container (int4/native), not a GGUF, so there
+    is no architecture to sniff — match the served families by narrow name markers.
+
+    GLM-5.2 and Kimi-K3 auto-claim by name (colibri is their only current engine here).
+    DeepSeek-V4 is deliberately NOT auto-claimed: ds4 owns the ``deepseek-v4`` name by
+    default, so colibri-served DeepSeek is opt-in via an explicit ``backend: colibri``
+    pin (or the colibri ``model_id`` alias) — the resolver would otherwise mark a
+    ds4+colibri box ambiguous and route DeepSeek nowhere.
+    """
+    name = model_name.lower()
+    short = name.split("/")[-1]
+    return ("glm-5.2" in name or "glm5.2" in name or "colibri" in short
+            or "kimi-k3" in name or "kimi_k3" in name or "kimik3" in name)
+
+
+def _k3_name_claims(model_name: str) -> bool:
+    """kimi-k3-in-c's own name rule (no pin, no enabled check). Serves Kimi-K3, whose
+    checkpoint is a directory (not a GGUF), so match by narrow Kimi-K3 name markers.
+    When colibri is ALSO enabled both claim Kimi by name — the resolver then flags it
+    ambiguous and a ``backend`` pin chooses the engine, which is the intended UX."""
+    name = model_name.lower()
+    short = name.split("/")[-1]
+    return "kimi-k3" in name or "kimi_k3" in name or "kimik3" in name or "kimi-k3" in short
+
+
+def _kt_name_claims(model_name: str) -> bool:
+    """ktransformers serves MANY families (DeepSeek/Kimi/Qwen/GLM/MiniMax), so it must
+    NEVER auto-claim by a broad name marker — it would collide with every other engine.
+    It routes ONLY via an explicit ``backend: "kt"`` pin (step 1) or its configured
+    ``model_id`` alias (step 2)."""
+    return False
+
+
+_ENGINE_NAME_CLAIMS = {
+    "ds4": _ds4_name_claims,
+    "colibri": _colibri_name_claims,
+    "k3": _k3_name_claims,
+    "kt": _kt_name_claims,
+}
+
+
+def resolve_engine_backend(model_name: str):
+    """Decide which external managed engine (if any) serves ``model_name``.
+
+    Returns the engine key ("ds4"/"colibri"/…) or None. See the module comment
+    above for the precedence rules. This is the single arbiter — the
+    ``*_should_handle`` helpers are thin wrappers over it, so no two engines ever
+    claim the same model.
+    """
+    if not model_name:
+        return None
+
+    # 1. An explicit per-model backend pin is authoritative. A pin to an engine
+    #    backend routes there when enabled (else nowhere); a pin to a *non-engine*
+    #    backend (vulkan/nvidia/transformers/…) means "use the normal path", so it
+    #    also blocks name-marker auto-grab.
+    pin = _pinned_backend(model_name)
+    if pin:
+        return pin if (pin in _ENGINE_BACKENDS and _engine_enabled(pin)) else None
+
+    # 2. An enabled engine's configured model_id alias.
+    for eng in _ENGINE_BACKENDS:
+        if _engine_enabled(eng) and _engine_id_match(eng, model_name):
+            return eng
+
+    # 3. Name/architecture auto-detect — unambiguous only.
+    claimers = [eng for eng in _ENGINE_BACKENDS
+                if _engine_enabled(eng) and _ENGINE_NAME_CLAIMS[eng](model_name)]
+    if len(claimers) == 1:
+        return claimers[0]
+    if len(claimers) > 1:
+        print(f"[engine-route] {model_name!r} is claimed by multiple enabled engines "
+              f"{claimers}; pin a 'backend' on the model entry to disambiguate — "
+              "routing to none", flush=True)
+    return None
+
+
+def ds4_should_handle(model_name: str) -> bool:
+    """True when ds4 is the resolved engine backend for ``model_name``."""
+    return resolve_engine_backend(model_name) == "ds4"
+
+
+def colibri_should_handle(model_name: str) -> bool:
+    """True when colibri is the resolved engine backend for ``model_name``."""
+    return resolve_engine_backend(model_name) == "colibri"
+
+
+def k3_should_handle(model_name: str) -> bool:
+    """True when k3 (kimi-k3-in-c) is the resolved engine backend for ``model_name``."""
+    return resolve_engine_backend(model_name) == "k3"
+
+
+def kt_should_handle(model_name: str) -> bool:
+    """True when kt (ktransformers) is the resolved engine backend for ``model_name``."""
+    return resolve_engine_backend(model_name) == "kt"
 
 
 def _trim_cpu_ram() -> None:
@@ -375,13 +525,35 @@ class ModelManager:
         # (in-process mux protocol) instead of the normal nvidia/vulkan backends.
         if colibri_should_handle(model_name):
             from codai.backends.colibri import ColibriBackend
-            print(f"Routing '{model_name}' to colibri (GLM-5.2) backend")
+            print(f"Routing '{model_name}' to colibri (GLM-5.2 / DeepSeek-V4 / Kimi-K3) backend")
             self.backend_type = "colibri"
             self.backend = ColibriBackend(get_active_colibri_config())
             self.backend.load_model(model_name, **kwargs)
             self.tool_parser = ModelParserAdapter(model_name=model_name)
             return
 
+        # Kimi-K3 via kimi-k3-in-c: when enabled, drive the managed k3 C engine
+        # (same in-process mux protocol as colibri) instead of the normal backends.
+        if k3_should_handle(model_name):
+            from codai.backends.k3 import K3Backend
+            print(f"Routing '{model_name}' to k3 (Kimi-K3 / kimi-k3-in-c) backend")
+            self.backend_type = "k3"
+            self.backend = K3Backend(get_active_k3_config())
+            self.backend.load_model(model_name, **kwargs)
+            self.tool_parser = ModelParserAdapter(model_name=model_name)
+            return
+
+        # ktransformers via SGLang: when enabled, proxy matching models to the managed
+        # SGLang subprocess (CPU+GPU heterogeneous) instead of the normal backends.
+        if kt_should_handle(model_name):
+            from codai.backends.ktransformers import KtransformersBackend
+            print(f"Routing '{model_name}' to ktransformers (SGLang) backend")
+            self.backend_type = "kt"
+            self.backend = KtransformersBackend(get_active_ktransformers_config())
+            self.backend.load_model(model_name, **kwargs)
+            self.tool_parser = ModelParserAdapter(model_name=model_name)
+            return
+
         available = detect_available_backends()
 
         # Check if model is a GGUF file. The name alone isn't reliable: a gguf's
@@ -2036,6 +2208,15 @@ class MultiModelManager:
         if model_type in (None, "text") and colibri_should_handle(requested_or_resolved):
             return True
 
+        # k3 (kimi-k3-in-c) served Kimi-K3, same story: accept for text when it matches.
+        if model_type in (None, "text") and k3_should_handle(requested_or_resolved):
+            return True
+
+        # ktransformers (SGLang) served models: accept for text when kt is the resolver's
+        # pick (explicit backend:kt pin or the kt model_id alias).
+        if model_type in (None, "text") and kt_should_handle(requested_or_resolved):
+            return True
+
         # If a model_type is specified, reject models registered under a
         # different type (e.g. an image GGUF requested via /v1/chat/completions).
         if model_type:
@@ -4935,6 +5116,16 @@ class MultiModelManager:
             mid = getattr(colibri_cfg, "model_id", "glm-5.2-colibri") or "glm-5.2-colibri"
             _add(mid, "text", {"backend": "colibri"})
 
+        k3_cfg = get_active_k3_config()
+        if k3_cfg is not None and getattr(k3_cfg, "enabled", False):
+            mid = getattr(k3_cfg, "model_id", "kimi-k3") or "kimi-k3"
+            _add(mid, "text", {"backend": "k3"})
+
+        kt_cfg = get_active_ktransformers_config()
+        if kt_cfg is not None and getattr(kt_cfg, "enabled", False):
+            mid = getattr(kt_cfg, "model_id", "ktransformers") or "ktransformers"
+            _add(mid, "text", {"backend": "kt"})
+
         return models
 
 

codai/ocr/__init__.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""Dedicated OCR subsystem for coderai.
+
+Purpose-built OCR engines (detection + recognition), NOT a vision LLM. Engines are
+pluggable behind :class:`~codai.ocr.base.OcrEngine` and selected per request; the
+:class:`~codai.ocr.manager.OcrManager` holds per-engine instance pools for concurrent
+GPU OCR. See docs/ocr.md.
+"""
+
+from codai.ocr.base import OcrEngine, OcrPage, OcrLine, OcrRegion, OcrError, load_pages
+
+__all__ = [
+    "OcrEngine",
+    "OcrPage",
+    "OcrLine",
+    "OcrRegion",
+    "OcrError",
+    "load_pages",
+]

codai/ocr/base.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""OCR engine interface + shared result types + input loading.
+
+Every OCR engine (PaddleOCR, docTR, Surya) subclasses :class:`OcrEngine` and returns
+:class:`OcrPage` objects. The rest of the subsystem (manager, API, detection,
+extraction) speaks only these types, so engines stay interchangeable.
+"""
+
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from typing import Any, List, Optional
+
+
+class OcrError(RuntimeError):
+    """Raised for OCR failures the API should surface (missing deps, bad input, …).
+
+    ``status`` maps to the HTTP status the endpoint should return (503 when an engine's
+    optional dependency is not installed, 400 for bad input, 500 otherwise).
+    """
+
+    def __init__(self, message: str, status: int = 500):
+        super().__init__(message)
+        self.status = status
+
+
+# ---------------------------------------------------------------------------
+# Result types
+# ---------------------------------------------------------------------------
+
+@dataclass
+class OcrLine:
+    """A recognised text line (or word), with its polygon bounding box."""
+    text: str
+    bbox: List[float]              # [x0, y0, x1, y1] in pixels
+    conf: float = 0.0
+
+    def to_dict(self) -> dict:
+        return {"text": self.text, "bbox": self.bbox, "conf": round(self.conf, 4)}
+
+
+@dataclass
+class OcrRegion:
+    """A layout region (title/text/table/figure/seal/…) from layout analysis.
+
+    Used both for structure output and — in the ``layout`` stamp/signature detection
+    mode — to flag likely stamps/seals/figures without a separate detector.
+    """
+    label: str
+    bbox: List[float]             # [x0, y0, x1, y1] in pixels
+    conf: float = 0.0
+
+    def to_dict(self) -> dict:
+        return {"label": self.label, "bbox": self.bbox, "conf": round(self.conf, 4)}
+
+
+@dataclass
+class OcrPage:
+    """OCR result for a single page/image."""
+    index: int
+    text: str = ""
+    width: int = 0
+    height: int = 0
+    lines: List[OcrLine] = field(default_factory=list)
+    regions: List[OcrRegion] = field(default_factory=list)
+    tables: List[Any] = field(default_factory=list)     # list of {bbox, html?, cells?}
+
+    def to_dict(self) -> dict:
+        return {
+            "index": self.index,
+            "text": self.text,
+            "width": self.width,
+            "height": self.height,
+            "lines": [l.to_dict() for l in self.lines],
+            "regions": [r.to_dict() for r in self.regions],
+            "tables": self.tables,
+        }
+
+
+# ---------------------------------------------------------------------------
+# Engine interface
+# ---------------------------------------------------------------------------
+
+class OcrEngine(ABC):
+    """Base class for a dedicated OCR engine.
+
+    Subclasses import their heavy dependency LAZILY inside :meth:`load` (never at module
+    import time) and raise :class:`OcrError` with ``status=503`` if it is missing, so the
+    server runs fine without any OCR library installed.
+    """
+
+    name: str = "base"
+
+    def __init__(self, cfg):
+        self.cfg = cfg
+        self._loaded = False
+
+    @abstractmethod
+    def load(self) -> None:
+        """Load models into memory (called once per instance, lazily). Idempotent."""
+
+    @abstractmethod
+    def recognize_image(self, image) -> OcrPage:
+        """OCR a single page image (a PIL.Image.Image) → :class:`OcrPage`."""
+
+    def ensure_loaded(self) -> None:
+        if not self._loaded:
+            self.load()
+            self._loaded = True
+
+
+# ---------------------------------------------------------------------------
+# Input loading (images + PDF) → list of PIL pages
+# ---------------------------------------------------------------------------
+
+# Extensions we accept directly as raster images.
+_IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".tif", ".tiff", ".bmp", ".webp", ".gif", ".ppm"}
+
+
+def load_pages(data: bytes, filename: str = "", content_type: str = "", dpi: int = 200) -> List["Any"]:
+    """Decode ``data`` (an image or a PDF) into a list of RGB ``PIL.Image`` pages.
+
+    PDFs are rasterised at ``dpi`` via pypdfium2 (preferred) or pdf2image/poppler.
+    Raises :class:`OcrError` (status 400) on undecodable input, 503 if PDF support is
+    needed but unavailable.
+    """
+    try:
+        from PIL import Image
+    except Exception as e:  # pragma: no cover - Pillow is a base dep
+        raise OcrError(f"Pillow (PIL) is required for OCR: {e}", status=503)
+
+    import io
+
+    name = (filename or "").lower()
+    is_pdf = (content_type or "").lower() == "application/pdf" or name.endswith(".pdf") \
+        or data[:5] == b"%PDF-"
+
+    if is_pdf:
+        return _rasterize_pdf(data, dpi)
+
+    try:
+        img = Image.open(io.BytesIO(data))
+        img.load()
+    except Exception as e:
+        raise OcrError(f"could not decode image: {e}", status=400)
+
+    # Multi-frame images (multi-page TIFF) → one page per frame.
+    pages = []
+    n = getattr(img, "n_frames", 1)
+    for i in range(n):
+        try:
+            img.seek(i)
+        except Exception:
+            pass
+        pages.append(img.convert("RGB"))
+    return pages
+
+
+def _rasterize_pdf(data: bytes, dpi: int) -> List["Any"]:
+    """Rasterise a PDF to a list of RGB PIL pages (pypdfium2 first, then pdf2image)."""
+    from PIL import Image  # noqa: F401  (imported for type parity / already validated)
+
+    # Preferred: pypdfium2 (self-contained, no system poppler).
+    try:
+        import pypdfium2 as pdfium
+    except Exception:
+        pdfium = None
+
+    if pdfium is not None:
+        try:
+            pdf = pdfium.PdfDocument(data)
+            scale = max(0.5, dpi / 72.0)
+            pages = []
+            for i in range(len(pdf)):
+                page = pdf[i]
+                bitmap = page.render(scale=scale)
+                pages.append(bitmap.to_pil().convert("RGB"))
+            return pages
+        except Exception as e:
+            raise OcrError(f"failed to rasterise PDF (pypdfium2): {e}", status=400)
+
+    # Fallback: pdf2image (needs system poppler).
+    try:
+        from pdf2image import convert_from_bytes
+    except Exception:
+        raise OcrError(
+            "PDF OCR needs 'pypdfium2' (pip install pypdfium2) or 'pdf2image' + poppler",
+            status=503,
+        )
+    try:
+        return [im.convert("RGB") for im in convert_from_bytes(data, dpi=dpi)]
+    except Exception as e:
+        raise OcrError(f"failed to rasterise PDF (pdf2image): {e}", status=400)

codai/ocr/detect.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""Stamp / signature detection for the OCR subsystem.
+
+Configurable via ``ocr.detect_mode``:
+
+- ``off``       — no detection.
+- ``layout``    — reuse the OCR engine's layout regions (PP-Structure etc.); non-text
+  regions (figure/seal/stamp) are flagged as candidate stamps. No extra model; coarse,
+  and signatures are rarely separable this way.
+- ``detector``  — run a small YOLO model (``ocr.detect_model_path``) trained to spot
+  signatures/stamps; returns tight boxes + confidence + class.
+- ``both``      — run both and merge.
+
+The detector model is loaded once per weights path and reused across requests.
+"""
+
+import threading
+from typing import Dict, List, Optional
+
+from codai.ocr.base import OcrPage, OcrError
+
+
+VALID_MODES = ("off", "layout", "detector", "both")
+
+# Class-name → bucket heuristics (English + Italian markers).
+_SIGNATURE_MARKERS = ("sign", "signature", "firma", "autograph")
+_STAMP_MARKERS = ("stamp", "seal", "timbro", "sigillo", "bollo")
+# Layout labels (PP-Structure / docTR / Surya) that plausibly cover a stamp/seal.
+_LAYOUT_STAMP_LABELS = ("figure", "seal", "stamp", "image", "graphic")
+
+
+def resolve_detect_mode(cfg, override: Optional[str]) -> str:
+    """Per-request override wins over the configured default; validated."""
+    mode = (override or getattr(cfg, "detect_mode", "off") or "off").strip().lower()
+    if mode not in VALID_MODES:
+        raise OcrError(f"invalid detect mode '{mode}' (use {'/'.join(VALID_MODES)})", status=400)
+    return mode
+
+
+def _classify(label: str) -> str:
+    lo = (label or "").lower()
+    if any(m in lo for m in _SIGNATURE_MARKERS):
+        return "signature"
+    if any(m in lo for m in _STAMP_MARKERS):
+        return "stamp"
+    return "stamp"   # unknown detector classes default to the stamp bucket (label kept)
+
+
+# ---------------------------------------------------------------------------
+# YOLO detector (lazy, cached per weights path)
+# ---------------------------------------------------------------------------
+
+_detector_cache: Dict[str, "YoloDetector"] = {}
+_cache_lock = threading.Lock()
+
+
+class YoloDetector:
+    def __init__(self, model_path: str, conf: float):
+        self.model_path = model_path
+        self.conf = conf
+        self._model = None
+        self._load_lock = threading.Lock()
+
+    def ensure_loaded(self):
+        if self._model is not None:
+            return
+        with self._load_lock:
+            if self._model is not None:
+                return
+            try:
+                from ultralytics import YOLO
+            except Exception as e:
+                raise OcrError(
+                    "ultralytics is not installed (needed for detector mode). Install: "
+                    f"`pip install ultralytics`. Import error: {e}",
+                    status=503,
+                )
+            import os
+            if not self.model_path or not os.path.isfile(self.model_path):
+                raise OcrError(
+                    "stamp/signature detector weights not found. Set ocr.detect_model_path "
+                    "to a YOLO .pt file trained on signatures/stamps.",
+                    status=400,
+                )
+            try:
+                self._model = YOLO(self.model_path)
+            except Exception as e:
+                raise OcrError(f"failed to load YOLO detector: {e}", status=500)
+
+    def detect(self, image) -> List[dict]:
+        self.ensure_loaded()
+        import numpy as np
+
+        arr = np.array(image.convert("RGB"))
+        out: List[dict] = []
+        try:
+            results = self._model.predict(arr, conf=self.conf, verbose=False)
+        except Exception as e:
+            raise OcrError(f"YOLO detection failed: {e}", status=500)
+        for res in results:
+            names = getattr(res, "names", {}) or {}
+            boxes = getattr(res, "boxes", None)
+            if boxes is None:
+                continue
+            for b in boxes:
+                try:
+                    xyxy = [float(v) for v in b.xyxy[0].tolist()]
+                    conf = float(b.conf[0].item()) if b.conf is not None else 0.0
+                    cls_id = int(b.cls[0].item()) if b.cls is not None else -1
+                    label = str(names.get(cls_id, cls_id))
+                    out.append({
+                        "bbox": xyxy, "conf": round(conf, 4),
+                        "label": label, "type": _classify(label), "source": "detector",
+                    })
+                except Exception:
+                    continue
+        return out
+
+
+# Default detector weight locations (baked in image, then persistent cache, then host).
+# A bundled signature detector ships at the first path; stamps are best covered by the
+# 'layout' mode or a custom multi-class model via ocr.detect_model_path.
+_DEFAULT_DETECTOR_NAME = "signature-yolo.pt"
+
+
+def _resolve_detector_path(cfg) -> str:
+    import os
+    if cfg.detect_model_path:
+        return cfg.detect_model_path
+    cache = os.environ.get("CODERAI_CACHE_DIR") or ("/cache" if os.path.isdir("/cache") else "")
+    cands = [f"/opt/coderai/models/ocr/{_DEFAULT_DETECTOR_NAME}"]
+    if cache:
+        cands.append(os.path.join(cache, "ocr", _DEFAULT_DETECTOR_NAME))
+    cands.append(os.path.expanduser(f"~/.coderai/models/ocr/{_DEFAULT_DETECTOR_NAME}"))
+    for c in cands:
+        if os.path.isfile(c):
+            return c
+    return ""   # none found → YoloDetector raises a clear 400
+
+
+def _get_detector(cfg) -> YoloDetector:
+    path = _resolve_detector_path(cfg)
+    key = f"{path}|{cfg.detect_conf}"
+    with _cache_lock:
+        det = _detector_cache.get(key)
+        if det is None:
+            det = YoloDetector(path, float(cfg.detect_conf))
+            _detector_cache[key] = det
+        return det
+
+
+# ---------------------------------------------------------------------------
+# Layout-based flagging (no extra model)
+# ---------------------------------------------------------------------------
+
+def _detect_from_layout(page: OcrPage) -> List[dict]:
+    out: List[dict] = []
+    for reg in page.regions:
+        lo = (reg.label or "").lower()
+        hit = (any(m in lo for m in _LAYOUT_STAMP_LABELS)
+               or any(m in lo for m in _STAMP_MARKERS)
+               or any(m in lo for m in _SIGNATURE_MARKERS))
+        if hit:
+            out.append({
+                "bbox": list(reg.bbox), "conf": round(float(reg.conf), 4),
+                "label": reg.label, "type": _classify(reg.label), "source": "layout",
+            })
+    return out
+
+
+# ---------------------------------------------------------------------------
+# Entry point (runs in a worker thread; sync)
+# ---------------------------------------------------------------------------
+
+def detect_page(image, page: OcrPage, cfg, mode: str) -> Dict[str, List[dict]]:
+    """Return {'stamps': [...], 'signatures': [...]} for one page under ``mode``."""
+    dets: List[dict] = []
+    if mode in ("layout", "both"):
+        dets.extend(_detect_from_layout(page))
+    if mode in ("detector", "both"):
+        dets.extend(_get_detector(cfg).detect(image))
+
+    stamps = [d for d in dets if d["type"] == "stamp"]
+    signatures = [d for d in dets if d["type"] == "signature"]
+    return {"stamps": stamps, "signatures": signatures}

codai/ocr/doctr.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""docTR (Mindee) OCR engine.
+
+Apache-2.0, pure-PyTorch (uses the torch already in the stack). Detection +
+recognition predictor; docTR groups words into blocks/lines with reading order, so
+transcription comes out ordered. Weaker on complex table layout than PaddleOCR/Surya.
+"""
+
+from typing import List
+
+from codai.ocr.base import OcrEngine, OcrPage, OcrLine, OcrError
+
+
+class DoctrEngine(OcrEngine):
+    name = "doctr"
+
+    def __init__(self, cfg):
+        super().__init__(cfg)
+        self._model = None
+
+    def load(self) -> None:
+        try:
+            from doctr.models import ocr_predictor
+        except Exception as e:
+            raise OcrError(
+                "docTR is not installed. Install it out of band: "
+                f"`pip install python-doctr[torch]`. Import error: {e}",
+                status=503,
+            )
+        try:
+            self._model = ocr_predictor(
+                det_arch=self.cfg.doctr_det_arch or "db_resnet50",
+                reco_arch=self.cfg.doctr_reco_arch or "crnn_vgg16_bn",
+                pretrained=True,
+            )
+        except Exception as e:
+            raise OcrError(f"failed to initialise docTR predictor: {e}", status=500)
+
+        # Device placement: default is GPU when torch sees CUDA. Force CPU if requested.
+        if not self.cfg.doctr_use_gpu:
+            try:
+                self._model = self._model.cpu()
+            except Exception:
+                pass
+        else:
+            try:
+                import torch
+                if torch.cuda.is_available():
+                    self._model = self._model.cuda()
+            except Exception:
+                pass
+
+    def recognize_image(self, image) -> OcrPage:
+        import numpy as np
+
+        arr = np.array(image.convert("RGB"))
+        h, w = arr.shape[0], arr.shape[1]
+        page = OcrPage(index=0, width=int(w), height=int(h))
+
+        try:
+            result = self._model([arr])
+            export = result.export()
+        except Exception as e:
+            raise OcrError(f"docTR recognition failed: {e}", status=500)
+
+        lines: List[OcrLine] = []
+        pages = export.get("pages") or []
+        if pages:
+            pg = pages[0]
+            dims = pg.get("dimensions") or (h, w)   # (height, width)
+            ph, pw = float(dims[0]), float(dims[1])
+            for block in pg.get("blocks", []):
+                for line in block.get("lines", []):
+                    words = line.get("words", [])
+                    text = " ".join(str(wd.get("value", "")) for wd in words).strip()
+                    if not text:
+                        continue
+                    confs = [float(wd.get("confidence", 0.0)) for wd in words] or [0.0]
+                    geom = line.get("geometry") or ((0, 0), (0, 0))
+                    (x0, y0), (x1, y1) = geom
+                    bbox = [x0 * pw, y0 * ph, x1 * pw, y1 * ph]
+                    lines.append(OcrLine(text=text, bbox=bbox, conf=sum(confs) / len(confs)))
+
+        page.lines = lines
+        page.text = "\n".join(l.text for l in lines)
+        return page

codai/ocr/extract.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""Structured field extraction from OCR text via an existing coderai text model.
+
+This is NOT a separate model — it reuses whatever text LLM is configured
+(``ocr.extract_model_id``) through the in-process ``/v1/chat/completions`` handler. The
+target schema is not hardcoded: it is resolved from the schema store (named schema,
+inline schema, or generic "auto" mode). See :mod:`codai.ocr.schemas`.
+"""
+
+import json
+from typing import Optional
+
+from codai.ocr.base import OcrError
+from codai.ocr.schemas import schema_store
+
+
+_SYSTEM_PROMPT = (
+    "You extract structured data from OCR'd documents. The OCR text may contain "
+    "recognition errors; correct them implicitly where obvious. Respond with a SINGLE "
+    "valid JSON object ONLY — no prose, no explanations, no code fences. Use null for "
+    "fields not present in the document."
+)
+
+_AUTO_INSTRUCTION = (
+    "No fixed schema was given. Extract the salient fields of this document as a flat "
+    "JSON object of key/value pairs (include a \"document_type\" key), plus a \"summary\"."
+)
+
+
+def _build_user_prompt(text: str, schema_obj: Optional[dict]) -> str:
+    parts = []
+    if schema_obj is None:
+        parts.append(_AUTO_INSTRUCTION)
+    else:
+        stype = schema_obj.get("type", "template")
+        hint = schema_obj.get("prompt_hint", "")
+        schema_json = json.dumps(schema_obj.get("schema", {}), ensure_ascii=False, indent=2)
+        if stype == "jsonschema":
+            parts.append(
+                "Return a JSON object conforming to this JSON Schema (same property "
+                "names and types):"
+            )
+        else:
+            parts.append(
+                "Extract the fields following EXACTLY this JSON template (same keys):"
+            )
+        parts.append(schema_json)
+        if hint:
+            parts.append(hint)
+    parts.append("Document OCR text:\n-----8<-----\n" + text + "\n-----8<-----")
+    parts.append("Return only the filled JSON.")
+    return "\n\n".join(parts)
+
+
+def _content_from_result(result) -> str:
+    """Pull the assistant message content from a chat_completions return value,
+    tolerating dict / pydantic model / JSONResponse shapes."""
+    r = None
+    if isinstance(result, dict):
+        r = result
+    elif hasattr(result, "model_dump"):
+        try:
+            r = result.model_dump()
+        except Exception:
+            r = None
+    if r is None and hasattr(result, "body"):
+        try:
+            r = json.loads(result.body)
+        except Exception:
+            r = None
+    if r is None and hasattr(result, "__dict__"):
+        r = result.__dict__
+    if not isinstance(r, dict):
+        return ""
+    choices = r.get("choices") or []
+    if not choices:
+        return ""
+    msg = choices[0].get("message") if isinstance(choices[0], dict) else None
+    if isinstance(msg, dict):
+        return msg.get("content") or ""
+    return ""
+
+
+def _parse_json_lenient(content: str):
+    """Best-effort JSON parse: strip code fences, then fall back to the outermost {...}."""
+    s = (content or "").strip()
+    if s.startswith("```"):
+        s = s.split("```", 2)
+        s = s[1] if len(s) > 1 else content
+        if s.lstrip().lower().startswith("json"):
+            s = s.lstrip()[4:]
+    s = s.strip()
+    try:
+        return json.loads(s)
+    except Exception:
+        pass
+    a, b = s.find("{"), s.rfind("}")
+    if a != -1 and b != -1 and b > a:
+        try:
+            return json.loads(s[a:b + 1])
+        except Exception:
+            pass
+    return None
+
+
+def _validate(parsed, schema_obj: Optional[dict]) -> Optional[str]:
+    """Validate ``parsed`` against a JSON Schema if applicable. Returns an error string
+    or None. Best-effort: silently skips when jsonschema is absent or type != jsonschema."""
+    if not schema_obj or schema_obj.get("type") != "jsonschema":
+        return None
+    try:
+        import jsonschema
+    except Exception:
+        return None
+    try:
+        jsonschema.validate(parsed, schema_obj.get("schema", {}))
+        return None
+    except Exception as e:
+        return str(getattr(e, "message", e))
+
+
+async def extract_fields(text: str, cfg, schema: Optional[str] = None) -> dict:
+    """Run field extraction. ``schema`` is a per-request spec (name, inline JSON, or
+    auto); falls back to ``cfg.extract_schema``. Returns {"schema": name, "fields": {...}}
+    or, on parse failure, {"_parse_error", "_raw"}. Raises OcrError(400) if not configured.
+    """
+    if not getattr(cfg, "extract_enabled", False):
+        raise OcrError("structured extraction is disabled (set ocr.extract_enabled)", status=400)
+    model_id = getattr(cfg, "extract_model_id", "") or ""
+    if not model_id:
+        raise OcrError("no extraction model set (ocr.extract_model_id)", status=400)
+
+    spec = schema if (schema is not None and schema != "") else getattr(cfg, "extract_schema", "")
+    schema_obj, _ = schema_store.resolve(spec)
+    schema_name = (schema_obj or {}).get("name") or ("auto" if schema_obj is None else "inline")
+
+    if not (text or "").strip():
+        return {"schema": schema_name, "fields": None, "_note": "empty OCR text"}
+
+    from codai.api.text import chat_completions
+    from codai.pydantic.textrequest import ChatCompletionRequest
+
+    req = ChatCompletionRequest(
+        model=model_id,
+        messages=[
+            {"role": "system", "content": _SYSTEM_PROMPT},
+            {"role": "user", "content": _build_user_prompt(text, schema_obj)},
+        ],
+        stream=False,
+        temperature=0.0,
+        max_tokens=int(getattr(cfg, "extract_max_tokens", 2048) or 2048),
+    )
+    try:
+        result = await chat_completions(req, None)
+    except Exception as e:
+        raise OcrError(f"extraction model call failed: {e}", status=500)
+
+    content = _content_from_result(result)
+    parsed = _parse_json_lenient(content)
+    if parsed is None:
+        return {"schema": schema_name, "fields": None,
+                "_parse_error": "model did not return valid JSON", "_raw": content}
+
+    out = {"schema": schema_name, "fields": parsed}
+    if bool(getattr(cfg, "extract_validate", True)):
+        err = _validate(parsed, schema_obj)
+        if err:
+            out["_validation_error"] = err
+    return out

codai/ocr/manager.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""OCR engine manager — per-engine instance pools for concurrent GPU OCR.
+
+Each engine is loaded as N resident instances (``*_instances`` in
+:class:`~codai.config.OcrConfig`); pages are fanned across the pool so many documents
+OCR concurrently on one GPU. An engine instance handles one page at a time (OCR runtimes
+are not reentrant), so pool size bounds that engine's concurrency; ``max_concurrency``
+bounds the whole subsystem.
+"""
+
+import asyncio
+from typing import Dict, List, Optional
+
+from codai.ocr.base import OcrEngine, OcrPage, OcrError, load_pages
+
+
+# Engine registry: name → factory(cfg) → OcrEngine. Imports are deferred inside each
+# factory so a missing OCR dependency never breaks manager import.
+def _paddle_factory(cfg):
+    from codai.ocr.paddle import PaddleEngine
+    return PaddleEngine(cfg)
+
+
+def _doctr_factory(cfg):
+    from codai.ocr.doctr import DoctrEngine
+    return DoctrEngine(cfg)
+
+
+def _surya_factory(cfg):
+    from codai.ocr.surya import SuryaEngine
+    return SuryaEngine(cfg)
+
+
+_ENGINE_FACTORIES = {
+    "paddle": _paddle_factory,
+    "doctr": _doctr_factory,
+    "surya": _surya_factory,
+}
+
+
+def _engine_enabled(cfg, name: str) -> bool:
+    if name == "paddle":
+        return bool(cfg.paddle_enabled)
+    if name == "doctr":
+        return bool(cfg.doctr_enabled)
+    if name == "surya":
+        return bool(cfg.surya_enabled and cfg.surya_accept_license)
+    return False
+
+
+def _engine_instances(cfg, name: str) -> int:
+    n = getattr(cfg, f"{name}_instances", 1)
+    try:
+        return max(1, int(n))
+    except Exception:
+        return 1
+
+
+class _Pool:
+    """A fixed-size pool of loaded engine instances, served via an asyncio.Queue."""
+
+    def __init__(self, name: str, cfg, size: int):
+        self.name = name
+        self.cfg = cfg
+        self.size = size
+        self._q: asyncio.Queue = asyncio.Queue()
+        self._built = False
+        self._build_lock = asyncio.Lock()
+
+    async def _ensure_built(self):
+        if self._built:
+            return
+        async with self._build_lock:
+            if self._built:
+                return
+            factory = _ENGINE_FACTORIES[self.name]
+            for _ in range(self.size):
+                eng = factory(self.cfg)
+                # Load the first instance synchronously-in-thread so a missing
+                # dependency surfaces as OcrError(503) before we spawn the rest.
+                await asyncio.to_thread(eng.ensure_loaded)
+                await self._q.put(eng)
+            self._built = True
+            print(f"[ocr] engine '{self.name}': {self.size} instance(s) ready")
+
+    async def recognize(self, image) -> OcrPage:
+        await self._ensure_built()
+        eng: OcrEngine = await self._q.get()
+        try:
+            return await asyncio.to_thread(eng.recognize_image, image)
+        finally:
+            await self._q.put(eng)
+
+
+class OcrManager:
+    """Holds per-engine pools; (re)configured from OcrConfig on demand."""
+
+    def __init__(self):
+        self._cfg = None
+        self._pools: Dict[str, _Pool] = {}
+        self._sem: Optional[asyncio.Semaphore] = None
+        self._lock = asyncio.Lock()
+
+    def configure(self, cfg) -> None:
+        """Point the manager at the current OcrConfig. Rebuilds pools if params changed."""
+        prev = self._cfg
+        self._cfg = cfg
+        if prev is None or self._pool_params(prev) != self._pool_params(cfg):
+            # Drop stale pools (instances are GC'd; their models freed on unload/GC).
+            self._pools = {}
+            self._sem = asyncio.Semaphore(max(1, int(getattr(cfg, "max_concurrency", 4))))
+
+    @staticmethod
+    def _pool_params(cfg) -> tuple:
+        return (
+            cfg.max_concurrency,
+            cfg.paddle_enabled, cfg.paddle_instances, cfg.paddle_use_gpu,
+            cfg.paddle_structure, cfg.paddle_lang,
+            cfg.paddle_det_model_dir, cfg.paddle_rec_model_dir,
+            cfg.paddle_venv, cfg.paddle_auto_build,
+            cfg.doctr_enabled, cfg.doctr_instances, cfg.doctr_use_gpu,
+            cfg.doctr_det_arch, cfg.doctr_reco_arch,
+            cfg.surya_enabled, cfg.surya_accept_license, cfg.surya_instances,
+            cfg.surya_langs, cfg.surya_venv, cfg.surya_auto_build,
+        )
+
+    def resolve_engine(self, requested: Optional[str]) -> str:
+        cfg = self._require_cfg()
+        name = (requested or cfg.default_engine or "paddle").strip().lower()
+        if name not in _ENGINE_FACTORIES:
+            raise OcrError(f"unknown OCR engine '{name}'", status=400)
+        if not _engine_enabled(cfg, name):
+            raise OcrError(
+                f"OCR engine '{name}' is not enabled (or license not accepted). "
+                f"Enable it in Settings → OCR.",
+                status=400,
+            )
+        return name
+
+    async def _get_pool(self, name: str) -> _Pool:
+        async with self._lock:
+            pool = self._pools.get(name)
+            if pool is None:
+                cfg = self._require_cfg()
+                pool = _Pool(name, cfg, _engine_instances(cfg, name))
+                self._pools[name] = pool
+            return pool
+
+    async def ocr_pages(self, images: List, engine: Optional[str] = None) -> (str, List[OcrPage]):
+        """OCR a list of PIL page images. Returns (engine_name, [OcrPage])."""
+        name = self.resolve_engine(engine)
+        pool = await self._get_pool(name)
+        sem = self._sem or asyncio.Semaphore(4)
+
+        async def _one(idx, img):
+            async with sem:
+                page = await pool.recognize(img)
+                page.index = idx
+                return page
+
+        pages = await asyncio.gather(*[_one(i, im) for i, im in enumerate(images)])
+        return name, list(pages)
+
+    async def ocr_document(self, data: bytes, filename: str = "", content_type: str = "",
+                           engine: Optional[str] = None, dpi: Optional[int] = None,
+                           detect: Optional[str] = None, structured: bool = False,
+                           schema: Optional[str] = None) -> dict:
+        """OCR raw bytes (image/PDF) end-to-end → response dict.
+
+        ``detect`` overrides the configured stamp/signature detection mode
+        (off|layout|detector|both). ``structured`` triggers field extraction via the
+        configured text model; ``schema`` overrides the extraction schema for this call.
+        """
+        cfg = self._require_cfg()
+        use_dpi = int(dpi) if dpi else int(cfg.dpi)
+        images = load_pages(data, filename=filename, content_type=content_type, dpi=use_dpi)
+        name, pages = await self.ocr_pages(images, engine=engine)
+        full_text = "\n\n".join(p.text for p in pages)
+
+        stamps, signatures = await self._detect_all(images, pages, detect)
+
+        structured_out = None
+        if structured:
+            from codai.ocr.extract import extract_fields
+            structured_out = await extract_fields(full_text, cfg, schema=schema)
+
+        return {
+            "engine": name,
+            "num_pages": len(pages),
+            "text": full_text,
+            "pages": [p.to_dict() for p in pages],
+            "stamps": stamps,
+            "signatures": signatures,
+            "structured": structured_out,
+        }
+
+    async def _detect_all(self, images, pages, detect: Optional[str]):
+        """Run stamp/signature detection across pages; returns (stamps, signatures)
+        with each detection tagged by page index. Empty when mode is 'off'."""
+        from codai.ocr.detect import resolve_detect_mode, detect_page
+
+        cfg = self._require_cfg()
+        mode = resolve_detect_mode(cfg, detect)
+        if mode == "off":
+            return [], []
+
+        sem = self._sem or asyncio.Semaphore(4)
+
+        async def _one(idx, img, page):
+            async with sem:
+                res = await asyncio.to_thread(detect_page, img, page, cfg, mode)
+                for d in res["stamps"]:
+                    d["page"] = idx
+                for d in res["signatures"]:
+                    d["page"] = idx
+                return res
+
+        results = await asyncio.gather(*[_one(i, im, pg) for i, (im, pg) in enumerate(zip(images, pages))])
+        stamps, signatures = [], []
+        for r in results:
+            stamps.extend(r["stamps"])
+            signatures.extend(r["signatures"])
+        return stamps, signatures
+
+    def _require_cfg(self):
+        if self._cfg is None:
+            raise OcrError("OCR subsystem is not configured", status=503)
+        if not getattr(self._cfg, "enabled", False):
+            raise OcrError("OCR subsystem is disabled (enable it in Settings → OCR)", status=400)
+        return self._cfg
+
+
+# Module-level singleton (mirrors multi_model_manager).
+ocr_manager = OcrManager()

codai/ocr/paddle.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+
+"""PaddleOCR engine — runs in an ISOLATED venv subprocess.
+
+PaddleOCR pulls opencv-contrib-python (clashes with the main venv's opencv-python/cv2) and
+its paddlepaddle-gpu wheel bundles its own CUDA runtime, so it lives in a dedicated venv
+(``ocr.paddle_venv``) driven by codai/ocr/workers/ocr_worker.py. See
+:mod:`codai.ocr.subprocess_engine`.
+"""
+
+import os
+
+from codai.ocr.subprocess_engine import SubprocessOcrEngine, _resolve_venv_dir
+
+# Repo-root requirements file for the paddle isolated venv (auto-build).
+_REQ = os.path.join(os.path.dirname(__file__), "..", "..", "requirements-ocr-paddle.txt")
+
+
+class PaddleEngine(SubprocessOcrEngine):
+    name = "paddle"
+    _worker_engine = "paddle"
+
+    def _venv_dir(self) -> str:
+        return _resolve_venv_dir(self.cfg.paddle_venv, "paddle_venv")
+
+    def _requirements(self) -> str:
+        return os.path.abspath(_REQ)
+
+    def _auto_build(self) -> bool:
+        return bool(getattr(self.cfg, "paddle_auto_build", False))
+
+    def _worker_opts(self) -> dict:
+        return {
+            "lang": self.cfg.paddle_lang or self.cfg.lang or "it",
+            "use_gpu": bool(self.cfg.paddle_use_gpu),
+            "structure": bool(self.cfg.paddle_structure),
+            "det_model_dir": self.cfg.paddle_det_model_dir or "",
+            "rec_model_dir": self.cfg.paddle_rec_model_dir or "",
+        }

codai/ocr/schemas.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+"""OCR extraction schema store.
+
+Schemas are DATA, not code — so any document type can be supported without touching the
+engine. A schema is resolved three ways (in precedence order), which is how we avoid
+trying to enumerate "every schema out there":
+
+1. **Inline** — the spec string starts with ``{`` / ``[``: used verbatim (a fill-in
+   template or a JSON Schema).
+2. **Named** — a bare name (``italian_sentenza``): looked up in the user schema dir
+   (``<config_dir>/ocr_schemas/*.json``) first, then the built-in seeds shipped with
+   coderai. User files shadow built-ins of the same name.
+3. **Auto** — empty / ``auto`` / ``generic``: no schema; the model is asked to extract
+   the salient key/value fields itself.
+
+A schema file is JSON:
+    {
+      "name": "italian_sentenza",
+      "description": "Italian judicial decision (sentenza)",
+      "language": "it",
+      "type": "template" | "jsonschema",   # template = fill-in example; jsonschema = JSON Schema
+      "schema": { ... },
+      "prompt_hint": "optional extra instruction for the model"
+    }
+"""
+
+import json
+import os
+import re
+from typing import List, Optional, Tuple
+
+from codai.ocr.base import OcrError
+
+_BUILTIN_DIR = os.path.join(os.path.dirname(__file__), "schemas_builtin")
+_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
+_AUTO_NAMES = ("", "auto", "generic")
+
+
+def _user_dir() -> str:
+    """User schema directory: ``<config_dir>/ocr_schemas`` (created on demand)."""
+    base = ""
+    try:
+        from codai.admin.routes import config_manager
+        if config_manager is not None:
+            base = str(getattr(config_manager, "config_dir", "") or "")
+    except Exception:
+        base = ""
+    if not base:
+        base = os.path.expanduser("~/.coderai")
+    return os.path.join(base, "ocr_schemas")
+
+
+def _valid_name(name: str) -> str:
+    if not name or not _NAME_RE.match(name):
+        raise OcrError(
+            "invalid schema name (use letters, digits, '.', '_', '-')", status=400
+        )
+    return name
+
+
+def _read_json(path: str) -> Optional[dict]:
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            return json.load(f)
+    except Exception:
+        return None
+
+
+class SchemaStore:
+    def list(self) -> List[dict]:
+        """Return schema summaries (built-ins + user), user shadowing built-ins."""
+        out = {}
+        for src, d in (("builtin", _BUILTIN_DIR), ("user", _user_dir())):
+            if not os.path.isdir(d):
+                continue
+            for fn in sorted(os.listdir(d)):
+                if not fn.endswith(".json"):
+                    continue
+                data = _read_json(os.path.join(d, fn)) or {}
+                name = data.get("name") or fn[:-5]
+                out[name] = {
+                    "name": name,
+                    "description": data.get("description", ""),
+                    "language": data.get("language", ""),
+                    "type": data.get("type", "template"),
+                    "source": src,
+                }
+        return sorted(out.values(), key=lambda x: x["name"])
+
+    def get(self, name: str) -> dict:
+        """Load a named schema (user dir wins over built-in). Raises 400 if missing."""
+        _valid_name(name)
+        for d in (_user_dir(), _BUILTIN_DIR):
+            path = os.path.join(d, f"{name}.json")
+            if os.path.isfile(path):
+                data = _read_json(path)
+                if data is None:
+                    raise OcrError(f"schema '{name}' is not valid JSON", status=500)
+                return data
+        avail = ", ".join(s["name"] for s in self.list()) or "(none)"
+        raise OcrError(f"unknown schema '{name}'. Available: {avail}", status=400)
+
+    def save(self, name: str, data: dict) -> dict:
+        """Create/overwrite a USER schema (built-ins are read-only)."""
+        _valid_name(name)
+        if not isinstance(data, dict) or "schema" not in data:
+            raise OcrError("schema payload must be an object with a 'schema' field", status=400)
+        data.setdefault("name", name)
+        data.setdefault("type", "template")
+        d = _user_dir()
+        os.makedirs(d, exist_ok=True)
+        with open(os.path.join(d, f"{name}.json"), "w", encoding="utf-8") as f:
+            json.dump(data, f, ensure_ascii=False, indent=2)
+        return data
+
+    def delete(self, name: str) -> None:
+        """Delete a USER schema. Built-ins cannot be deleted."""
+        _valid_name(name)
+        path = os.path.join(_user_dir(), f"{name}.json")
+        if not os.path.isfile(path):
+            raise OcrError(f"user schema '{name}' not found (built-ins are read-only)", status=404)
+        os.remove(path)
+
+    # -- resolution --------------------------------------------------------
+
+    def resolve(self, spec: Optional[str]) -> Tuple[Optional[dict], Optional[str]]:
+        """Resolve a spec → (schema_dict_or_None, error_note).
+
+        Returns ``(None, None)`` for auto/generic mode (no schema). For inline or named
+        schemas returns the loaded/parsed schema dict.
+        """
+        s = (spec or "").strip()
+        if s.lower() in _AUTO_NAMES:
+            return None, None
+        if s[:1] in ("{", "["):
+            try:
+                parsed = json.loads(s)
+            except Exception as e:
+                raise OcrError(f"inline schema is not valid JSON: {e}", status=400)
+            # Wrap a bare schema object into the standard envelope.
+            if isinstance(parsed, dict) and "schema" in parsed and (
+                    "type" in parsed or "name" in parsed):
+                return parsed, None
+            inferred = "jsonschema" if isinstance(parsed, dict) and (
+                "$schema" in parsed or "properties" in parsed) else "template"
+            return {"name": "inline", "type": inferred, "schema": parsed}, None
+        return self.get(s), None
+
+
+schema_store = SchemaStore()

codai/ocr/schemas_builtin/generic_document.json

+{
+  "name": "generic_document",
+  "description": "Generic document metadata + free key/value fields (any language)",
+  "language": "",
+  "type": "template",
+  "prompt_hint": "Add any additional salient fields you find under 'fields' as key/value pairs.",
+  "schema": {
+    "document_type": "string|null",
+    "title": "string|null",
+    "date": "YYYY-MM-DD|null",
+    "authors_or_parties": ["string"],
+    "reference_numbers": ["string"],
+    "summary": "string|null",
+    "fields": {"<detected_field_name>": "<value>"}
+  }
+}

codai/ocr/schemas_builtin/invoice.json

+{
+  "name": "invoice",
+  "description": "Invoice / fattura — demonstrates a strict JSON Schema (validated when jsonschema is installed)",
+  "language": "",
+  "type": "jsonschema",
+  "prompt_hint": "Amounts as numbers without currency symbols; put the currency in 'currency'.",
+  "schema": {
+    "$schema": "http://json-schema.org/draft-07/schema#",
+    "type": "object",
+    "properties": {
+      "invoice_number": {"type": ["string", "null"]},
+      "issue_date": {"type": ["string", "null"], "description": "YYYY-MM-DD"},
+      "seller": {"type": ["string", "null"]},
+      "buyer": {"type": ["string", "null"]},
+      "currency": {"type": ["string", "null"]},
+      "subtotal": {"type": ["number", "null"]},
+      "tax": {"type": ["number", "null"]},
+      "total": {"type": ["number", "null"]},
+      "line_items": {
+        "type": "array",
+        "items": {
+          "type": "object",
+          "properties": {
+            "description": {"type": "string"},
+            "quantity": {"type": ["number", "null"]},
+            "unit_price": {"type": ["number", "null"]},
+            "amount": {"type": ["number", "null"]}
+          }
+        }
+      }
+    }
+  }
+}

codai/ocr/schemas_builtin/italian_sentenza.json

+{
+  "name": "italian_sentenza",
+  "description": "Italian judicial decision (sentenza / provvedimento)",
+  "language": "it",
+  "type": "template",
+  "prompt_hint": "Il testo OCR puo' contenere errori di riconoscimento: correggili implicitamente dove ovvio. Le date in formato ISO YYYY-MM-DD.",
+  "schema": {
+    "tribunale": "string|null",
+    "sezione": "string|null",
+    "numero_sentenza": "string|null",
+    "data_sentenza": "YYYY-MM-DD|null",
+    "tipo_procedimento": "civile|penale|amministrativo|null",
+    "giudice": "string|null",
+    "parti": {
+      "attore_o_ricorrente": "string|null",
+      "convenuto_o_imputato": "string|null",
+      "pubblico_ministero": "string|null"
+    },
+    "oggetto": "string|null",
+    "dispositivo": "string|null",
+    "esito": "string|null",
+    "spese_di_giudizio": "string|null",
+    "riferimenti_normativi": ["string"],
+    "note": "string|null"
+  }
+}

codai/ocr/subprocess_engine.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+
+"""Isolated-venv subprocess OCR engine.
+
+PaddleOCR and Surya have dependencies that conflict with the main coderai venv
+(PaddleOCR pulls opencv-contrib-python; Surya caps pillow<11). Each therefore runs in its
+OWN virtualenv, driven by ``codai/ocr/workers/ocr_worker.py`` over a newline-delimited
+JSON pipe — the same isolation pattern coderai uses for the DINOv2-SALAD embedder.
+
+Isolation bonus for PaddleOCR: paddlepaddle-gpu wheels bundle their own CUDA runtime, so a
+cu12x Paddle wheel runs on GPU under a newer driver (e.g. CUDA-13 host) regardless of the
+main venv's torch CUDA.
+"""
+
+import json
+import os
+import subprocess
+import sys
+import threading
+
+from codai.ocr.base import OcrEngine, OcrPage, OcrLine, OcrRegion, OcrError
+
+_WORKER = os.path.join(os.path.dirname(__file__), "workers", "ocr_worker.py")
+
+
+def _resolve_venv_dir(configured: str, name: str) -> str:
+    """Resolve an isolated-venv directory for an OCR engine.
+
+    Precedence: explicit config path > a venv BAKED into the image
+    (/opt/coderai/<name>) > a build on the PERSISTENT cache mount
+    (CODERAI_CACHE_DIR or /cache → <cache>/ocr/<name>, survives container restarts) >
+    ~/.coderai/<name> for host/source installs.
+
+    The order matters: a runtime-built venv must NOT land in the image's writable layer
+    (lost on `coderai docker` restart) — it goes on the /cache mount, like the colibri/k3
+    engines.
+    """
+    if configured:
+        return configured
+    baked = f"/opt/coderai/{name}"
+    if os.path.isdir(baked):
+        return baked
+    cache = os.environ.get("CODERAI_CACHE_DIR") or ("/cache" if os.path.isdir("/cache") else "")
+    if cache and os.path.isdir(cache):
+        return os.path.join(cache, "ocr", name)
+    return os.path.expanduser(f"~/.coderai/{name}")
+
+
+class SubprocessOcrEngine(OcrEngine):
+    """Base for engines that run in an isolated venv subprocess."""
+
+    # Subclasses set these:
+    name = "subprocess"
+    _worker_engine = ""          # "paddle" | "surya"
+
+    def __init__(self, cfg):
+        super().__init__(cfg)
+        self._proc = None
+        self._lock = threading.Lock()
+
+    # -- subclass hooks ----------------------------------------------------
+
+    def _venv_dir(self) -> str:
+        raise NotImplementedError
+
+    def _requirements(self) -> str:
+        """Path to the requirements file for auto-build (or '' to disable auto-build)."""
+        return ""
+
+    def _auto_build(self) -> bool:
+        return False
+
+    def _worker_opts(self) -> dict:
+        return {}
+
+    def _pip_install_cmd(self, py: str, req: str):
+        """Return the pip command list to build the venv. Overridable (e.g. paddle index)."""
+        return [py, "-m", "pip", "install", "-r", req]
+
+    # -- lifecycle ---------------------------------------------------------
+
+    def _venv_python(self) -> str:
+        return os.path.join(os.path.expanduser(self._venv_dir()), "bin", "python")
+
+    def _ensure_venv(self) -> str:
+        py = self._venv_python()
+        if os.path.isfile(py):
+            return py
+        if not self._auto_build():
+            raise OcrError(
+                f"{self.name}: isolated venv not found at {os.path.dirname(os.path.dirname(py))}. "
+                f"Create it (python3 -m venv <dir> && <dir>/bin/pip install -r "
+                f"{os.path.basename(self._requirements())}) or enable auto-build.",
+                status=503,
+            )
+        venv_dir = os.path.expanduser(self._venv_dir())
+        req = self._requirements()
+        if not req or not os.path.isfile(req):
+            raise OcrError(f"{self.name}: requirements file not found ({req})", status=500)
+        try:
+            print(f"[ocr.{self.name}] creating isolated venv at {venv_dir} …", flush=True)
+            os.makedirs(os.path.dirname(venv_dir) or ".", exist_ok=True)
+            subprocess.run([sys.executable, "-m", "venv", venv_dir], check=True)
+            subprocess.run([py, "-m", "pip", "install", "-U", "pip"], check=True)
+            subprocess.run(self._pip_install_cmd(py, req), check=True)
+        except subprocess.CalledProcessError as e:
+            raise OcrError(f"{self.name}: venv build failed: {e}", status=500)
+        if not os.path.isfile(py):
+            raise OcrError(f"{self.name}: venv python missing after build", status=500)
+        return py
+
+    def load(self) -> None:
+        py = self._ensure_venv()
+        env = dict(os.environ)
+        try:
+            self._proc = subprocess.Popen(
+                [py, _WORKER, self._worker_engine],
+                stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                stderr=None, text=True, bufsize=1, env=env,
+            )
+        except Exception as e:
+            raise OcrError(f"{self.name}: failed to launch worker: {e}", status=500)
+        # Handshake: wait for {"ready": true}
+        ready = self._readline()
+        if not ready.get("ready"):
+            raise OcrError(f"{self.name}: worker did not signal ready", status=500)
+        resp = self._rpc({"cmd": "load", "opts": self._worker_opts()})
+        if not resp.get("ok"):
+            self._kill()
+            raise OcrError(resp.get("error") or f"{self.name}: load failed",
+                           status=int(resp.get("status", 500)))
+
+    def _readline(self) -> dict:
+        line = self._proc.stdout.readline()
+        if not line:
+            raise OcrError(f"{self.name}: worker closed the pipe", status=500)
+        try:
+            return json.loads(line)
+        except Exception as e:
+            raise OcrError(f"{self.name}: bad worker output: {e}", status=500)
+
+    def _rpc(self, msg: dict) -> dict:
+        if self._proc is None or self._proc.poll() is not None:
+            raise OcrError(f"{self.name}: worker not running", status=500)
+        self._proc.stdin.write(json.dumps(msg) + "\n")
+        self._proc.stdin.flush()
+        return self._readline()
+
+    def _kill(self):
+        try:
+            if self._proc:
+                self._proc.kill()
+        except Exception:
+            pass
+        self._proc = None
+
+    def recognize_image(self, image) -> OcrPage:
+        import base64
+        import io
+        buf = io.BytesIO()
+        image.convert("RGB").save(buf, "PNG")
+        b64 = base64.b64encode(buf.getvalue()).decode("ascii")
+        with self._lock:                       # one request at a time per subprocess
+            resp = self._rpc({"cmd": "ocr", "image": b64})
+        if not resp.get("ok"):
+            raise OcrError(resp.get("error") or f"{self.name}: ocr failed",
+                           status=int(resp.get("status", 500)))
+        return self._page_from_dict(resp.get("page") or {})
+
+    @staticmethod
+    def _page_from_dict(d: dict) -> OcrPage:
+        page = OcrPage(index=int(d.get("index", 0)), text=d.get("text", ""),
+                       width=int(d.get("width", 0)), height=int(d.get("height", 0)))
+        page.lines = [OcrLine(text=l.get("text", ""), bbox=list(l.get("bbox", [0, 0, 0, 0])),
+                              conf=float(l.get("conf", 0.0))) for l in d.get("lines", [])]
+        page.regions = [OcrRegion(label=r.get("label", ""), bbox=list(r.get("bbox", [0, 0, 0, 0])),
+                                  conf=float(r.get("conf", 0.0))) for r in d.get("regions", [])]
+        page.tables = list(d.get("tables", []))
+        return page

codai/ocr/surya.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+
+"""Surya OCR engine — runs in an ISOLATED venv subprocess.
+
+Surya caps pillow<11, which conflicts with the main coderai venv (pillow>=12), so it lives
+in a dedicated venv (``ocr.surya_venv``) driven by codai/ocr/workers/ocr_worker.py. See
+:mod:`codai.ocr.subprocess_engine`.
+
+LICENSE: GPL (compatible with coderai's GPLv3). Enabled only when
+``ocr.surya_accept_license`` is True (enforced by the manager before this is constructed).
+"""
+
+import os
+
+from codai.ocr.subprocess_engine import SubprocessOcrEngine, _resolve_venv_dir
+from codai.ocr.base import OcrError
+
+_REQ = os.path.join(os.path.dirname(__file__), "..", "..", "requirements-surya.txt")
+
+
+class SuryaEngine(SubprocessOcrEngine):
+    name = "surya"
+    _worker_engine = "surya"
+
+    def _venv_dir(self) -> str:
+        return _resolve_venv_dir(self.cfg.surya_venv, "surya_venv")
+
+    def _requirements(self) -> str:
+        return os.path.abspath(_REQ)
+
+    def _auto_build(self) -> bool:
+        return bool(getattr(self.cfg, "surya_auto_build", False))
+
+    def _worker_opts(self) -> dict:
+        return {"langs": self.cfg.surya_langs or "it"}
+
+    def load(self) -> None:
+        if not self.cfg.surya_accept_license:
+            raise OcrError(
+                "Surya is license-gated. Set ocr.surya_accept_license = true to use it.",
+                status=400,
+            )
+        super().load()

codai/ocr/venv_build.py

+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+
+"""Background builder for the isolated OCR engine venvs (Paddle / Surya).
+
+Building a venv (python -m venv + pip install torch/paddle/surya) takes minutes, so the
+web UI triggers it asynchronously and polls status. Also usable headless. The engine
+classes own the venv dir / requirements / pip command, so this just drives them.
+"""
+
+import os
+import subprocess
+import sys
+import threading
+from collections import deque
+from typing import Optional
+
+# engine -> {"state": idle|running|done|error, "log": [..], "returncode": int|None}
+_STATE = {}
+_LOCK = threading.Lock()
+_LOG_MAX = 400
+
+
+def _engine_for(cfg, engine: str):
+    engine = (engine or "").strip().lower()
+    if engine == "paddle":
+        from codai.ocr.paddle import PaddleEngine
+        return PaddleEngine(cfg)
+    if engine == "surya":
+        from codai.ocr.surya import SuryaEngine
+        return SuryaEngine(cfg)
+    raise ValueError(f"no isolated venv for engine '{engine}' (only paddle/surya)")
+
+
+def status(engine: str) -> dict:
+    with _LOCK:
+        st = _STATE.get(engine)
+        if not st:
+            return {"engine": engine, "state": "idle", "log": [], "returncode": None}
+        return {"engine": engine, "state": st["state"],
+                "log": list(st["log"])[-40:], "returncode": st["returncode"]}
+
+
+def is_running(engine: str) -> bool:
+    with _LOCK:
+        st = _STATE.get(engine)
+        return bool(st and st["state"] == "running")
+
+
+def build_async(cfg, engine: str) -> dict:
+    """Kick off a background build for the engine's isolated venv. Idempotent while running."""
+    engine = (engine or "").strip().lower()
+    eng = _engine_for(cfg, engine)          # validates engine name early
+    if is_running(engine):
+        return {"engine": engine, "state": "running", "already": True}
+
+    venv_dir = os.path.expanduser(eng._venv_dir())
+    req = eng._requirements()
+    py = os.path.join(venv_dir, "bin", "python")
+
+    with _LOCK:
+        _STATE[engine] = {"state": "running", "log": deque(maxlen=_LOG_MAX), "returncode": None}
+
+    def _log(line):
+        with _LOCK:
+            _STATE[engine]["log"].append(line.rstrip("\n"))
+
+    def _run():
+        try:
+            if not req or not os.path.isfile(req):
+                raise RuntimeError(f"requirements file not found: {req}")
+            _log(f"[venv] target {venv_dir}")
+            if not os.path.isfile(py):
+                os.makedirs(os.path.dirname(venv_dir) or ".", exist_ok=True)
+                _log("[venv] creating virtualenv…")
+                _stream([sys.executable, "-m", "venv", venv_dir], _log)
+            _log("[pip] upgrading pip…")
+            _stream([py, "-m", "pip", "install", "-U", "pip"], _log)
+            _log(f"[pip] installing from {os.path.basename(req)} (this can take several minutes)…")
+            _stream(eng._pip_install_cmd(py, req), _log)
+            # sanity: the worker can import its engine
+            _log("[check] importing engine in the new venv…")
+            mod = "paddleocr" if engine == "paddle" else "surya"
+            _stream([py, "-c", f"import {mod}; print('{mod} import OK')"], _log)
+            with _LOCK:
+                _STATE[engine]["state"] = "done"; _STATE[engine]["returncode"] = 0
+            _log("[done] venv ready")
+        except subprocess.CalledProcessError as e:
+            _log(f"[error] command failed (rc={e.returncode})")
+            with _LOCK:
+                _STATE[engine]["state"] = "error"; _STATE[engine]["returncode"] = e.returncode
+        except Exception as e:
+            _log(f"[error] {e}")
+            with _LOCK:
+                _STATE[engine]["state"] = "error"; _STATE[engine]["returncode"] = -1
+
+    threading.Thread(target=_run, name=f"ocr-venv-build-{engine}", daemon=True).start()
+    return {"engine": engine, "state": "running", "venv_dir": venv_dir}
+
+
+def _stream(cmd, log_fn):
+    """Run cmd, streaming combined output to log_fn; raise CalledProcessError on failure."""
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1)
+    for line in p.stdout:
+        log_fn(line)
+    rc = p.wait()
+    if rc != 0:
+        raise subprocess.CalledProcessError(rc, cmd)

codai/ocr/workers/__init__.py

+# OCR isolated-venv subprocess workers (self-contained; no coderai imports).

codai/ocr/workers/ocr_worker.py

+#!/usr/bin/env python3
+# CoderAI - OpenAI-compatible API server
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+# GPLv3 - see the project LICENSE.
+#
+# Isolated-venv OCR worker. Runs INSIDE a dedicated virtualenv (PaddleOCR or Surya) so
+# their conflicting deps (opencv-contrib / pillow<11) never touch the main coderai venv.
+# SELF-CONTAINED: imports only stdlib + PIL + the one OCR engine present in this venv.
+# Never import coderai here.
+#
+# Protocol: newline-delimited JSON on stdin/stdout.
+#   startup            → worker prints {"ready": true}
+#   {"cmd":"load","opts":{...}}   → {"ok":true} | {"ok":false,"error":..,"status":..}
+#   {"cmd":"ocr","image":"<b64 png>"} → {"ok":true,"page":{...}} | {"ok":false,...}
+#   {"cmd":"ping"}     → {"ok":true}
+# argv[1] = engine name ("paddle" | "surya").
+
+import base64
+import io
+import json
+import sys
+
+
+def _emit(obj):
+    sys.stdout.write(json.dumps(obj) + "\n")
+    sys.stdout.flush()
+
+
+def _poly_to_xyxy(poly):
+    try:
+        pts = list(poly)
+        if len(pts) == 4 and all(isinstance(p, (int, float)) for p in pts):
+            return [float(pts[0]), float(pts[1]), float(pts[2]), float(pts[3])]
+        xs = [float(p[0]) for p in pts]
+        ys = [float(p[1]) for p in pts]
+        return [min(xs), min(ys), max(xs), max(ys)]
+    except Exception:
+        return [0.0, 0.0, 0.0, 0.0]
+
+
+def _lines_to_text(lines):
+    if not lines:
+        return ""
+    ordered = sorted(lines, key=lambda l: (l["bbox"][1], l["bbox"][0]))
+    rows = []
+    for ln in ordered:
+        y0 = ln["bbox"][1]
+        placed = False
+        for row in rows:
+            ry = row[0]["bbox"][1]
+            rh = max(1.0, row[0]["bbox"][3] - row[0]["bbox"][1])
+            if abs(y0 - ry) <= 0.6 * rh:
+                row.append(ln); placed = True; break
+        if not placed:
+            rows.append([ln])
+    out = []
+    for row in rows:
+        row.sort(key=lambda l: l["bbox"][0])
+        out.append(" ".join(l["text"] for l in row if l["text"]))
+    return "\n".join(out)
+
+
+# ---------------------------------------------------------------------------
+# PaddleOCR
+# ---------------------------------------------------------------------------
+
+class PaddleWorker:
+    def __init__(self, opts):
+        self.opts = opts
+        self._ocr = None
+        self._structure = None
+
+    def load(self):
+        from paddleocr import PaddleOCR
+        o = self.opts
+        kw = dict(lang=o.get("lang", "it"), use_angle_cls=True, show_log=False,
+                  use_gpu=bool(o.get("use_gpu", True)))
+        if o.get("det_model_dir"):
+            kw["det_model_dir"] = o["det_model_dir"]
+        if o.get("rec_model_dir"):
+            kw["rec_model_dir"] = o["rec_model_dir"]
+        self._ocr = _construct_tolerant(PaddleOCR, kw)
+        if o.get("structure", True):
+            try:
+                from paddleocr import PPStructure
+                self._structure = _construct_tolerant(
+                    PPStructure, dict(show_log=False, lang=o.get("lang", "it"),
+                                      use_gpu=bool(o.get("use_gpu", True))))
+            except Exception:
+                self._structure = None
+
+    def ocr(self, img):
+        import numpy as np
+        rgb = np.array(img.convert("RGB"))
+        bgr = rgb[:, :, ::-1]
+        h, w = rgb.shape[0], rgb.shape[1]
+        lines = []
+        for box, text, conf in self._run(bgr):
+            lines.append({"text": text, "bbox": _poly_to_xyxy(box), "conf": conf})
+        regions, tables = [], []
+        if self._structure is not None:
+            try:
+                regions, tables = self._run_structure(bgr)
+            except Exception:
+                pass
+        return {"index": 0, "width": int(w), "height": int(h),
+                "lines": lines, "regions": regions, "tables": tables,
+                "text": _lines_to_text(lines)}
+
+    def _run(self, bgr):
+        out = []
+        ocr = self._ocr
+        raw = None
+        if hasattr(ocr, "ocr"):
+            try:
+                raw = ocr.ocr(bgr, cls=True)
+            except TypeError:
+                raw = ocr.ocr(bgr)
+        elif hasattr(ocr, "predict"):
+            raw = ocr.predict(bgr)
+        if not raw:
+            return out
+        for page in raw:
+            if not page:
+                continue
+            if isinstance(page, dict):
+                texts = page.get("rec_texts") or []
+                polys = page.get("dt_polys") or page.get("rec_polys") or []
+                scores = page.get("rec_scores") or []
+                for i, t in enumerate(texts):
+                    box = polys[i] if i < len(polys) else [0, 0, 0, 0]
+                    conf = float(scores[i]) if i < len(scores) else 0.0
+                    out.append((box, str(t), conf))
+                continue
+            for item in page:
+                try:
+                    box = item[0]; txt = item[1]
+                    if isinstance(txt, (list, tuple)):
+                        out.append((box, str(txt[0]), float(txt[1])))
+                    else:
+                        out.append((box, str(txt), 0.0))
+                except Exception:
+                    continue
+        return out
+
+    def _run_structure(self, bgr):
+        regions, tables = [], []
+        for reg in (self._structure(bgr) or []):
+            try:
+                label = str(reg.get("type", "region"))
+                bbox = [float(v) for v in reg.get("bbox", [0, 0, 0, 0])]
+                regions.append({"label": label, "bbox": bbox, "conf": float(reg.get("score", 0.0))})
+                if label.lower() == "table":
+                    r = reg.get("res")
+                    tables.append({"bbox": bbox, "html": r.get("html") if isinstance(r, dict) else None})
+            except Exception:
+                continue
+        return regions, tables
+
+
+def _construct_tolerant(cls, kwargs):
+    try:
+        return cls(**kwargs)
+    except TypeError:
+        k = dict(kwargs)
+        for bad in list(k.keys()):
+            try:
+                return cls(**k)
+            except TypeError:
+                k.pop(bad, None)
+        return cls()
+
+
+# ---------------------------------------------------------------------------
+# Surya
+# ---------------------------------------------------------------------------
+
+class SuryaWorker:
+    def __init__(self, opts):
+        self.opts = opts
+        self._mode = None
+        self._rec = None
+        self._det = None
+        self._det_proc = None
+        self._rec_proc = None
+        self._run_ocr = None
+        self._langs = [l.strip() for l in str(opts.get("langs", "it")).split(",") if l.strip()] or ["it"]
+
+    def load(self):
+        try:
+            from surya.recognition import RecognitionPredictor
+            from surya.detection import DetectionPredictor
+            self._rec = RecognitionPredictor()
+            self._det = DetectionPredictor()
+            self._mode = "predictor"
+            return
+        except Exception:
+            pass
+        from surya.ocr import run_ocr
+        from surya.model.detection.model import (
+            load_model as ldm, load_processor as ldp)
+        from surya.model.recognition.model import load_model as lrm
+        from surya.model.recognition.processor import load_processor as lrp
+        self._run_ocr = run_ocr
+        self._det = ldm(); self._det_proc = ldp()
+        self._rec = lrm(); self._rec_proc = lrp()
+        self._mode = "run_ocr"
+
+    def ocr(self, img):
+        img = img.convert("RGB")
+        if self._mode == "predictor":
+            preds = self._rec([img], [self._langs], self._det)
+        else:
+            preds = self._run_ocr([img], [self._langs], self._det, self._det_proc, self._rec, self._rec_proc)
+        res = preds[0] if preds else None
+        lines = []
+        tl = getattr(res, "text_lines", None)
+        if tl is None and isinstance(res, dict):
+            tl = res.get("text_lines")
+        for t in (tl or []):
+            text = getattr(t, "text", None)
+            bbox = getattr(t, "bbox", None)
+            conf = getattr(t, "confidence", 0.0)
+            if text is None and isinstance(t, dict):
+                text = t.get("text"); bbox = t.get("bbox"); conf = t.get("confidence", 0.0)
+            if not text:
+                continue
+            bb = [float(v) for v in (bbox or [0, 0, 0, 0])][:4]
+            if len(bb) < 4:
+                bb = [0.0, 0.0, 0.0, 0.0]
+            lines.append({"text": str(text), "bbox": bb, "conf": float(conf or 0.0)})
+        return {"index": 0, "width": img.width, "height": img.height,
+                "lines": lines, "regions": [], "tables": [], "text": "\n".join(l["text"] for l in lines)}
+
+
+# ---------------------------------------------------------------------------
+# Main loop
+# ---------------------------------------------------------------------------
+
+def main():
+    engine = sys.argv[1] if len(sys.argv) > 1 else "paddle"
+    from PIL import Image  # noqa: F401 (ensure Pillow present before READY)
+    _emit({"ready": True})
+
+    worker = None
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            msg = json.loads(line)
+        except Exception as e:
+            _emit({"ok": False, "error": f"bad json: {e}"}); continue
+        cmd = msg.get("cmd")
+        if cmd == "ping":
+            _emit({"ok": True}); continue
+        if cmd == "load":
+            try:
+                opts = msg.get("opts") or {}
+                worker = PaddleWorker(opts) if engine == "paddle" else SuryaWorker(opts)
+                worker.load()
+                _emit({"ok": True})
+            except Exception as e:
+                _emit({"ok": False, "error": f"{engine} load failed: {e}", "status": 503})
+            continue
+        if cmd == "ocr":
+            if worker is None:
+                _emit({"ok": False, "error": "not loaded", "status": 500}); continue
+            try:
+                from PIL import Image
+                data = base64.b64decode(msg["image"])
+                img = Image.open(io.BytesIO(data)); img.load()
+                _emit({"ok": True, "page": worker.ocr(img)})
+            except Exception as e:
+                _emit({"ok": False, "error": f"{engine} ocr failed: {e}", "status": 500})
+            continue
+        if cmd == "shutdown":
+            _emit({"ok": True}); return
+        _emit({"ok": False, "error": f"unknown cmd {cmd}"})
+
+
+if __name__ == "__main__":
+    main()

docs/kimi-k3.md

+# Kimi-K3 via kimi-k3-in-c (`k3` backend)
+
+CoderAI can serve **Kimi-K3** (2.78T params, 16/896 experts active, 93 layers) through
+[FareedKhan-dev/kimi-k3-in-c](https://github.com/FareedKhan-dev/kimi-k3-in-c), a
+portable-C CPU engine that streams the always-on dense trunk + routed experts from disk,
+running the full model in as little as ~8 GB RAM (more RAM = faster, not "more capable").
+
+Upstream `kimi-k3-in-c` is a **one-shot batch CLI**. CoderAI applies
+`packaging/patch-k3.py` to add a **resident serve loop** that speaks the same mux
+stdin/stdout protocol as [colibri](glm-colibri.md), so the same in-process
+`MuxEngine` client drives it — the model, packed trunk, expert cache and tokenizer stay
+warm across requests (no ~108 GB cold-start trunk read per prompt).
+
+## Requirements
+
+- **CPU**: x86-64 with **AVX2 + FMA** (no AVX-512 required).
+- **Storage**: ~1.7 TB fast local storage — the ~1.56 TB HF checkpoint **plus** a
+  ~109 GB packed dense trunk produced by the repo's `scripts/pack-trunk.sh`.
+- **RAM**: 8 GB minimum; pin more of the trunk for speed (see `trunk_gb` / presets).
+- CoderAI never downloads the checkpoint — point the config at it.
+
+## Enable & configure
+
+Settings → **Kimi-K3 (kimi-k3-in-c)** card, or `config.json` `"k3"`:
+
+| field | meaning |
+|---|---|
+| `enabled` | turn the engine on |
+| `model_path` | the Kimi-K3 checkpoint directory (config.json + tokenizer + shards) |
+| `trunk_dir` | packed dense trunk dir (`scripts/pack-trunk.sh`); blank = fully resident (~114 GB RAM) |
+| `tok_dir` | tokenizer dir; blank = use the checkpoint dir |
+| `model_id` | id/alias that routes here (default `kimi-k3`) |
+| `preset` | `laptop`/`desktop`/`workstation`/`server`/`max` (blank = use `trunk_gb`/`cache_gb`) |
+| `trunk_gb` / `cache_gb` | GiB pinned of the trunk / expert LRU cache |
+| `ctx` | serve context capacity (engine `K3_MAXT`) |
+| `extra_env` | `K3_EXPERT_GB`, `K3_BITS`, `K3_DIRECT`, `K3_PIPE`, … |
+| `auto_build` | clone + patch + `make` the engine binary on first use |
+
+First use clones the repo, applies the serve-loop patch, and builds `bin/k3`
+(`build.sh --k3` prebuilds it for packaging).
+
+## Routing
+
+A request routes to `k3` when (highest precedence first):
+
+1. the model's models.json entry has `backend: "k3"`, **or**
+2. `k3` is enabled and the name equals its `model_id` alias, **or**
+3. the name contains a Kimi-K3 marker (`kimi-k3`) **and** only one Kimi-capable engine
+   is enabled.
+
+If **both** `k3` and `colibri` (which also serves Kimi-K3) are enabled, a bare Kimi
+name is **ambiguous** — set the model's `backend` to choose. See
+[`resolve_engine_backend`](../codai/models/manager.py).
+
+## Notes
+
+- **CPU-only** (no CUDA/GPU tier here) — for GPU Kimi-K3, use colibri's `kimi_k3`
+  engine (optional Vulkan tier) or ktransformers instead.
+- Greedy incremental decode, single conversation at a time (single KV slot); KV /
+  recurrent state is reset between requests (stateless HTTP turns).
+- The Kimi-K3 chat template is rendered as an XTML string
+  (`render_kimi_xtml`, `codai/backends/colibri_families.py`) and tokenized by the engine.

docs/ktransformers.md

+# ktransformers via SGLang (`kt` backend)
+
+CoderAI can serve large MoE models through
+[ktransformers](https://github.com/kvcache-ai/ktransformers) (KVCache.AI, Apache-2.0), a
+**CPU+GPU heterogeneous** inference engine — Intel AMX/AVX512/AVX2 CPU kernels for the
+quantized experts + GPU for the dense trunk/attention, with NUMA-aware expert placement
+and disk offload. It exposes an OpenAI-compatible HTTP server via **SGLang**, so — like
+[ds4](deepseek-ds4.md) — CoderAI launches it as a managed subprocess and proxies to it.
+
+One kt backend can serve many families: **DeepSeek-V3/R1/V4, Kimi-K2/K2.5, Qwen3,
+GLM-5/5.2, MiniMax**. Because it overlaps every other engine, it is selected **only** by
+an explicit `backend: "kt"` pin or its configured `model_id` alias — never by a broad
+name marker (that would collide with ds4/colibri/k3).
+
+## Requirements
+
+- **SGLang + kt-kernel installed out of band** (kt-kernel is a native build). CoderAI
+  does not compile it per-request; with `auto_build` it attempts `pip install sglang`,
+  but kt-kernel must be provided. Best throughput needs **AMX / AVX-512** CPUs.
+- The HF model directory + the KT quantized weights (not downloaded by CoderAI).
+
+## Enable & configure
+
+Settings → **ktransformers (SGLang)** card, or `config.json` `"ktransformers"`:
+
+| field | meaning |
+|---|---|
+| `enabled` | turn the engine on |
+| `model_path` | HF model directory (SGLang `--model`) |
+| `kt_weight_path` | KT quantized weights directory (`--kt-weight-path`) |
+| `model_id` | id/alias that routes here + SGLang `--served-model-name` (default `ktransformers`) |
+| `host` / `port` | SGLang bind host / port (`0` = auto-pick a free port) |
+| `ctx` | context length (`--context-length`) |
+| `extra_args` | extra `sglang.launch_server` flags (e.g. `--tp-size 1`) |
+| `extra_env` | free-form `KEY=VALUE` env for the subprocess |
+| `auto_build` | attempt `pip install sglang` if missing (heavy; off by default) |
+
+On first use CoderAI runs, roughly:
+
+```
+python -m sglang.launch_server --host <h> --port <p> --model <model_path> \
+       --kt-weight-path <kt_weight_path> --served-model-name <model_id> \
+       --context-length <ctx> [extra_args]
+```
+
+then health-checks `/v1/models` and proxies `/v1/chat/completions` to it.
+
+## Routing
+
+Route to `kt` only via an explicit `backend: "kt"` model entry or a request whose model
+matches the kt `model_id`. To run, e.g., DeepSeek-V4 on ktransformers instead of ds4, add
+a models.json entry `{ "path": "…", "backend": "kt", "alias": "deepseek-v4-kt" }`.
+
+## Notes
+
+- Heaviest to set up of the four engines, but a single maintained backend covering many
+  families — a good long-term option where AMX/multi-GPU serving is available.
+- Managed like ds4: the subprocess is torn down on model eviction / shutdown.

docs/ocr.md

+# OCR subsystem (dedicated OCR engines)
+
+CoderAI OCRs documents with **purpose-built OCR engines** (detection + recognition) —
+**not** a vision LLM. That means faithful transcription with per-line bounding boxes and
+layout, GPU batching, and no hallucinated text. Three engines are integrated and
+selectable per request:
+
+| engine | id | license | how it runs |
+|---|---|---|---|
+| [docTR](https://github.com/mindee/doctr) | `doctr` | Apache-2.0 | **in-process** (uses the main venv's torch); works on GPU. Recommended on new-CUDA boxes. |
+| [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) + PP-Structure | `paddle` | Apache-2.0 | **isolated venv subprocess** (layout + tables) |
+| [Surya](https://github.com/VikParuchuri/surya) | `surya` | GPL (compatible with coderai's GPLv3) | **isolated venv subprocess**; best layout/reading-order; opt-in |
+
+Per document you get three outputs: **plain text**, **structured JSON fields**, and
+**stamp/signature flags**.
+
+## Why isolated venvs for Paddle & Surya
+
+Their dependencies conflict with the main coderai venv, so each runs in its **own
+virtualenv + subprocess worker** (`codai/ocr/workers/ocr_worker.py`, JSON over a pipe) —
+the same isolation pattern coderai uses for the DINOv2-SALAD embedder:
+
+- **PaddleOCR** pulls `opencv-contrib-python` (clashes with `opencv-python`/`cv2`), and
+  `paddlepaddle-gpu` wheels **bundle their own CUDA runtime** — so a cu12x Paddle wheel
+  runs on GPU even when the host is on **newer CUDA (e.g. CUDA 13)**, as long as the driver
+  is recent. Isolation makes both facts safe.
+- **Surya** caps `pillow<11`, incompatible with the main venv's `pillow>=12`.
+
+docTR is torch-based and compatible, so it stays in-process.
+
+## Install
+
+**Main venv (docTR + detection + PDF + validation):**
+```
+pip install -r requirements-ocr.txt          # or: ./build.sh nvidia --ocr
+```
+`pypdfium2` is already in the base requirements, so image OCR works even without this. Any
+engine whose dependency is missing returns **HTTP 503** instead of crashing.
+
+**PaddleOCR isolated venv** (`ocr.paddle_venv`, default `~/.coderai/paddle_venv`):
+```
+python3 -m venv ~/.coderai/paddle_venv
+~/.coderai/paddle_venv/bin/pip install -r requirements-ocr-paddle.txt   # GPU wheel via Paddle's index
+```
+**Surya isolated venv** (`ocr.surya_venv`, default `~/.coderai/surya_venv`):
+```
+python3 -m venv ~/.coderai/surya_venv
+~/.coderai/surya_venv/bin/pip install -r requirements-surya.txt
+```
+Or set `paddle_auto_build` / `surya_auto_build` to have coderai create the venv and install
+on first use. In the OCI image these venvs are **baked in** at `/opt/coderai/paddle_venv` /
+`/opt/coderai/surya_venv` (same as the `lipsync_venv` / `parler-venv`), so the image is
+self-contained; the engines prefer a baked venv, then `~/.coderai/<name>_venv`, then the
+config path. Surya loads only when `surya_accept_license` is set.
+
+## Enable & configure
+
+Settings → **OCR** card, or `config.json` `"ocr"`. Key fields: `enabled`,
+`default_engine`, `dpi`, `max_concurrency`, `lang`; per-engine `*_enabled` /
+`*_instances` / `*_use_gpu` / lang; `detect_mode` (+ `detect_model_path`, `detect_conf`);
+`extract_enabled` / `extract_model_id` / `extract_schema` / `extract_validate`.
+
+**Concurrency on one GPU** comes from loading multiple instances of an engine
+(`paddle_instances` etc.) and fanning pages across them (bounded by `max_concurrency`).
+OCR models are small, so a 24 GB card (e.g. RTX 3090) fits several instances — good
+aggregate throughput without vLLM (a continuous-batching path is a future option; see
+[vllm.md](vllm.md)).
+
+## Endpoints
+
+```
+POST /v1/ocr            multipart: file=@doc.pdf [engine=] [dpi=] [detect=] [structured=] [schema=]
+POST /v1/ocr/batch      multipart: files=@a.pdf files=@b.png [...same fields...]
+GET/POST/DELETE /v1/ocr/schemas[/{name}]   manage extraction schemas
+```
+
+Response: `{engine, num_pages, text, pages[], stamps[], signatures[], structured}`. Each
+page carries `lines[]` (text + bbox + conf), `regions[]` (layout), and `tables[]`. The
+`ocr` **pipeline step type** chains OCR with `text_gen` in custom pipelines.
+
+```bash
+curl -F file=@sentenza.pdf -F structured=true -F detect=both -F schema=italian_sentenza \
+     http://localhost:8776/v1/ocr
+```
+
+## Stamp / signature detection (`detect_mode`)
+
+- `off` — none.
+- `layout` — flags the engine's layout regions (figure/seal/stamp/signature markers,
+  EN + IT: *timbro/sigillo/firma*). Free but coarse.
+- `detector` — a small **YOLO** model (`detect_model_path`, needs `ultralytics`) trained
+  to spot signatures/stamps; tight boxes + confidence, bucketed into `stamps` /
+  `signatures`.
+- `both` — layout + detector.
+
+Detection *locates/flags* stamps and signatures — it does not *verify* a signature.
+
+## Structured extraction — schemas are data, not code
+
+Extraction feeds the OCR text to an existing coderai text model
+(`extract_model_id`). The target schema is **not hardcoded**; it is resolved three ways:
+
+1. **Inline** — the `schema` field starts with `{`/`[`: a fill-in template or a JSON
+   Schema, used verbatim.
+2. **Named** — a bare name (`italian_sentenza`): looked up in
+   `<config_dir>/ocr_schemas/*.json` (user schemas) then the built-in seeds; user files
+   shadow built-ins.
+3. **Auto** — empty / `auto`: the model extracts salient key/value fields itself.
+
+Built-in seeds: `italian_sentenza` (Italian judicial decision, template),
+`generic_document` (template), `invoice` (JSON Schema, validated when `jsonschema` is
+installed). Manage your own from the **Settings → OCR** card (picker + raw-JSON editor +
+field builder) or the `/v1/ocr/schemas` API. A `jsonschema`-typed schema is validated
+against the model output when `extract_validate` is on (`_validation_error` is surfaced,
+never fatal).
+
+Adding a new document type — *atto di citazione*, *decreto ingiuntivo*, contracts, … — is
+just dropping a JSON file in `ocr_schemas/` (or POSTing it); no code change.

docs/vllm.md

+# vLLM backend (future option)
+
+> **Status: NOT YET IMPLEMENTED — design note / decision record.**
+> For now, parallel throughput is achieved with **multiple llama.cpp instances**
+> of the same model (see the manager's multi-instance loading). This document
+> captures why and how we would add a vLLM backend later.
+
+## Why add vLLM
+
+coderai's current inference backends serve **one request at a time per loaded
+instance** — the Vulkan/CUDA llama.cpp path holds a per-instance generation lock,
+and the big-MoE engines (ds4/colibri/k3/kt) are single-stream serve loops. To run
+many requests concurrently today you must load **several instances** of a model and
+let the manager fan requests across them. That works but wastes VRAM (N full copies)
+and gives sub-linear throughput.
+
+[vLLM](https://github.com/vllm-project/vllm) provides **continuous batching + paged
+KV cache**: a single loaded model interleaves many in-flight requests, so aggregate
+tokens/sec on one GPU (e.g. an RTX 3090, 24 GB) far exceeds N serialized llama.cpp
+instances for the same VRAM. It exposes an **OpenAI-compatible HTTP server**
+(`vllm serve` / `python -m vllm.entrypoints.openai.api_server`), so it slots into
+coderai exactly like the existing **ds4 / ktransformers** HTTP-proxy pattern.
+
+This is the right long-term answer for high-concurrency workloads — bulk OCR of a
+large corpus, batch classification/extraction, high-QPS chat.
+
+## How it would integrate (mirror the ds4/kt pattern)
+
+- `codai/config.py` — `VllmConfig` dataclass (enabled, install_dir/model_path,
+  host, port, ctx/`max_model_len`, `gpu_memory_utilization`, `tensor_parallel_size`,
+  `max_num_seqs`, dtype/quantization, `served_model_name`/`model_id`, extra_args,
+  extra_env, auto_build) + parent Config field + from_dict/to_dict.
+- `codai/api/vllm_worker.py` — manage
+  `python -m vllm.entrypoints.openai.api_server --host <h> --port <p>
+  --model <path> --served-model-name <id> --max-model-len <ctx>
+  --gpu-memory-utilization <f> [extra]` as a subprocess; health-gate on
+  `/v1/models`; auto-pick a free port. Mirror `codai/api/kt_worker.py`.
+- `codai/backends/vllm.py` — OpenAI HTTP proxy for `/v1/chat/completions`
+  (+ stream), near-copy of `codai/backends/ds4.py` / `codai/backends/ktransformers.py`.
+- Manager wiring — register `"vllm"` in `_ENGINE_BACKENDS`, add
+  `get_active_vllm_config`, a `_vllm_name_claims` that returns **False** (pin/alias
+  only, never auto-claim — same as `kt`, so it never steals models from other
+  engines), a load branch, text-accept, and `/v1/models` surfacing.
+- Front-proxy — `required_capability` params, `registry._DEFAULT_CAPS` (GPU nodes),
+  and thread the config through assignment/app/engine_supervisor.
+- Admin — `routes.py` config get/set, `settings.html` card, `models.html` backend
+  dropdown option `vllm`.
+- Packaging/docs — `auto_build` gated `pip install vllm` (heavy, CUDA-specific);
+  document that vLLM needs a matching CUDA/torch build.
+
+## Notes / caveats
+
+- vLLM is **CUDA-first** (this box's 3090s are fine; AMD/Vulkan is not the target).
+- Vision models: vLLM supports **Qwen2.5-VL** and other VLMs with continuous
+  batching — so a vLLM backend would *also* be the high-throughput path if we ever
+  serve a VLM for image tasks. (For OCR proper we integrate a dedicated OCR engine,
+  not a VLM — see the OCR docs.)
+- Selection stays pin/alias-only to avoid collisions with ds4/colibri/k3/kt, exactly
+  like ktransformers.

packaging/linux/Dockerfile.oci-venv

@@ -58,7 +58,7 @@ RUN set -eux; \
       mkdir -p /opt/coderai/py310; \
       rsync -a /tmp/local-bundle/py310/ /opt/coderai/py310/; \
     fi; \
-    for d in lipsync_venv Wav2Lip SadTalker ds4 colibri; do \
+    for d in lipsync_venv Wav2Lip SadTalker ds4 colibri kimi-k3-in-c; do \
       if [ -d "/tmp/local-bundle/$d" ]; then \
         mkdir -p "/opt/coderai/$d"; \
         rsync -a "/tmp/local-bundle/$d/" "/opt/coderai/$d/"; \
@@ -104,7 +104,8 @@ ENV DEBIAN_FRONTEND=noninteractive \
     CODERAI_SADTALKER_SRC=/opt/coderai/SadTalker \
     CODERAI_SADTALKER_DIR=/cache/lipsync/SadTalker \
     CODERAI_DS4_DIR=/cache/ds4 \
-    CODERAI_COLIBRI_DIR=/cache/colibri
+    CODERAI_COLIBRI_DIR=/cache/colibri \
+    CODERAI_K3_DIR=/cache/kimi-k3-in-c
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
       ca-certificates \

packaging/linux/build_oci_image.sh

@@ -34,6 +34,7 @@ WAV2LIP_DIR="${CODERAI_WAV2LIP_SRC:-$HOME/.coderai/Wav2Lip}"
 SADTALKER_DIR="${CODERAI_SADTALKER_SRC:-$HOME/.coderai/SadTalker}"
 DS4_DIR="${CODERAI_DS4_DIR:-$HOME/.coderai/ds4}"
 COLIBRI_DIR="${CODERAI_COLIBRI_DIR:-$HOME/.coderai/colibri}"
+K3_DIR="${CODERAI_K3_DIR:-$HOME/.coderai/kimi-k3-in-c}"
 # After a successful build, export the image and assemble the final distribution
 # bundle (image tarball + install.sh + coderai-docker runner). Disable with
 # --no-dist (just builds the image).
@@ -87,6 +88,11 @@ Options:
   --prune                 Prune dangling build images after building (default ON).
   -h, --help              Show this help.
 
+Note: this script CAPTURES an existing virtualenv (it does not pip-install). To include
+the dedicated OCR engine stack (PaddleOCR/docTR/YOLO) in the image, build that venv with
+the OCR deps first:  ./build.sh nvidia --ocr   (installs requirements-ocr.txt). The light
+PDF dep (pypdfium2) is in the base requirements and is always included.
+
 Examples:
   ./build-oci.sh
   source venv_all/bin/activate && ./build-oci.sh --from-venv coderai:venv
@@ -307,6 +313,9 @@ discover_local_binaries() {
     "/usr/local/bin/ds4-server"
     "${CODERAI_DS4_DIR:-$HOME/.coderai/ds4}/ds4-server"
     "${CODERAI_COLIBRI_DIR:-$HOME/.coderai/colibri}/c/colibri"
+    "${CODERAI_COLIBRI_DIR:-$HOME/.coderai/colibri}/c/deepseek_v4"
+    "${CODERAI_COLIBRI_DIR:-$HOME/.coderai/colibri}/c/kimi_k3"
+    "${CODERAI_K3_DIR:-$HOME/.coderai/kimi-k3-in-c}/bin/k3"
     "/usr/local/bin/rife-ncnn-vulkan"
     "$HOME/.local/bin/rife-ncnn-vulkan"
   )
@@ -401,11 +410,18 @@ prepare_venv_bundle() {
       rsync -a --exclude 'gguf/' --exclude '*.gguf' --exclude '*.gguf.*' "$DS4_DIR/" "$bundle/ds4/"
       echo "Bundled ds4 (binary + scripts, no weights)"
     fi
-    # colibri: repo + built C engine, minus the ~372 GB GLM-5.2 int4 container.
+    # colibri: repo + built C engines (colibri/deepseek_v4/kimi_k3), minus containers.
     if [[ -d "$COLIBRI_DIR" ]]; then
       rsync -a --exclude '*.safetensors' --exclude 'glm52_i4/' --exclude '*.bin' \
         --exclude 'web/node_modules/' "$COLIBRI_DIR/" "$bundle/colibri/"
-      echo "Bundled colibri (repo + engine binary, no model container)"
+      echo "Bundled colibri (repo + engine binaries, no model container)"
+    fi
+    # kimi-k3-in-c: repo + patched bin/k3, minus build artifacts + the ~1.56 TB
+    # checkpoint / packed trunk (those live outside the image).
+    if [[ -d "$K3_DIR" ]]; then
+      rsync -a --exclude '.git/' --exclude 'build/' --exclude '*.safetensors' \
+        --exclude '*.gguf' --exclude '*.bin' "$K3_DIR/" "$bundle/kimi-k3-in-c/"
+      echo "Bundled kimi-k3-in-c (repo + bin/k3, no checkpoint/trunk)"
     fi
   fi
 

packaging/linux/launcher/coderai-entrypoint

@@ -73,19 +73,40 @@ if [ -n "${HOME:-}" ]; then
   fi
 fi
 
-# Seed the ds4 working dir on the cache volume from the bundled binary + scripts
-# (DeepSeek-V4 weights download here at runtime, so it must be writable/persistent).
-if [ -d /opt/coderai/ds4 ] && [ ! -e "$CODERAI_CACHE_DIR/ds4/ds4-server" ]; then
-  mkdir -p "$CODERAI_CACHE_DIR/ds4"
-  cp -an /opt/coderai/ds4/. "$CODERAI_CACHE_DIR/ds4/" 2>/dev/null || true
+# Seed — or UPGRADE — the ds4 working dir on the cache volume from the bundled binary
+# + scripts (DeepSeek-V4 weights download here at runtime, so it must be writable/
+# persistent). Refresh when the bundled binary differs from the seeded one so an image
+# upgrade actually reaches an existing /cache volume; cp (no -n) clobbers the tree, but
+# the bundle carries NO weights and cp never deletes, so downloaded GGUFs are preserved.
+if [ -d /opt/coderai/ds4 ]; then
+  if [ ! -e "$CODERAI_CACHE_DIR/ds4/ds4-server" ] || \
+     ! cmp -s /opt/coderai/ds4/ds4-server "$CODERAI_CACHE_DIR/ds4/ds4-server"; then
+    mkdir -p "$CODERAI_CACHE_DIR/ds4"
+    cp -a /opt/coderai/ds4/. "$CODERAI_CACHE_DIR/ds4/" 2>/dev/null || true
+  fi
 fi
 
-# Seed the colibri working dir on the cache volume from the bundled repo + engine
-# binary (the GLM-5.2 int4 container lives outside the image; point colibri.model_path
-# at it). Persistent so a runtime rebuild survives restarts.
-if [ -d /opt/coderai/colibri ] && [ ! -e "$CODERAI_CACHE_DIR/colibri/c/colibri" ]; then
-  mkdir -p "$CODERAI_CACHE_DIR/colibri"
-  cp -an /opt/coderai/colibri/. "$CODERAI_CACHE_DIR/colibri/" 2>/dev/null || true
+# Seed — or UPGRADE — the colibri working dir likewise (the GLM-5.2 / DeepSeek-V4 /
+# Kimi-K3 container lives outside the image; point the model_path at it). Refresh on
+# upgrade so a new engine binary reaches the volume; runtime rebuilds still survive
+# restarts because the bundled binary matches after an upgrade.
+if [ -d /opt/coderai/colibri ]; then
+  if [ ! -e "$CODERAI_CACHE_DIR/colibri/c/colibri" ] || \
+     ! cmp -s /opt/coderai/colibri/c/colibri "$CODERAI_CACHE_DIR/colibri/c/colibri"; then
+    mkdir -p "$CODERAI_CACHE_DIR/colibri"
+    cp -a /opt/coderai/colibri/. "$CODERAI_CACHE_DIR/colibri/" 2>/dev/null || true
+  fi
+fi
+
+# Seed — or UPGRADE — the kimi-k3-in-c working dir likewise (the ~1.56 TB Kimi-K3
+# checkpoint + packed trunk live outside the image; point k3.model_path/trunk_dir at
+# them). Refresh on upgrade so a new patched bin/k3 reaches the volume.
+if [ -d /opt/coderai/kimi-k3-in-c ]; then
+  if [ ! -e "$CODERAI_CACHE_DIR/kimi-k3-in-c/bin/k3" ] || \
+     ! cmp -s /opt/coderai/kimi-k3-in-c/bin/k3 "$CODERAI_CACHE_DIR/kimi-k3-in-c/bin/k3"; then
+    mkdir -p "$CODERAI_CACHE_DIR/kimi-k3-in-c"
+    cp -a /opt/coderai/kimi-k3-in-c/. "$CODERAI_CACHE_DIR/kimi-k3-in-c/" 2>/dev/null || true
+  fi
 fi
 
 # If invoked with arguments, run them directly (debugging / one-off commands)

packaging/linux/smoke_test_services.sh

@@ -61,6 +61,16 @@ if "${DK[@]}" exec "$NAME" sh -lc "test -x /cache/ds4/ds4-server"; then ok "/cac
 echo "== colibri seeded on the cache volume =="
 if "${DK[@]}" exec "$NAME" sh -lc "test -x /cache/colibri/c/colibri"; then ok "/cache/colibri/c/colibri"; else bad "/cache/colibri/c/colibri" "missing (optional — bundle with build.sh --colibri)"; fi
 
+echo "== kimi-k3-in-c seeded on the cache volume =="
+if "${DK[@]}" exec "$NAME" sh -lc "test -x /cache/kimi-k3-in-c/bin/k3"; then ok "/cache/kimi-k3-in-c/bin/k3"; else bad "/cache/kimi-k3-in-c/bin/k3" "missing (optional — bundle with build.sh --k3)"; fi
+
+echo "== OCR subsystem present (core imports; engine deps optional) =="
+if "${DK[@]}" exec "$NAME" sh -lc "cd /opt/coderai/app && python -c 'import codai.ocr; from codai.ocr.schemas import schema_store; assert any(s[\"name\"]==\"italian_sentenza\" for s in schema_store.list())'"; then
+  ok "codai.ocr imports + built-in schemas present"
+else
+  bad "codai.ocr" "import or built-in schema check failed"
+fi
+
 echo "== shared lip-sync venv (py3.10 + torch) =="
 if "${DK[@]}" exec "$NAME" /opt/coderai/lipsync_venv/bin/python -c "import torch,sys; print(sys.version.split()[0], torch.__version__)" >/dev/null 2>&1; then
   ok "lipsync venv imports torch"

packaging/patch-colibri.py

@@ -35,18 +35,25 @@ if "g_paused" not in src:
 
 # 2) handle PAUSE / RESUME command lines in mux_submit (line is already NUL-terminated,
 #    trailing newline stripped) — right after the CANCEL handler's NOT_FOUND return.
+#    v1.5.0 added a STOP handler whose NOT_FOUND block is byte-identical to CANCEL's, so
+#    the bare NOT_FOUND+`}` anchor is no longer unique; pin it to the CANCEL→submit
+#    boundary by including the following `ColiSubmit sub;` line (only present after the
+#    CANCEL handler), which also keeps the insertion in the intended place.
 _anchor = ('        printf("ERROR %llu NOT_FOUND\\n",id); fflush(stdout); free(line); return 0;\n'
-           '    }\n')
+           '    }\n'
+           '    ColiSubmit sub; int valid=coli_submit_parse(line,&sub);\n')
 if 'strcmp(line,"PAUSE")' not in src:
     if _anchor not in src:
-        print("[patch-colibri] ERROR: CANCEL/NOT_FOUND anchor not found — upstream changed; "
-              "patch NOT applied", file=sys.stderr)
+        print("[patch-colibri] ERROR: CANCEL/NOT_FOUND→ColiSubmit anchor not found — "
+              "upstream changed; patch NOT applied", file=sys.stderr)
         sys.exit(2)
     src = src.replace(
         _anchor,
-        _anchor +
+        '        printf("ERROR %llu NOT_FOUND\\n",id); fflush(stdout); free(line); return 0;\n'
+        '    }\n'
         '    if(!strcmp(line,"PAUSE")){  g_paused=1; printf("PAUSED\\n");  fflush(stdout); free(line); return 0; }\n'
-        '    if(!strcmp(line,"RESUME")){ g_paused=0; printf("RESUMED\\n"); fflush(stdout); free(line); return 0; }\n',
+        '    if(!strcmp(line,"RESUME")){ g_paused=0; printf("RESUMED\\n"); fflush(stdout); free(line); return 0; }\n'
+        '    ColiSubmit sub; int valid=coli_submit_parse(line,&sub);\n',
         1)
     applied.append("PAUSE/RESUME handler")
 

packaging/patch-k3.py

+#!/usr/bin/env python3
+# CoderAI - kimi-k3-in-c serve-loop patch
+# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net> — GPLv3 (see the main LICENSE).
+"""Idempotently patch FareedKhan-dev/kimi-k3-in-c (``src/cli/k3_run.c``) to add a
+resident serve loop, so coderai can drive it like the other native engines instead of
+paying k3's one-shot cold start (a ~108 GB dense-trunk read) on every request.
+
+Upstream k3 is a batch CLI: it loads the model, runs ONE prompt, and exits. This patch
+adds:
+
+  * ``k3_serve_loop()`` — keeps the model / packed trunk / expert cache / tokenizer
+    resident and answers many prompts over the SAME line-oriented mux stdin/stdout
+    protocol colibri uses, so coderai's existing ``MuxEngine`` client drives it
+    unchanged:
+        engine → server : ``\\x01\\x01READY\\x01\\x01`` + ``STAT …``
+        server → engine : ``SUBMIT <id> <slot> <bytes> <max> <temp> <top_p>\\n<payload>\\n``
+        engine → server : ``DATA <id> <n>\\n<n bytes>\\n`` … ``DONE <id> STAT …``
+    The payload is a fully rendered prompt (the coderai backend owns the Kimi-K3 chat
+    template); the engine tokenizes it with the loaded tokenizer. Greedy, single-slot,
+    incremental decode; KV / recurrent state is reset between turns (stateless HTTP).
+
+  * a ``SERVE=1`` branch in ``main()`` that, after the model/trunk/cache are loaded,
+    loads the tokenizer (``--tok`` dir, else the model dir) and enters the serve loop —
+    reusing all of the one-shot setup rather than duplicating it.
+
+Serve context capacity comes from ``K3_MAXT`` (else ``CTX``, else 4096).
+
+Safe to run repeatedly (each edit is guarded / self-idempotent).
+"""
+import sys
+
+path = sys.argv[1] if len(sys.argv) > 1 else "src/cli/k3_run.c"
+src = open(path, encoding="utf-8", errors="surrogateescape").read()
+orig = src
+applied = []
+
+# --------------------------------------------------------------------------- #
+# 1) the serve loop + a SERVE detector, inserted just before main()
+# --------------------------------------------------------------------------- #
+_SERVE_FN = r'''
+/* ================= coderai serve loop (added by packaging/patch-k3.py) =========
+ * Keep the model, packed trunk, expert cache and tokenizer RESIDENT and answer many
+ * prompts over the colibri-compatible mux stdin/stdout protocol, so coderai's MuxEngine
+ * can drive k3 without paying the ~108 GB cold-start trunk read per request. One prompt
+ * at a time (single KV slot), greedy incremental decode, state reset between turns. */
+static int k3_serve_loop(Weights *w, const K3Cfg *c, K3Cache *cache, Tok *tok, int maxt)
+{
+    const int NL = c->n_layers;
+    const int E  = c->hidden;
+    const int maxb = c->n_layers / c->attn_res_block + 2;
+    const int P  = c->kda_heads * c->kda_head_dim;
+    const size_t kper = (size_t)P * c->kda_head_dim + (size_t)3 * P * (c->conv_k - 1);
+    size_t sc_need = k3_layer_scratch(c, maxt);
+    { size_t ic = k3_mla_scratch_cached(c, maxt, maxt, 1); if (ic > sc_need) sc_need = ic; }
+
+    float *h   = (float *)malloc((size_t)maxt * E * sizeof(float));
+    float *br  = (float *)malloc((size_t)maxt * maxb * E * sizeof(float));
+    float *ks  = (float *)malloc(kper * (size_t)NL * sizeof(float));
+    float *sc  = (float *)malloc(sc_need * sizeof(float));
+    float *lg  = (float *)malloc((size_t)c->vocab * sizeof(float));
+    int   *seq = (int   *)malloc((size_t)maxt * sizeof(int));
+    size_t payload_cap = (size_t)1 << 20;
+    char  *payload = (char *)malloc(payload_cap);
+    if (!h || !br || !ks || !sc || !lg || !seq || !payload) {
+        fprintf(stderr, "k3 serve: OOM allocating serve buffers\n"); return 1; }
+
+    w->mla_slot = (int *)malloc((size_t)NL * sizeof(int));
+    if (!w->mla_slot) { fprintf(stderr, "k3 serve: OOM mla_slot\n"); return 1; }
+    w->n_mla = 0;
+    for (int L = 0; L < NL; L++) w->mla_slot[L] = k3_is_mla(c, L) ? w->n_mla++ : -1;
+    w->kv_cap = maxt;
+    const size_t kvper = (size_t)maxt * c->n_heads * (c->qk_nope + c->v_head);
+    const size_t rpper = (size_t)maxt * c->qk_rope;
+    w->kvc   = (float *)calloc(kvper * (size_t)w->n_mla, sizeof(float));
+    w->ropec = (float *)calloc(rpper * (size_t)w->n_mla, sizeof(float));
+    if (!w->kvc || !w->ropec) { fprintf(stderr, "k3 serve: OOM KV cache\n"); return 1; }
+
+    const int eos = tok_id_of(tok, "<|end_of_msg|>");
+
+    setvbuf(stdin,  NULL, _IONBF, 0);
+    setvbuf(stdout, NULL, _IONBF, 0);
+    fputs("\x01\x01READY\x01\x01\n", stdout);
+    printf("STAT 0 0.0 0.0 %.2f 0 0\n", peak_rss_bytes() / 1e9);
+    fflush(stdout);
+
+    char line[512], dbuf[1024];
+    while (fgets(line, sizeof line, stdin)) {
+        char cmd[16] = {0}, id[64] = {0};
+        int slot = 0, plen = 0, maxtok = 0; float temp = 0.f, top_p = 1.f;
+        int nf = sscanf(line, "%15s %63s %d %d %d %f %f",
+                        cmd, id, &slot, &plen, &maxtok, &temp, &top_p);
+        (void)slot; (void)temp; (void)top_p;
+        if (nf < 2) continue;
+        if (strcmp(cmd, "SUBMIT")) continue;   /* CANCEL/STOP between turns: nothing to do */
+        if (nf < 5 || plen < 0) { printf("ERROR %s BAD_FRAME\n", id); fflush(stdout); continue; }
+        if ((size_t)plen + 1 > payload_cap) {
+            char *n2 = (char *)realloc(payload, (size_t)plen + 1);
+            if (!n2) { printf("ERROR %s OOM\n", id); fflush(stdout); return 1; }
+            payload = n2; payload_cap = (size_t)plen + 1;
+        }
+        if (plen > 0 && fread(payload, 1, (size_t)plen, stdin) != (size_t)plen) break;
+        payload[plen] = 0;
+        (void)fgetc(stdin);   /* trailing newline after the payload */
+
+        /* fresh conversation each turn (stateless HTTP): drop carried recurrent/KV */
+        memset(ks, 0, kper * (size_t)NL * sizeof(float));
+        if (w->n_mla) {
+            memset(w->kvc,   0, kvper * (size_t)w->n_mla * sizeof(float));
+            memset(w->ropec, 0, rpper * (size_t)w->n_mla * sizeof(float));
+        }
+        w->cached = 0;
+
+        int np = tok_encode(tok, payload, plen, seq, maxt - 1);
+        if (np <= 0) { printf("ERROR %s EMPTY_PROMPT\n", id); fflush(stdout); continue; }
+        int cap_tok = maxtok > 0 ? maxtok : 8;
+        if (np + cap_tok > maxt) cap_tok = maxt - np;
+        if (cap_tok < 0) cap_tok = 0;
+
+        const double t0 = now_s();
+        int emitted = 0, limited = 1;
+        if (forward(w, c, cache, seq, np, lg, sc, h, br, ks, NULL) != 0) {
+            printf("DONE %s STAT 0 0.0 0.0 %.2f %d 0\n", id, peak_rss_bytes() / 1e9, np);
+            fflush(stdout); continue;
+        }
+        w->cached = np;
+        int t = argmax_(lg, c->vocab);
+        int T = np;
+        for (int g = 0; g < cap_tok; g++) {
+            if (t == eos) { limited = 0; break; }
+            int n = tok_decode(tok, &t, 1, dbuf, (int)sizeof dbuf);
+            if (n > 0) {
+                printf("DATA %s %d\n", id, n);
+                fwrite(dbuf, 1, (size_t)n, stdout);
+                fputc('\n', stdout);
+                fflush(stdout);
+            }
+            emitted++;
+            if (T >= maxt) break;
+            seq[T++] = t;
+            if (forward(w, c, cache, seq + w->cached, 1, lg, sc, h, br, ks, NULL) != 0) break;
+            w->cached += 1;
+            t = argmax_(lg, c->vocab);
+        }
+        double dt = now_s() - t0;
+        double tps = (emitted > 0 && dt > 0.0) ? emitted / dt : 0.0;
+        printf("DONE %s STAT %d %.3f 0.0 %.2f %d %d\n",
+               id, emitted, tps, peak_rss_bytes() / 1e9, np, limited);
+        fflush(stdout);
+    }
+
+    free(h); free(br); free(ks); free(sc); free(lg); free(seq); free(payload);
+    return 0;
+}
+
+'''
+
+_MAIN_ANCHOR = "int main(int argc, char **argv)\n{"
+if "k3_serve_loop" not in src:
+    if _MAIN_ANCHOR not in src:
+        print("[patch-k3] ERROR: main() anchor not found — upstream changed; patch NOT applied",
+              file=sys.stderr)
+        sys.exit(2)
+    src = src.replace(_MAIN_ANCHOR, _SERVE_FN + _MAIN_ANCHOR, 1)
+    applied.append("k3_serve_loop()")
+
+# --------------------------------------------------------------------------- #
+# 2) SERVE detector near the top of main()
+# --------------------------------------------------------------------------- #
+_ARGC_ANCHOR = "    if (argc < 2) { usage(stderr); return 2; }\n"
+if "int serving =" not in src:
+    if _ARGC_ANCHOR not in src:
+        print("[patch-k3] ERROR: argc-check anchor not found — upstream changed; patch NOT applied",
+              file=sys.stderr)
+        sys.exit(2)
+    src = src.replace(
+        _ARGC_ANCHOR,
+        _ARGC_ANCHOR +
+        "\n    /* coderai: resident serve mode (SERVE=1) over the mux stdin/stdout "
+        "protocol. */\n"
+        "    int serving = getenv(\"SERVE\") && getenv(\"SERVE\")[0] == '1';\n",
+        1)
+    applied.append("SERVE detector")
+
+# --------------------------------------------------------------------------- #
+# 3) let serve mode start without a --ids/--prompt source
+# --------------------------------------------------------------------------- #
+_NSRC_ANCHOR = "        if (nsrc == 0) {"
+if _NSRC_ANCHOR in src:
+    src = src.replace(_NSRC_ANCHOR, "        if (nsrc == 0 && !serving) {", 1)
+    applied.append("skip prompt-required when serving")
+
+# --------------------------------------------------------------------------- #
+# 4) enter the serve loop after the expert cache is initialised
+# --------------------------------------------------------------------------- #
+_CACHE_ANCHOR = ("    K3Cache cache;\n"
+                 "    if (k3_cache_init(&cache, &st, &c, (int64_t)(cache_gb * 1e9)) != 0) return 1;\n")
+if "return k3_serve_loop(" not in src:
+    if _CACHE_ANCHOR not in src:
+        print("[patch-k3] ERROR: k3_cache_init anchor not found — upstream changed; "
+              "patch NOT applied", file=sys.stderr)
+        sys.exit(2)
+    src = src.replace(
+        _CACHE_ANCHOR,
+        _CACHE_ANCHOR +
+        "\n    if (serving) {\n"
+        "        const char *k3_td = tok_dir ? tok_dir : dir;\n"
+        "        if (!have_tok) {\n"
+        "            /* Refuse cleanly if there is no tokenizer to load — k3_tok_load has\n"
+        "             * no error return and would otherwise crash on a bad/empty dir. */\n"
+        "            char k3_tp[4096]; int k3_tok_ok = 0;\n"
+        "            const char *k3_tc[] = {\"tokenizer.json\", \"tiktoken.model\", \"tokenizer.model\"};\n"
+        "            for (int k3i = 0; k3i < 3; k3i++) {\n"
+        "                snprintf(k3_tp, sizeof k3_tp, \"%s/%s\", k3_td, k3_tc[k3i]);\n"
+        "                FILE *k3_tf = fopen(k3_tp, \"rb\");\n"
+        "                if (k3_tf) { fclose(k3_tf); k3_tok_ok = 1; break; }\n"
+        "            }\n"
+        "            if (!k3_tok_ok) {\n"
+        "                fprintf(stderr, \"k3 serve: no tokenizer under '%s' (need \"\n"
+        "                        \"tokenizer.json / tiktoken.model); set --tok\\n\", k3_td);\n"
+        "                return 2;\n"
+        "            }\n"
+        "            k3_tok_load(&tok, k3_td); have_tok = 1;\n"
+        "        }\n"
+        "        int serve_maxt = getenv(\"K3_MAXT\") ? atoi(getenv(\"K3_MAXT\"))\n"
+        "                        : getenv(\"CTX\") ? atoi(getenv(\"CTX\")) : 4096;\n"
+        "        if (serve_maxt < 32) serve_maxt = 4096;\n"
+        "        return k3_serve_loop(&w, &c, &cache, &tok, serve_maxt);\n"
+        "    }\n",
+        1)
+    applied.append("serve branch in main()")
+
+if src != orig:
+    open(path, "w", encoding="utf-8", errors="surrogateescape").write(src)
+    print("[patch-k3] applied to %s: %s" % (path, ", ".join(applied)))
+else:
+    print("[patch-k3] already patched (no change): %s" % path)

requirements-nvidia.txt

@@ -26,6 +26,7 @@ openai-whisper>=20231117       # Whisper for subtitle generation
 # Image/video/audio utilities
 Pillow>=10.0.0
 numpy>=1.24.0
+pypdfium2>=4.20.0             # PDF rasterisation for the OCR subsystem (/v1/ocr); self-contained, no system poppler
 imageio[ffmpeg]>=2.33.0       # frame I/O + ffmpeg bridge for video generation
 scipy>=1.11.0
 sentence-transformers>=2.7.0  # /v1/embeddings

requirements-ocr-paddle.txt

+# ---------------------------------------------------------------------------
+# CoderAI — PaddleOCR ISOLATED venv.
+#
+# PaddleOCR pulls opencv-contrib-python (clashes with the main venv's opencv-python/cv2),
+# and paddlepaddle-gpu wheels bundle their OWN CUDA runtime — so it lives in its own venv +
+# subprocess worker. The bundled CUDA means a cu12x wheel runs on GPU even when the HOST is
+# on newer CUDA (e.g. CUDA 13), as long as the NVIDIA driver is recent.
+#
+#   python3 -m venv ~/.coderai/paddle_venv
+#   ~/.coderai/paddle_venv/bin/pip install -r requirements-ocr-paddle.txt
+#
+# GPU wheels come from Paddle's own index (below). Pick the cuXY matching a wheel that
+# exists for your Python — adjust the index URL / version if needed. For CPU-only, drop the
+# --extra-index-url line and use `paddlepaddle` instead of `paddlepaddle-gpu`.
+# ---------------------------------------------------------------------------
+--extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu126/
+paddlepaddle-gpu>=3.0.0
+paddleocr>=2.7.0
+pypdfium2>=4.20.0

requirements-ocr.txt

+# ---------------------------------------------------------------------------
+# CoderAI — OCR subsystem (dedicated OCR engines, NOT a vision LLM).
+#
+# These are HEAVY / environment-specific dependencies for the /v1/ocr backends.
+# They are kept OUT of the base requirements so the default install and OCI image
+# stay lean. Install with:
+#
+#     pip install -r requirements-ocr.txt
+#
+# or, via the build script (opt-in):
+#
+#     ./build.sh nvidia --ocr
+#
+# The subsystem degrades gracefully: an engine whose dependency is missing returns
+# HTTP 503 from /v1/ocr instead of crashing the server.
+# ---------------------------------------------------------------------------
+
+# --- PDF rasterisation (also pinned in the base requirements; harmless to repeat) ---
+pypdfium2>=4.20.0
+# Optional fallback rasteriser (needs system 'poppler-utils'); pypdfium2 is preferred.
+# pdf2image>=1.17.0
+
+# --- PaddleOCR (optional): Apache-2.0, layout + tables via PP-Structure ---
+# NOTE: paddlepaddle-gpu is CUDA-version-specific and has NO wheel for very new CUDA
+#   (e.g. CUDA 13) — pick the wheel matching your CUDA from
+#   https://www.paddlepaddle.org.cn/en/install/quick , or use CPU 'paddlepaddle'.
+#   Also pulls opencv-contrib-python, which can clash with opencv-python (cv2) — install
+#   deliberately. On CUDA-13 boxes prefer docTR (below), which uses the existing torch.
+# paddlepaddle-gpu>=2.6.0    # uncomment + pin to YOUR CUDA
+# paddleocr>=2.7.0
+
+# --- docTR (Mindee): Apache-2.0, pure-PyTorch (uses the torch already installed) ---
+# The 'torch' extra was dropped in docTR 1.x — plain python-doctr uses the resident torch.
+python-doctr>=0.8.0
+
+# --- Surya: BEST layout/reading-order. GPL (compatible with coderai's GPLv3). ---
+# NOT installed in the main venv — Surya caps pillow<11, which conflicts with the rest of
+# coderai (pillow>=12). It runs in an ISOLATED venv + subprocess worker (same pattern as
+# DINOv2-SALAD); see requirements-surya.txt and the surya worker. Enabled at runtime only
+# when ocr.surya_accept_license = true.
+
+# --- stamp / signature detection (O3): small YOLO detector ---
+ultralytics>=8.1.0
+
+# --- structured extraction (O4): optional JSON Schema validation of model output ---
+jsonschema>=4.0.0

requirements-surya.txt

+# ---------------------------------------------------------------------------
+# CoderAI — Surya OCR ISOLATED venv.
+#
+# Surya caps pillow<11, which conflicts with the main coderai venv (pillow>=12), so it
+# CANNOT be installed alongside the rest of the stack. It runs in its own virtualenv +
+# subprocess worker (same isolation pattern as the DINOv2-SALAD embedder). Create the
+# venv separately, e.g.:
+#
+#     python3 -m venv ~/.coderai/surya_venv
+#     ~/.coderai/surya_venv/bin/pip install -r requirements-surya.txt
+#
+# The coderai Surya backend shells out to this venv's python. GPL license (compatible
+# with coderai's GPLv3); some releases add a revenue-capped commercial clause — check the
+# version you install for commercial use.
+# ---------------------------------------------------------------------------
+surya-ocr>=0.6.0
+pypdfium2>=4.20.0

requirements-vulkan.txt

@@ -32,3 +32,6 @@ voicefixer>=0.1.3
 # Face swap (insightface INSwapper — downloads inswapper_128.onnx on first use)
 insightface>=0.7.3
 onnxruntime>=1.20.0            # CPU ONNX runtime (use onnxruntime-gpu for GPU acceleration)
+
+# PDF rasterisation for the OCR subsystem (/v1/ocr); self-contained, no system poppler
+pypdfium2>=4.20.0

requirements.txt

@@ -60,6 +60,7 @@ litellm>=1.40.0
 # Image/video processing utilities
 Pillow>=10.0.0
 numpy>=1.24.0
+pypdfium2>=4.20.0             # PDF rasterisation for the OCR subsystem (/v1/ocr); self-contained, no system poppler
 imageio[ffmpeg]>=2.33.0       # frame I/O + ffmpeg bridge for video generation
 scipy>=1.11.0                  # audio/signal processing (wav export in audio_gen)