-
Stefy Lanza (nextime / spora ) authored
The mux serve path (SERVE_BATCH=1, run_serve_mux) force-disables the int8 MTP speculative-decode head ("speculation is not ragged-safe across KV slots") and its batched decode loop has no MTP path at all — MTP lives only in run_serve's single-sequence spec_decode. Since coderai drives colibri with kv_slots=1, drive run_serve instead (SERVE_BATCH=0): accepted MTP drafts mean fewer forward passes = fewer expert-streaming rounds, a real decode win on this IO-bound engine. MuxEngine now selects single-client mode when kv_slots==1: launches run_serve, drains its one-shot startup telemetry, and runs turns synchronously over the run_serve byte protocol (\x02PROMPT frame in; raw text + \x01\x01END\x01\x01 + STAT out) with SIGINT-based cancellation. Multi-slot keeps the batched mux path. pause/resume are no-ops in single-client mode (run_serve would misread the frame; thermal falls back to the front's SIGSTOP). Bump 0.1.81. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0196Gm4PNkcuybDj9yfcz3CR
98963164