• Stefy Lanza (nextime / spora )'s avatar
    colibri: stop at GLM turn boundary + clean output + resolve model_id alias · 58f945de
    Stefy Lanza (nextime / spora ) authored
    GLM output was garbage: colibri keeps only EOS as a hard stop in serve mode (it filters
    the non-EOS special-token stops for tool-call safety, #401), so the model ran past its
    turn and emitted <|user|>/<think>/</think> and repeated the prompt.
    
    - backend: clean_glm_output cuts the reply at the first turn marker (<|user|>,
      <|observation|>, <|assistant|>, <|system|>, <|endoftext|>) and strips control tokens
      (<think>,</think>,[gMASK],<sop>). Applied to both non-stream and stream paths; the
      stream holds back a short tail so a marker split across chunks never leaks.
    - Early stop: generate paths pass a `cancelled` hook that fires the moment a turn
      marker appears, so colibri CANCELs instead of decoding to max_tokens — a big saving
      at streaming-bound decode speeds. MuxEngine.run now treats our own CANCELLED ack as a
      clean early stop (returns partial stats) instead of raising.
    - alias: _resolve_container now resolves the configured colibri model_id (e.g.
      glm-5.2-colibri) and colibri.model_path to the container, not only the full repo path
      — so `{"model":"glm-5.2-colibri"}` loads instead of "no container resolved".
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
    58f945de
Name
Last commit
Last update
..
__init__.py Loading commit data...
base.py Loading commit data...
colibri.py Loading commit data...
cuda.py Loading commit data...
ds4.py Loading commit data...
vulkan.py Loading commit data...