• Stefy Lanza (nextime / spora )'s avatar
    colibri: stop at GLM turn boundary + clean output + resolve model_id alias · 58f945de
    Stefy Lanza (nextime / spora ) authored
    GLM output was garbage: colibri keeps only EOS as a hard stop in serve mode (it filters
    the non-EOS special-token stops for tool-call safety, #401), so the model ran past its
    turn and emitted <|user|>/<think>/</think> and repeated the prompt.
    
    - backend: clean_glm_output cuts the reply at the first turn marker (<|user|>,
      <|observation|>, <|assistant|>, <|system|>, <|endoftext|>) and strips control tokens
      (<think>,</think>,[gMASK],<sop>). Applied to both non-stream and stream paths; the
      stream holds back a short tail so a marker split across chunks never leaks.
    - Early stop: generate paths pass a `cancelled` hook that fires the moment a turn
      marker appears, so colibri CANCELs instead of decoding to max_tokens — a big saving
      at streaming-bound decode speeds. MuxEngine.run now treats our own CANCELLED ack as a
      clean early stop (returns partial stats) instead of raising.
    - alias: _resolve_container now resolves the configured colibri model_id (e.g.
      glm-5.2-colibri) and colibri.model_path to the container, not only the full repo path
      — so `{"model":"glm-5.2-colibri"}` loads instead of "no container resolved".
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
    58f945de
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...