• Stefy Lanza (nextime / spora )'s avatar
    choose the tool-call style from the model, not from its name (v0.4.63) · 20112637
    Stefy Lanza (nextime / spora ) authored
    The admin model dialog has had a "Chat template parser" dropdown all along
    (auto/llama/mistral/chatml/phi/gemma/qwen). It is saved on the model entry, and
    seed_model_env carries `parser` in its keep-list so it travels to rented pods.
    NOTHING EVER READ IT: choosing a parser for a model changed nothing anywhere,
    and selection stayed a substring match on the name -- with a second, separate
    guess in the browser (_autoDetectParser).
    
    A tool call is text in the format the model was trained on, and the chat
    template is where that format is written down, so the style belongs to the
    model. It now resolves, strongest first:
    
      1. the model's own `parser` setting -- that dropdown now does something;
      2. the global default, read the way resolve_gemma_tool_mode reads its own;
      3. the model's chat template;
      4. the name.
    
    The template reading moves out of vllm_worker into codai/models/chat_style.py,
    with ONE style table mapping each style to both consumers -- vLLM's
    --tool-call-parser and coderai's own parser class -- so the two choices cannot
    drift apart. Qwen XML, Hermes JSON, GLM <arg_key>, Mistral [TOOL_CALLS], Llama
    <|python_tag|> and DeepSeek's call tokens. Cached per model path: a model's
    template cannot change under us, and this is consulted per request on some
    paths.
    
    The template does NOT override a name that matched a family. A per-family
    parser carries more than the tool syntax -- gemma's token repair, Qwen's
    repetition guard -- so one template marker should not displace a family that
    matched. It decides where the name places the model in no family, which fell
    to the catch-all before: _family_name only resolves an alias for an ADAPTER,
    so a plain alias like `lisa` named no family and got the generic parser even
    though its entry says exactly where the weights are.
    
    A value the dispatcher does not know falls back to detection rather than
    silently disabling tool calls, and an unreadable template changes nothing --
    not knowing is survivable, inventing a parser is not.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    20112637
test_vllm_qwen_tool_parser_choice.py 4.89 KB