-
Stefy Lanza (nextime / spora ) authored
The admin model dialog has had a "Chat template parser" dropdown all along (auto/llama/mistral/chatml/phi/gemma/qwen). It is saved on the model entry, and seed_model_env carries `parser` in its keep-list so it travels to rented pods. NOTHING EVER READ IT: choosing a parser for a model changed nothing anywhere, and selection stayed a substring match on the name -- with a second, separate guess in the browser (_autoDetectParser). A tool call is text in the format the model was trained on, and the chat template is where that format is written down, so the style belongs to the model. It now resolves, strongest first: 1. the model's own `parser` setting -- that dropdown now does something; 2. the global default, read the way resolve_gemma_tool_mode reads its own; 3. the model's chat template; 4. the name. The template reading moves out of vllm_worker into codai/models/chat_style.py, with ONE style table mapping each style to both consumers -- vLLM's --tool-call-parser and coderai's own parser class -- so the two choices cannot drift apart. Qwen XML, Hermes JSON, GLM <arg_key>, Mistral [TOOL_CALLS], Llama <|python_tag|> and DeepSeek's call tokens. Cached per model path: a model's template cannot change under us, and this is consulted per request on some paths. The template does NOT override a name that matched a family. A per-family parser carries more than the tool syntax -- gemma's token repair, Qwen's repetition guard -- so one template marker should not displace a family that matched. It decides where the name places the model in no family, which fell to the catch-all before: _family_name only resolves an alias for an ADAPTER, so a plain alias like `lisa` named no family and got the generic parser even though its entry says exactly where the weights are. A value the dispatcher does not know falls back to detection rather than silently disabling tool calls, and an unreadable template changes nothing -- not knowing is survivable, inventing a parser is not. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
20112637