-
Stefy Lanza (nextime / spora ) authored
From ONE pod, ONE set of weights, behind two aliases: qwen38 -> clean answer diogene-alpha -> "The user is asking to list hearings from 2026-10-21 to..." The entire chain of thought went to the client as the answer. Four breaks: 1. THE GATE ASKED THE NAME. _reasoning_active was a regex (qwen3|qwq| deepseek-r1|...) over the request's model name: `qwen3` matches "qwen38" and not "diogene-alpha", so one alias of one model separated its thought and the other did not. It is read from the template now -- whose generation prompt ends with '<think>\n', so the model starts INSIDE the thought and emits a bare </think>, the form that leaks when nothing looks for it. The regex stays as a last resort for a template that cannot be read. 2. THE BUFFERED FORMATTER TOOK NO `reasoning`. Where the thought WAS separated it was then dropped: no leak, but no thinking either. litellm's Message has no such field, so it is attached after the dump rather than lost -- under both spellings, since Qwen's own template reads `reasoning_content`. 3. THE PROXY HOP CARRIED NEITHER FORM. Buffered, the remote sends reasoning_content beside an already-cleaned content, which the string contract cannot hold: get_last_reasoning(), like get_last_tool_calls(). Streamed, it sends reasoning DELTAS, and dropping them meant a client saw nothing at all until the answer began -- so the <think>/</think> tags go back on as they arrive and the caller's existing reasoning gate splits them live. An all-thinking stream leaves the tag open on purpose: the gate flushes an unclosed thought as reasoning, and there is no content for a close tag to precede. 4. THINKING COULD NOT BE TURNED OFF. enable_thinking defaulted to False, so "unset" and "explicitly off" were the same value, and the field sat in _CHAT_CONSUMED_FIELDS while no backend forwarded it. It defaults to None now: unset sends nothing and the model keeps its own default (a thinking model thinks), an explicit value travels as chat_template_kwargs. That needed the POD backend to gain passthrough at all -- the local vLLM backend has forwarded unknown fields for releases, a pod got none of them, so chat_template_kwargs could never reach a pod-served model. ChatMessage silently dropped `reasoning_content`, so a previous turn's thinking never reached the template that re-renders it inside <think>. Accepted and forwarded now. The chat page hardcoded stream:false, so thinking could never arrive live there, and it read only message.content. It has `stream` and `thinking` toggles (both on), renders the thought in a block that fills AS IT ARRIVES and collapses when the answer starts, and keeps it in the history it replays. Streaming is the request's choice, not forced: cleared, it takes the buffered path exactly as before. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
67aa1f5f