Skip to content
Commit b86e7e94 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

a think block closed before the first token means the model does not think



datalab-to/chandra-ocr-2 is an OCR fine-tune of a Qwen3.5 base, and its chat
template still mentions <think>. thinks() therefore said yes, the pod launched
with --reasoning-parser qwen3, and that parser -- which assumes the stream
starts INSIDE the block and ends at </think> -- filed every page of
transcription as reasoning_content and returned content empty.

The pod answered 200 OK twenty-eight times and read nothing: no error anywhere,
a healthy pod, and $3.34 for six pages that came back blank. The model config
had not changed; the :latest pod image had moved to one carrying the reasoning
auto-detection, and moving the tag is the deploy.

Chandra's generation prompt emits '<think>\n\n</think>\n\n' unconditionally --
no enable_thinking switch anywhere in the template -- so the block is shut
before the model emits a token and its output contains no closing tag at all.
That is the opposite of the leaking form this detector exists for, which
pre-fills an OPEN tag and emits a bare </think>.

enable_thinking is what separates them, and it is checked first: a template
carrying that switch pre-closes the block only in its thinking-disabled branch
(Qwen3.5-9B does exactly this), so the model can still think when asked.
Reading the pre-closed pair without checking for the switch would silence every
real thinking model, which is far worse than the bug being fixed.

Scoped to the generation prompt, because these templates reuse the same tag
pair to replay a past assistant turn that had reasoning of its own, and that
says nothing about what the model is about to emit.

Measured against the four real templates: chandra False, Qwen3.5-9B True,
surya-ocr-2 and olmOCR False (no tags at all, unchanged).

Co-Authored-By: default avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
parent 46415d44
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment