Skip to content
  • Stefy Lanza (nextime / spora )'s avatar
    b86e7e94
    a think block closed before the first token means the model does not think · b86e7e94
    Stefy Lanza (nextime / spora ) authored
    
    
    datalab-to/chandra-ocr-2 is an OCR fine-tune of a Qwen3.5 base, and its chat
    template still mentions <think>. thinks() therefore said yes, the pod launched
    with --reasoning-parser qwen3, and that parser -- which assumes the stream
    starts INSIDE the block and ends at </think> -- filed every page of
    transcription as reasoning_content and returned content empty.
    
    The pod answered 200 OK twenty-eight times and read nothing: no error anywhere,
    a healthy pod, and $3.34 for six pages that came back blank. The model config
    had not changed; the :latest pod image had moved to one carrying the reasoning
    auto-detection, and moving the tag is the deploy.
    
    Chandra's generation prompt emits '<think>\n\n</think>\n\n' unconditionally --
    no enable_thinking switch anywhere in the template -- so the block is shut
    before the model emits a token and its output contains no closing tag at all.
    That is the opposite of the leaking form this detector exists for, which
    pre-fills an OPEN tag and emits a bare </think>.
    
    enable_thinking is what separates them, and it is checked first: a template
    carrying that switch pre-closes the block only in its thinking-disabled branch
    (Qwen3.5-9B does exactly this), so the model can still think when asked.
    Reading the pre-closed pair without checking for the switch would silence every
    real thinking model, which is far worse than the bug being fixed.
    
    Scoped to the generation prompt, because these templates reuse the same tag
    pair to replay a past assistant turn that had reasoning of its own, and that
    says nothing about what the model is about to emit.
    
    Measured against the four real templates: chandra False, Qwen3.5-9B True,
    surya-ocr-2 and olmOCR False (no tags at all, unchanged).
    
    Co-Authored-By: default avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    b86e7e94
    a think block closed before the first token means the model does not think
    Stefy Lanza (nextime / spora ) authored
    
    
    datalab-to/chandra-ocr-2 is an OCR fine-tune of a Qwen3.5 base, and its chat
    template still mentions <think>. thinks() therefore said yes, the pod launched
    with --reasoning-parser qwen3, and that parser -- which assumes the stream
    starts INSIDE the block and ends at </think> -- filed every page of
    transcription as reasoning_content and returned content empty.
    
    The pod answered 200 OK twenty-eight times and read nothing: no error anywhere,
    a healthy pod, and $3.34 for six pages that came back blank. The model config
    had not changed; the :latest pod image had moved to one carrying the reasoning
    auto-detection, and moving the tag is the deploy.
    
    Chandra's generation prompt emits '<think>\n\n</think>\n\n' unconditionally --
    no enable_thinking switch anywhere in the template -- so the block is shut
    before the model emits a token and its output contains no closing tag at all.
    That is the opposite of the leaking form this detector exists for, which
    pre-fills an OPEN tag and emits a bare </think>.
    
    enable_thinking is what separates them, and it is checked first: a template
    carrying that switch pre-closes the block only in its thinking-disabled branch
    (Qwen3.5-9B does exactly this), so the model can still think when asked.
    Reading the pre-closed pair without checking for the switch would silence every
    real thinking model, which is far worse than the bug being fixed.
    
    Scoped to the generation prompt, because these templates reuse the same tag
    pair to replay a past assistant turn that had reasoning of its own, and that
    says nothing about what the model is about to emit.
    
    Measured against the four real templates: chandra False, Qwen3.5-9B True,
    surya-ocr-2 and olmOCR False (no tags at all, unchanged).
    
    Co-Authored-By: default avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
Loading