-
Stefy Lanza (nextime / spora ) authored
gemma-4 GGUF finetunes don't emit a clean EOS after a tool call — they continue and hallucinate a fake tool result (<tool_call|><|tool_response>response:NAME{…} … <turn|>), wasting tokens on a result llama.cpp never runs. Add a per-model stop augmentation (_augment_model_stops): for gemma, inject <|tool_response>, <tool_response|>, <turn|> and the real <end_of_turn> as stop strings (merged with any client-supplied stops). llama.cpp matches and trims stop strings, so the call ends cleanly and the fake-response tail is never generated. Applied to both chat methods and the raw completion paths; non-gemma models are unaffected. Complements 5135f8f8 (parser repair of <|"|> quotes and malformed markers). Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
ddb48b2d