• Stefy Lanza (nextime / spora )'s avatar
    text: stop runaway tool-call loops + honor client repetition penalties · a535c27f
    Stefy Lanza (nextime / spora ) authored
    Some quantized fine-tunes (seen with an "Aggressive" Qwen3.6-35B Q4_K_M) collapse
    into a runaway repetition loop — emitting a malformed parallel tool-call flood of
    1700+ tokens that never terminates — when top_p=1.0 and no repetition penalty are
    in effect (exactly the conditions Qwen's own docs warn cause endless repetitions).
    
    Two fixes:
    
    1. Anti-loop generation stop in stream_chat_response: a model-agnostic detector
       normalises away the variable parts of the tail (quoted strings, filesystem
       paths, whitespace) so a loop whose only per-cycle difference is an arg/path
       still reads as periodic, then breaks generation when a short structural unit
       repeats >=5x back-to-back. Tuned to not trip on prose, repetitive code, or a
       legit handful of distinct tool calls.
    
    2. Honor client-supplied repetition controls. The chat paths previously forwarded
       only temperature/top_p, silently dropping repeat/presence/frequency penalty —
       so a caller (e.g. Kilo) setting them per-model had no effect. Plumb them through
       generate_chat_stream / generate_chat to both backends (cuda already accepts
       them; vulkan now does too) with graceful signature fallbacks. Defaults are
       no-ops, so unset clients are unaffected.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    a535c27f
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...