• Stefy Lanza (nextime / spora )'s avatar
    feat: per-model auto-compact of the conversation context (off by default) · a019905f
    Stefy Lanza (nextime / spora ) authored
    When enabled for a model, if the prompt would exceed auto_compact_pct% of the
    model's context window, the conversation is shrunk to ~65% before generation
    instead of erroring on overflow. Per-model config (auto_compact / auto_compact_pct
    / auto_compact_strategy) with three strategies:
      - drop_oldest    : keep system messages + the most recent turns that fit.
      - keep_head_tail : also keep the first user turn as an anchor + a count note.
      - summarize      : replace the dropped middle with a best-effort LLM summary
                         (generated by the loaded model; falls back to a count note).
    
    Token size is a cheap chars/4 estimate; membership uses object identity so
    value-equal turns don't collide. Wired into the chat path (codai/api/text.py),
    the model-configure whitelist, and the model config modal UI.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    a019905f
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...