• Stefy Lanza (nextime / spora )'s avatar
    feat: per-model auto-compact of the conversation context (off by default) · a019905f
    Stefy Lanza (nextime / spora ) authored
    When enabled for a model, if the prompt would exceed auto_compact_pct% of the
    model's context window, the conversation is shrunk to ~65% before generation
    instead of erroring on overflow. Per-model config (auto_compact / auto_compact_pct
    / auto_compact_strategy) with three strategies:
      - drop_oldest    : keep system messages + the most recent turns that fit.
      - keep_head_tail : also keep the first user turn as an anchor + a count note.
      - summarize      : replace the dropped middle with a best-effort LLM summary
                         (generated by the loaded model; falls back to a count note).
    
    Token size is a cheap chars/4 estimate; membership uses object identity so
    value-equal turns don't collide. Wired into the chat path (codai/api/text.py),
    the model-configure whitelist, and the model config modal UI.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    a019905f
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
characters.py Loading commit data...
custom_pipelines.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
images.py Loading commit data...
log.py Loading commit data...
loras.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
ratelimit.py Loading commit data...
spatial.py Loading commit data...
state.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...