• Stefy Lanza (nextime / spora )'s avatar
    ds4: on-disk KV-cache age cleanup + HF search on Enter · ce9c2943
    Stefy Lanza (nextime / spora ) authored
    - ds4: configurable janitor that age-prunes the on-disk KV-checkpoint cache
      (--kv-disk-dir, defaulted to <offload>/ds4-kv). New Ds4Config fields
      kv_cache_cleanup_enabled / kv_cache_max_age_hours (7d) /
      kv_cache_cleanup_interval_minutes (6h); new codai/api/ds4_kv_janitor.py
      reuses the tmp_janitor sweep (newest-mtime, so active sessions are spared),
      started from main.py only when ds4 + cleanup are both on. Settings UI +
      get/save wired.
    - ds4: corrected the perf note — i-quants (IQ2/IQ3) and Q2_K load but fail
      ds4's CUDA prefill (gpu layer 0 ffn batch encode failed → empty reply);
      use K-quants Q4_K and up.
    - models: pressing Enter in the HuggingFace search field now runs the search.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    ce9c2943
main.py 62 KB