• Stefy Lanza (nextime / spora )'s avatar
    coderai: global host-RAM cap with leak watch + disk-offload eviction · 99f8ba85
    Stefy Lanza (nextime / spora ) authored
    Add a server-wide host-RAM ceiling (OffloadConfig.max_ram_gb) alongside the
    existing VRAM budgeting:
    
    - hf_loading clamps the accelerate CPU-offload budget to the headroom under
      the cap, so overflow spills to the disk offload folder instead of growing RSS.
    - manager: process-tree RSS accounting, true-LRU (active_in_vram property stamps
      _last_used), shared _evict_one, and _evict_models_for_ram; idle models are
      evicted before a new load when RSS nears the cap.
    - ram_monitor.py: background watcher samples RSS, flags a suspected leak when it
      climbs while the scheduler is idle, and runs a mitigation ladder
      (gc -> empty_cache -> malloc_trim -> drop upscaler cache -> evict idle).
    - admin /status returns a ram block; Settings page exposes max RAM + evict/
      leak-watch toggles (applied live); dashboard shows a RAM gauge + leak badge.
    
    Also fold loaded upscalers (_UPSCALER_CACHE) into the dashboard models-loaded
    count so an active upscale no longer reports '0 models loaded'.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    99f8ba85
Name
Last commit
Last update
codai Loading commit data...
docs/superpowers Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...