• Stefy Lanza (nextime / spora )'s avatar
    perf: cache the cache-scans + gpu-stats so the models page paints instantly · dc3800fe
    Stefy Lanza (nextime / spora ) authored
    The models page was slow because the work itself is slow, not because of routing:
    cached-models walks the HF/GGUF cache doing per-model capability detection, and
    cache-stats walks every file summing sizes — many seconds each on a large cache.
    
    - routes.py: cached-models + cache-stats are now stale-while-revalidate (TTL 25s).
      First call computes; afterwards they return the last result instantly and refresh
      in a background thread. Mutating ops (clear-cache, delete cached model, free-disk)
      force the next refresh. The coderai-system worker pre-warms both scans on startup,
      so even the first page load is fast.
    - gpu_detect.gpu_stats(): short 1.5s TTL cache — the Tasks page polls it ~every
      second and nvidia-smi is slow on a saturated GPU, so collapse repeats.
    
    Verified: _cached returns stale instantly and refreshes in the background
    (2 underlying calls across first+expire); all modules compile.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    dc3800fe
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...