• Stefy Lanza (nextime / spora )'s avatar
    perf: cache the cache-scans + gpu-stats so the models page paints instantly · dc3800fe
    Stefy Lanza (nextime / spora ) authored
    The models page was slow because the work itself is slow, not because of routing:
    cached-models walks the HF/GGUF cache doing per-model capability detection, and
    cache-stats walks every file summing sizes — many seconds each on a large cache.
    
    - routes.py: cached-models + cache-stats are now stale-while-revalidate (TTL 25s).
      First call computes; afterwards they return the last result instantly and refresh
      in a background thread. Mutating ops (clear-cache, delete cached model, free-disk)
      force the next refresh. The coderai-system worker pre-warms both scans on startup,
      so even the first page load is fast.
    - gpu_detect.gpu_stats(): short 1.5s TTL cache — the Tasks page polls it ~every
      second and nvidia-smi is slow on a saturated GPU, so collapse repeats.
    
    Verified: _cached returns stale instantly and refreshes in the background
    (2 underlying calls across first+expire); all modules compile.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    dc3800fe
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...