-
Stefy Lanza (nextime / spora ) authored
The models page was slow because the work itself is slow, not because of routing: cached-models walks the HF/GGUF cache doing per-model capability detection, and cache-stats walks every file summing sizes — many seconds each on a large cache. - routes.py: cached-models + cache-stats are now stale-while-revalidate (TTL 25s). First call computes; afterwards they return the last result instantly and refresh in a background thread. Mutating ops (clear-cache, delete cached model, free-disk) force the next refresh. The coderai-system worker pre-warms both scans on startup, so even the first page load is fast. - gpu_detect.gpu_stats(): short 1.5s TTL cache — the Tasks page polls it ~every second and nvidia-smi is slow on a saturated GPU, so collapse repeats. Verified: _cached returns stale instantly and refreshes in the background (2 underlying calls across first+expire); all modules compile. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
dc3800fe
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| frontproxy | ||
| models | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| main.py | ||
| platform_paths.py | ||
| system_app.py |