• Stefy Lanza (nextime / spora )'s avatar
    front: route GGUF pinned to torch engine -> gguf sibling; sync cached-models refresh · ec1f7218
    Stefy Lanza (nextime / spora ) authored
    Two fixes after the GGUF process-isolation split:
    
    1. Routing: a model with engine=nvidia in its config (a hard pin) but a GGUF body
       503'd, because the torch nvidia engine dropped the `gguf` capability. pick_engine
       now falls back to the co-located "<name>-gguf" sibling (same card) when the pinned
       engine can't serve the required capability (and symmetrically for a gguf-pinned
       model needing transformers). The pin's intent — that physical card — is honoured.
    
    2. Config-save visibility: the cached-models scan is stale-while-revalidate, so the
       first read after an invalidation returned stale and refreshed in the background —
       a saved config didn't appear until a second refresh. _invalidate_cache_scan now
       sets force_sync so the next read recomputes synchronously (fresh immediately). The
       system worker invalidates on reload-config, so a save shows on the next page load.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    ec1f7218
router.py 7.82 KB