• Stefy Lanza (nextime / spora )'s avatar
    manager: don't evict a model to make room for itself · 2a8bb4c2
    Stefy Lanza (nextime / spora ) authored
    0.1.95 fixed the size — "sizing 'lisa-32k' by its own config" now fires and the
    requirement fell from 52.0 GB to the correct 22.9 GB — but the second request still
    evicted:
    
      Loading '…gemma-4-26B…gguf': need 22.9 GB VRAM, have 4.6 GB free — evicting
      On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB)
    
    Only 4.6 GB was free BECAUSE lisa-32k was resident and holding 22.9 GB of the card.
    The reservation asked whether there was room for a model that had already been
    loaded, found none, and evicted that very model to make room for it.
    
    It reaches ensure_vram_for at all because residency is checked under one id while
    the reservation arrives under another — self.models is keyed by the alias, the
    reservation by the weights path. So the guard checks every id the request answers
    to (model_key, resolved_name, requested_id) and returns early when any is resident.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    2a8bb4c2
manager.py 263 KB