• Stefy Lanza (nextime / spora )'s avatar
    manager: don't evict a model to make room for itself · 2a8bb4c2
    Stefy Lanza (nextime / spora ) authored
    0.1.95 fixed the size — "sizing 'lisa-32k' by its own config" now fires and the
    requirement fell from 52.0 GB to the correct 22.9 GB — but the second request still
    evicted:
    
      Loading '…gemma-4-26B…gguf': need 22.9 GB VRAM, have 4.6 GB free — evicting
      On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB)
    
    Only 4.6 GB was free BECAUSE lisa-32k was resident and holding 22.9 GB of the card.
    The reservation asked whether there was room for a model that had already been
    loaded, found none, and evicted that very model to make room for it.
    
    It reaches ensure_vram_for at all because residency is checked under one id while
    the reservation arrives under another — self.models is keyed by the alias, the
    reservation by the weights path. So the guard checks every id the request answers
    to (model_key, resolved_name, requested_id) and returns early when any is resident.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    2a8bb4c2
Name
Last commit
Last update
..
cache Loading commit data...
__init__.py Loading commit data...
acceleration.py Loading commit data...
capabilities.py Loading commit data...
gpu_lock.py Loading commit data...
gpu_query.py Loading commit data...
grammar.py Loading commit data...
hf_loading.py Loading commit data...
manager.py Loading commit data...
parser.py Loading commit data...
peft_compat.py Loading commit data...
pipeline_cache.py Loading commit data...
quant.py Loading commit data...
ram_monitor.py Loading commit data...
templates.py Loading commit data...
thermal.py Loading commit data...
tmp_janitor.py Loading commit data...
tool_call_grammar.gbnf Loading commit data...
turboquant.py Loading commit data...
utils.py Loading commit data...