manager: don't evict a model to make room for itself

0.1.95 fixed the size — "sizing 'lisa-32k' by its own config" now fires and the
requirement fell from 52.0 GB to the correct 22.9 GB — but the second request still
evicted:

  Loading '…gemma-4-26B…gguf': need 22.9 GB VRAM, have 4.6 GB free — evicting
  On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB)

Only 4.6 GB was free BECAUSE lisa-32k was resident and holding 22.9 GB of the card.
The reservation asked whether there was room for a model that had already been
loaded, found none, and evicted that very model to make room for it.

It reaches ensure_vram_for at all because residency is checked under one id while
the reservation arrives under another — self.models is keyed by the alias, the
reservation by the weights path. So the guard checks every id the request answers
to (model_key, resolved_name, requested_id) and returns early when any is resident.
Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
parent e6c2bb76
......@@ -4656,6 +4656,15 @@ class MultiModelManager:
room for per-request additions the base estimate can't know about (e.g.
dynamically-applied LoRA adapters).
"""
# Already resident? Then there is nothing to reserve. Without this, a
# request for a model that is ALREADY loaded asks "do I have 22.9 GB free?"
# — and the answer is no precisely BECAUSE that model is occupying the card,
# so it evicts itself to make room for itself. It can look up as not-loaded
# under one of its ids (self.models is keyed by the alias, the reservation
# arrives with the path), so check every id this request answers to.
for _k in (model_key, resolved_name, requested_id):
if _k and _k in self.models:
return
needed_gb = self._get_model_used_vram_gb(model_key, resolved_name, requested_id)
if extra_vram_gb and extra_vram_gb > 0:
needed_gb += extra_vram_gb
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment