manager: canonicalize GGUF model_key to the file path (concurrent-race instance dedup)

Harden the max-instances guard against the concurrent case: request_model now
resolves a GGUF text model to its absolute .gguf path (via _resolve_local_gguf)
before building model_key, so every name form — bare id, basename.gguf, full path,
or alias (e.g. "lisa") — collapses to ONE model_key. That gives a single
self.models entry and a single instance pool, so two simultaneous first-requests
under different forms converge on the same pool and the second queues instead of
loading a second instance. Complements the earlier already-loaded fuzzy match
(which only covered the sequential case).
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 83bc76d7
...@@ -3658,6 +3658,21 @@ class MultiModelManager: ...@@ -3658,6 +3658,21 @@ class MultiModelManager:
resolved_name = _m resolved_name = _m
break break
# Step 1d: Canonicalize a GGUF text model to its absolute file path, so EVERY
# name form (bare id, basename.gguf, full path, alias) collapses to ONE
# model_key — and therefore one self.models entry and one instance pool.
# Without this, two CONCURRENT first-requests under different forms (e.g. a
# bare name and an alias) each compute a distinct key and each load an
# instance, exceeding max_model_instances=1. (The earlier fuzzy match only
# catches the case where one is ALREADY loaded.)
if not model_type or model_type == "text":
try:
_canon_path = _resolve_local_gguf(resolved_name)
if _canon_path:
resolved_name = _canon_path
except Exception:
pass
# Step 2: Build the model key (prefixed with type) # Step 2: Build the model key (prefixed with type)
if model_type and model_type != "text": if model_type and model_type != "text":
model_key = f"{model_type}:{resolved_name}" model_key = f"{model_type}:{resolved_name}"
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment