manager: alias/bare name must reuse the loaded instance (was loading a 2nd despite max 1)

request_model's "already loaded?" fuzzy match compared basenames literally, so a
bare request "gemma-…-Q4_0" missed the loaded "/AI/…/gemma-…-Q4_0.gguf" (the .gguf
suffix differed) and a SECOND instance was loaded even with max_model_instances=1.
A "lisa" alias resolved to the full path and matched, but the bare name didn't —
so two requests for the same model (one via alias, one via name) ran as two
instances instead of queueing on one. Normalize the .gguf extension (and compare
short basenames) when matching, so every name form of one model maps to the same
loaded instance and the second request queues.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 2d8d5a22
...@@ -3667,14 +3667,24 @@ class MultiModelManager: ...@@ -3667,14 +3667,24 @@ class MultiModelManager:
# Step 3: Check if already loaded in self.models # Step 3: Check if already loaded in self.models
existing_model = self.models.get(model_key) existing_model = self.models.get(model_key)
# Fuzzy fallback: if not found by exact key, check for org-prefix # Fuzzy fallback: if not found by exact key, check for org-prefix and
# variations (e.g. "image:Model" vs "image:org/Model"). # .gguf-extension variations, so all name forms of ONE model map to the
# SAME loaded instance (e.g. "image:Model" vs "image:org/Model", and a bare
# "gemma-…-Q4_0" request vs the loaded "/AI/…/gemma-…-Q4_0.gguf"). Without
# the .gguf normalization the bare name missed the loaded model and a SECOND
# instance was loaded despite max_model_instances=1.
if existing_model is None: if existing_model is None:
type_prefix = f"{model_type}:" if model_type and model_type != "text" else "" type_prefix = f"{model_type}:" if model_type and model_type != "text" else ""
short = resolved_name.split("/")[-1]
def _norm_id(s):
s = (s or "").split("/")[-1]
return s[:-5] if s.lower().endswith(".gguf") else s
short = _norm_id(resolved_name)
for k, v in self.models.items(): for k, v in self.models.items():
k_name = k[len(type_prefix):] if type_prefix and k.startswith(type_prefix) else k k_name = k[len(type_prefix):] if type_prefix and k.startswith(type_prefix) else k
if k_name == resolved_name or k_name.split("/")[-1] == short: if (k_name == resolved_name or k_name.split("/")[-1] == resolved_name.split("/")[-1]
or _norm_id(k_name) == short):
model_key = k model_key = k
resolved_name = k_name resolved_name = k_name
existing_model = v existing_model = v
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment