manager: classify gguf by resolving alias to a local .gguf file

load_model() decided gguf-vs-HF purely from the literal string 'gguf' in
the model name. A gguf whose alias carries only the quant suffix (e.g.
'coe-gemma4-coding-hc-14b-a4b-q4_k_m', no literal 'gguf') was mis-routed
to the HF/transformers backend, which then failed with "is not a valid
model identifier" (503). Fall back to _resolve_local_gguf(): if the alias
maps to an actual local .gguf, treat it as gguf and route to llama.cpp.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 2eda7574
...@@ -323,9 +323,18 @@ class ModelManager: ...@@ -323,9 +323,18 @@ class ModelManager:
available = detect_available_backends() available = detect_available_backends()
# Check if model is a GGUF file # Check if model is a GGUF file. The name alone isn't reliable: a gguf's
# alias may carry only the quant suffix (e.g. '…-q4_k_m') with no literal
# 'gguf', which would mis-route it to the HF/transformers path. Fall back
# to resolving the alias to an actual local .gguf file.
is_gguf = model_name.endswith('.gguf') or 'gguf' in model_name.lower() is_gguf = model_name.endswith('.gguf') or 'gguf' in model_name.lower()
if not is_gguf:
try:
if _resolve_local_gguf(model_name):
is_gguf = True
except Exception:
pass
# Determine backend # Determine backend
if backend_type == "auto": if backend_type == "auto":
if available.get('nvidia'): if available.get('nvidia'):
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment