manager: alias/bare name must reuse the loaded instance (was loading a 2nd despite max 1)
request_model's "already loaded?" fuzzy match compared basenames literally, so a
bare request "gemma-…-Q4_0" missed the loaded "/AI/…/gemma-…-Q4_0.gguf" (the .gguf
suffix differed) and a SECOND instance was loaded even with max_model_instances=1.
A "lisa" alias resolved to the full path and matched, but the bare name didn't —
so two requests for the same model (one via alias, one via name) ran as two
instances instead of queueing on one. Normalize the .gguf extension (and compare
short basenames) when matching, so every name form of one model maps to the same
loaded instance and the second request queues.
Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com>
Showing
Please
register
or
sign in
to comment