-
Stefy Lanza (nextime / spora ) authored
Switching a model's `runpod.engine` from `coderai-vllm` to `coderai` -- the correct setting for a capability pod -- silently removed its capability. `_CODERAI_SERVER_ENGINES["coderai-vllm"]` had been supplying the classification; plain `coderai` supplies none, and `model_capability()` returned "". Both halves of the failure follow from that: * the front could not classify the request, so the model was loaded LOCALLY on a GPU-less orchestrator; * default_capability_image("") is "", so a blank image field raises "set `image` on the runpod block" at provisioning time. It presented as "tries to load on the wrong machine", which is a long way from "this entry has no capability". The cause: model_capability() receives a BARE DICT, and _model_entry_for() iterates md.values() -- it matches inside `embedding_models` and discards the section key. Nothing had ever told it that an entry in `embedding_models` is an embeddings model. Now it asks: by object identity first (the dict the loader holds), falling back to the id so a copied or rebuilt entry still resolves, and returning False on any failure because a capability GUESS is never worth an exception in a placement path. An explicit `capability` field still wins, which is what that field is for. Only `embedding_models` is inferred, deliberately. The other sections work today, and remapping them all to fix one would change placement for every model kind -- a blast radius this does not need. A test asserts the inference does not reach into text_models, image_models, video_models or tts_models. Found by a user report ("broken -- front loads bge-m3 locally on the GPU-less host; not in /v1/models") after I made exactly that config change. I had already noted this gap when the capability came back empty during the switch, said it was worth raising rather than working around, and then worked around it in the config anyway -- which is how it reached production. 11 tests. Suite at the 25 pre-existing failures. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
e3dc237b