• Stefy Lanza (nextime / spora )'s avatar
    runpod: a seeded pod filed every model under text (v0.3.83) · 02a58b2c
    Stefy Lanza (nextime / spora ) authored
    In models.json a model's kind is the dict KEY it sits under. Only the
    ENTRY travels to a pod, and the pod's seeder reads
    
        section = e.get("model_type") or "text_models"
    
    A models.json entry carries no `model_type` -- so every capability pod
    registered every seeded model as a TEXT model, whatever it was.
    
    Found on a real A40 immediately after the gateway fix finally let a
    request reach a pod. It booted in 303s, served /v1/models, listed
    BAAI/bge-m3, and answered:
    
        Model 'BAAI/bge-m3' is a text model and cannot be used for embedding
        generation. Use the text endpoint instead.
    
    Which is the expensive part: nothing looks broken. The routing is right,
    the image is right, the catalogue is right, and the error blames the
    caller for calling the wrong endpoint.
    
    Both seed builders now carry the section -- `capability_seed_entries`
    already had it and dropped it at `out.append(entry)`, and
    `seed_model_env` gets it from the new `entry_model_type()` (by identity,
    then by id, "" on any failure). Stamped on a COPY, so nothing writes a
    field into models.json that this install never had, and an entry that
    already declares its own `model_type` is not second-guessed.
    
    Text pods were unaffected, which is why this survived: a vLLM pod serves
    text, and OCR engines are picked by name rather than registered at all.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    02a58b2c
__init__.py 4.21 KB