• Stefy Lanza (nextime / spora )'s avatar
    load an embedder the way its entry says, and refuse an impossible request before the load · 2833d7f7
    Stefy Lanza (nextime / spora ) authored
    Three faults, two of them reported from a production render on 2026-10-29.
    
    **A local GGUF embedder was fetched from the Hub.**
    `gme-Qwen2-VL-2B-Instruct-Q4_K_M` and `dinov2-large-Q8_0` answered
    
        500 Failed to load embedding model: 404 Client Error …
        huggingface.co/sentence-transformers/<id>/resolve/main/
    
    for files sitting in /AI/guffcache. The loader dispatches on the NAME — `.gguf`
    goes to llama.cpp, or for a ViT to the dinov2-embed server, and anything else
    to sentence-transformers, which prepends `sentence-transformers/` to a name
    with no slash and asks the Hub. These entries are registered under three ids
    (path, basename, basename without extension) and the request used the last.
    The config lookup missed for the same reason, which is worse than it looks: it
    carries `mmproj`, so the vision GGUF would have lost its image tower even with
    the dispatch right. Both now resolve through the entry.
    
    **An impossible request paid for a model load before being refused.** Four
    texts and four images to a text-only 8 GB embedder: forty seconds of loading,
    then the 400 it could have had immediately — and a caller cannot tell that
    from a hang. `_may_embed_images` answers from the config and the HF config
    before anything loads, and only when something positively says "no image
    tower"; what it cannot determine still goes to the load, where the exact
    check on the loaded object decides as before.
    
    **XTTS installed no torch.** `pip install coqui-tts soundfile requests`
    brought coqui-tts 0.28.0 and its ninety dependencies and no torch at all, so
    the install reported success and `import TTS` failed. It has a pinned
    requirements-xtts.txt now, with the same torch/torchaudio pair as parler.
    
    Both TTS engines also stopped hiding the reason: venv_env.import_ok and
    venv_env.gpu_conv_ok carry the real message out, /v1/audio/speech/capabilities
    serves it as env_detail, and the XTTS 501 no longer claims no venv was found
    over a venv that is there and populated with the wrong stack.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    2833d7f7
README.md 60.9 KB