• Stefy Lanza (nextime / spora )'s avatar
    embeddings: multimodal (text+image) support + correct VRAM tracking/eviction · dd848607
    Stefy Lanza (nextime / spora ) authored
    Wire up real image embeddings and make embedding models first-class in the
    VRAM lifecycle.
    
    - api/embeddings.py: detect CLIP/SigLIP dual encoders and drive them through
      transformers get_text_features/get_image_features so text and images share
      one projected space (ST path kept for repos shipping a native recipe).
      request.image is now actually read (URL/data-URI/path/base64), vectors are
      appended after the text ones, and text-only models return a clear 400.
      Handle the transformers 5.x pooled-output return shape.
    - Wrap the loaded model in _EmbeddingModel (unpacks as (backend, model) but
      exposes cleanup()) and register it via add_model() + record_vram_delta() on
      the request path, so it is measured, LRU-tracked, and cleanly evicted like
      every other model type instead of leaking as a bare tuple.
    - admin: fix the model-load button for embeddings (was routed to the diffusers
      loader) to use _load_embedding_model, matching the request path.
    - admin: backfill used_vram_gb after a download completes, since the entry is
      saved before the weights exist on disk; factor the estimate into a shared
      _estimate_used_vram_gb helper. A freshly-downloaded CLIP/SigLIP entry now
      always carries a reasonable estimate so pre-load eviction sizes correctly.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01EPLnsRpNBzWCHLgkXATqRz
    dd848607
Name
Last commit
Last update
..
static Loading commit data...
templates Loading commit data...
__init__.py Loading commit data...
auth.py Loading commit data...
download_worker.py Loading commit data...
routes.py Loading commit data...