-
Stefy Lanza (nextime / spora ) authored
Wire up real image embeddings and make embedding models first-class in the VRAM lifecycle. - api/embeddings.py: detect CLIP/SigLIP dual encoders and drive them through transformers get_text_features/get_image_features so text and images share one projected space (ST path kept for repos shipping a native recipe). request.image is now actually read (URL/data-URI/path/base64), vectors are appended after the text ones, and text-only models return a clear 400. Handle the transformers 5.x pooled-output return shape. - Wrap the loaded model in _EmbeddingModel (unpacks as (backend, model) but exposes cleanup()) and register it via add_model() + record_vram_delta() on the request path, so it is measured, LRU-tracked, and cleanly evicted like every other model type instead of leaking as a bare tuple. - admin: fix the model-load button for embeddings (was routed to the diffusers loader) to use _load_embedding_model, matching the request path. - admin: backfill used_vram_gb after a download completes, since the entry is saved before the weights exist on disk; factor the estimate into a shared _estimate_used_vram_gb helper. A freshly-downloaded CLIP/SigLIP entry now always carries a reasonable estimate so pre-load eviction sizes correctly. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPLnsRpNBzWCHLgkXATqRz
dd848607