-
Stefy Lanza (nextime / spora ) authored
Add BAAI/bge-m3 hybrid embedding (dense + sparse lexical, optional colbert) without the FlagEmbedding dependency (which pins older transformers/peft and would conflict with the transformers 5.x stack): the sparse/colbert heads are plain Linear layers over XLM-RoBERTa's last hidden state, reproduced natively with transformers + torch already in the venv. - new 'bge-m3' embedding backend: detected by xlm-roberta model_type + a sparse_linear.pt head (so plain bge-large etc. stay on the dense ST path); loads the base model + sparse_linear.pt (+ colbert_linear.pt if present). - _bge_m3_encode: dense = L2-normalized CLS; sparse = max-pooled relu(sparse_linear·h) per token id (specials dropped) as {token_id: weight}; colbert = L2-normalized colbert_linear·h per content token. - /v1/embeddings extended: request.embedding_types selects any of dense|sparse|colbert (overrides the model-config default); response items gain sparse_embedding {indices, values} and colbert_embedding. Dense stays in the standard `embedding` field (base64 honored). Configurable default via the model's embedding_types config. - cleanup()/eviction branch moves the base model + heads off GPU. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
66f5540b