• Stefy Lanza (nextime / spora )'s avatar
    embeddings: native bge-m3 dense+sparse(+colbert) multi-vector support · 66f5540b
    Stefy Lanza (nextime / spora ) authored
    Add BAAI/bge-m3 hybrid embedding (dense + sparse lexical, optional colbert)
    without the FlagEmbedding dependency (which pins older transformers/peft and
    would conflict with the transformers 5.x stack): the sparse/colbert heads are
    plain Linear layers over XLM-RoBERTa's last hidden state, reproduced natively
    with transformers + torch already in the venv.
    
    - new 'bge-m3' embedding backend: detected by xlm-roberta model_type + a
      sparse_linear.pt head (so plain bge-large etc. stay on the dense ST path);
      loads the base model + sparse_linear.pt (+ colbert_linear.pt if present).
    - _bge_m3_encode: dense = L2-normalized CLS; sparse = max-pooled
      relu(sparse_linear·h) per token id (specials dropped) as {token_id: weight};
      colbert = L2-normalized colbert_linear·h per content token.
    - /v1/embeddings extended: request.embedding_types selects any of
      dense|sparse|colbert (overrides the model-config default); response items
      gain sparse_embedding {indices, values} and colbert_embedding. Dense stays
      in the standard `embedding` field (base64 honored). Configurable default via
      the model's embedding_types config.
    - cleanup()/eviction branch moves the base model + heads off GPU.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    66f5540b
embeddings.py 82.7 KB