Skip to content
Commit 66f5540b authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

embeddings: native bge-m3 dense+sparse(+colbert) multi-vector support



Add BAAI/bge-m3 hybrid embedding (dense + sparse lexical, optional colbert)
without the FlagEmbedding dependency (which pins older transformers/peft and
would conflict with the transformers 5.x stack): the sparse/colbert heads are
plain Linear layers over XLM-RoBERTa's last hidden state, reproduced natively
with transformers + torch already in the venv.

- new 'bge-m3' embedding backend: detected by xlm-roberta model_type + a
  sparse_linear.pt head (so plain bge-large etc. stay on the dense ST path);
  loads the base model + sparse_linear.pt (+ colbert_linear.pt if present).
- _bge_m3_encode: dense = L2-normalized CLS; sparse = max-pooled
  relu(sparse_linear·h) per token id (specials dropped) as {token_id: weight};
  colbert = L2-normalized colbert_linear·h per content token.
- /v1/embeddings extended: request.embedding_types selects any of
  dense|sparse|colbert (overrides the model-config default); response items
  gain sparse_embedding {indices, values} and colbert_embedding. Dense stays
  in the standard `embedding` field (base64 honored). Configurable default via
  the model's embedding_types config.
- cleanup()/eviction branch moves the base model + heads off GPU.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
parent 2c419247
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment