-
Stefy Lanza (nextime / spora ) authored
bge-m3's sparse and colbert come from linear heads (sparse_linear.pt / colbert_linear.pt) applied to the last hidden state -- a transformers forward pass, not an embedding server's output. Two consequences that were both silent: A DENSE-ONLY BACKEND NOW SAYS SO. `embedding_types` was simply ignored outside the native path: a caller asking for sparse got a normal-looking response with only `embedding` in it, its hybrid retrieval then ran on half the signal, and the result was merely a worse score -- which reads as a model problem, not a configuration one. Now a 400 that names the cause, where it DOES work (coderai-embeddings), and why the vLLM image cannot: that image is LIGHT core with no torch in its main venv BY DESIGN, so _load_bge_m3 could not even import. Dense-only requests are untouched. MULTI-VECTOR LONG INPUTS ARE WINDOWED. 8192 tokens is ~6000 words, so this bites on whole documents rather than passages -- and when it bit, the tokenizer's `truncation=True` kept the opening and said nothing. Each type is merged the way the encoder already merges WITHIN a window, which is what makes these choices defensible rather than invented: dense mean, RE-NORMALIZED. It is an L2-normalized CLS vector and a mean of unit vectors is not one; cosine similarity on a non-unit vector is quietly wrong. sparse MAX per token id -- the encoder max-pools per token id already (scatter_reduce amax), and a term's weight should be its strongest occurrence, not an average diluted by the pages that never mention it. A test guards against someone later tidying max into mean. colbert CONCATENATED: max-sim scores each query token against ALL document token vectors, so every content token of every window belongs. `embed_long_input=truncate` keeps the old behaviour explicitly, and `first` keeps only the opening window, for an operator who would rather have that than a merge. Also removes three customer references I had put in shipped profile comments, two of which went out in 0.3.69. A capability must be scoped and justified by what it does, never by one install's needs, and a customer name does not belong in a released image. The measurements keep their force without the attribution: "hybrid retrieval wants dense and sparse from one pass", "a production fleet", "a GPU-less orchestrator". 39 tests. Suite at the 25 pre-existing failures. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
da2859f0
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| common | ||
| dinov2cpp | ||
| linux | ||
| runpod | ||
| windows | ||
| build-rpc-server.sh | ||
| cosign.pub | ||
| patch-colibri.py | ||
| patch-k3.py | ||
| setup-salad-venv.sh | ||
| sign-images.sh | ||
| versions.env |