• Stefy Lanza (nextime / spora )'s avatar
    embeddings: multi-vector on the image that can serve it, and a 400 where it cannot (v0.3.73) · da2859f0
    Stefy Lanza (nextime / spora ) authored
    bge-m3's sparse and colbert come from linear heads (sparse_linear.pt /
    colbert_linear.pt) applied to the last hidden state -- a transformers forward
    pass, not an embedding server's output. Two consequences that were both silent:
    
    A DENSE-ONLY BACKEND NOW SAYS SO. `embedding_types` was simply ignored outside
    the native path: a caller asking for sparse got a normal-looking response with
    only `embedding` in it, its hybrid retrieval then ran on half the signal, and
    the result was merely a worse score -- which reads as a model problem, not a
    configuration one. Now a 400 that names the cause, where it DOES work
    (coderai-embeddings), and why the vLLM image cannot: that image is LIGHT core
    with no torch in its main venv BY DESIGN, so _load_bge_m3 could not even import.
    Dense-only requests are untouched.
    
    MULTI-VECTOR LONG INPUTS ARE WINDOWED. 8192 tokens is ~6000 words, so this bites
    on whole documents rather than passages -- and when it bit, the tokenizer's
    `truncation=True` kept the opening and said nothing. Each type is merged the way
    the encoder already merges WITHIN a window, which is what makes these choices
    defensible rather than invented:
    
      dense    mean, RE-NORMALIZED. It is an L2-normalized CLS vector and a mean of
               unit vectors is not one; cosine similarity on a non-unit vector is
               quietly wrong.
      sparse   MAX per token id -- the encoder max-pools per token id already
               (scatter_reduce amax), and a term's weight should be its strongest
               occurrence, not an average diluted by the pages that never mention
               it. A test guards against someone later tidying max into mean.
      colbert  CONCATENATED: max-sim scores each query token against ALL document
               token vectors, so every content token of every window belongs.
    
    `embed_long_input=truncate` keeps the old behaviour explicitly, and `first`
    keeps only the opening window, for an operator who would rather have that than a
    merge.
    
    Also removes three customer references I had put in shipped profile comments,
    two of which went out in 0.3.69. A capability must be scoped and justified by
    what it does, never by one install's needs, and a customer name does not belong
    in a released image. The measurements keep their force without the attribution:
    "hybrid retrieval wants dense and sparse from one pass", "a production fleet",
    "a GPU-less orchestrator".
    
    39 tests. Suite at the 25 pre-existing failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    da2859f0
Name
Last commit
Last update
..
audio.build-deps Loading commit data...
audio.nodeps Loading commit data...
audio.txt Loading commit data...
core-light.txt Loading commit data...
core.txt Loading commit data...
embeddings.txt Loading commit data...
engines-kt.apt Loading commit data...
engines-kt.light Loading commit data...
engines-kt.txt Loading commit data...
engines-kt.venv-sglang.check Loading commit data...
engines-kt.venv-sglang.txt Loading commit data...
engines-kt.venv-sglang.uv Loading commit data...
engines.dockerfile Loading commit data...
engines.light Loading commit data...
engines.txt Loading commit data...
faceswap.txt Loading commit data...
images.txt Loading commit data...
llama.dockerfile Loading commit data...
llama.light Loading commit data...
llama.txt Loading commit data...
ocr-paddle.light Loading commit data...
ocr-paddle.txt Loading commit data...
ocr-paddle.venv-paddleocr.txt Loading commit data...
ocr.txt Loading commit data...
ocr.venv-surya.check Loading commit data...
ocr.venv-surya.txt Loading commit data...
speaker.light Loading commit data...
speaker.txt Loading commit data...
speaker.venv-pyannote.check Loading commit data...
speaker.venv-pyannote.txt Loading commit data...
stt-crisper.light Loading commit data...
stt-crisper.txt Loading commit data...
stt-crisper.venv-transformers.txt Loading commit data...
stt-nemo.light Loading commit data...
stt-nemo.txt Loading commit data...
stt-nemo.venv-nemo.txt Loading commit data...
stt.txt Loading commit data...
text.txt Loading commit data...
tts-xtts.light Loading commit data...
tts-xtts.txt Loading commit data...
tts-xtts.venv-TTS.check Loading commit data...
tts-xtts.venv-TTS.txt Loading commit data...
tts.txt Loading commit data...
video-longcat.apt Loading commit data...
video-longcat.dockerfile Loading commit data...
video-longcat.light Loading commit data...
video-longcat.txt Loading commit data...
video.txt Loading commit data...
vllm-ocr.apt Loading commit data...
vllm-ocr.light Loading commit data...
vllm-ocr.txt Loading commit data...
vllm-ocr.venv-surya.check Loading commit data...
vllm-ocr.venv-surya.txt Loading commit data...
vllm-ocr.venv-vllm.check Loading commit data...
vllm-ocr.venv-vllm.txt Loading commit data...
vllm-ocr.venv-vllm.uv Loading commit data...
vllm.apt Loading commit data...
vllm.light Loading commit data...
vllm.txt Loading commit data...
vllm.venv-vllm.check Loading commit data...
vllm.venv-vllm.txt Loading commit data...
vllm.venv-vllm.uv Loading commit data...
voice.build-deps Loading commit data...
voice.txt Loading commit data...