• Stefy Lanza (nextime / spora )'s avatar
    an image-only embedder must refuse text, not hang on it · 1ce4e50e
    Stefy Lanza (nextime / spora ) authored
    SALAD answered four sentences with nothing at all: no response in 180s, recorded
    by the sweep as a 500 with no error logged anywhere, while dinov2 given the
    identical request returned a clean 400 immediately.
    
    The refusal for the other image-only models keys on the backend name
    (`vision`, `dinov2cpp`). SALAD is `vpr-server`, and its contract — like
    EigenPlaces' `vpr` — puts the image in `input` rather than `image`, so the
    refusal cannot key on the field that was used, only on what is in it. `vpr`
    decodes locally and so fails cleanly; `vpr-server` handed the strings to an
    isolated subprocess that could not answer and did not reply. A hang is worse
    than a refusal: it holds a request slot and tells the caller nothing.
    
    Both backends now check their inputs look like image references — the same four
    shapes _decode_image accepts, tested without decoding so a real image is not
    decoded twice. A short base64-valid token is treated as text, because no picture
    is eight bytes and the alternative is sending "hello" to the subprocess again.
    
    The first version of this guard sat inside the embed path, after the load: it
    stopped the hang but answered 400 in 42 seconds, because SALAD had to spawn its
    subprocess first. The call site of the mirror check records that exact lesson
    eleven lines away —
    
        # Refuse an impossible request BEFORE loading anything. The same refusal
        # used to come after the load: images sent to a text-only 8 GB embedder
        # spent forty seconds loading it and then answered 400.
    
    — so repeating it in the other direction was avoidable. _takes_image_in_input
    answers from the model NAME, reading the existing _VPR_IDS/_SALAD_IDS sets rather
    than a second list that would drift from the ones the loaders dispatch on. The
    post-load check stays as a backstop for a model reached by an alias those sets do
    not know.
    
    Measured after: text 400 in 0.13s, a real image still returns its 8448-dim
    embedding.
    
    Also repairs a pre-existing test whose `[:4000]` source window this change
    pushed `_load_embedding_locked` past, so it failed with "substring not found" on
    correct code. It now bounds `_run_embeddings` by structure rather than a byte
    count — the third time a fixed source window has broken in this repo.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1ce4e50e
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pipelines Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...