• Stefy Lanza (nextime / spora )'s avatar
    embeddings: native GGUF Qwen2-VL image embeddings via llama.cpp mtmd · a3e473c6
    Stefy Lanza (nextime / spora ) authored
    A GGUF embedder with a configured `mmproj` now loads as a vision-capable
    'llama-vl' backend: the mtmd image tower feeds an embeddings context
    (pooling LAST) under the GME chat prompt — the same scheme as the HF
    qwenvl backend, so both produce the same vector space (~0.89 cosine
    agreement at Q4). Runs natively on whatever the llama.cpp build targets
    (Vulkan on radeon). Text goes through the same prompt+last-token path
    so both modalities share the GME space; PIL-decoded images are passed
    as raw RGB bitmaps (mtmd's stb can't read AVIF). Serialized on the
    per-model lock; cleanup frees the mtmd ctx.
    
    Reverts the front's image-reroute-to-HF-sibling: a model pinned to an
    engine must be served there, not silently moved.
    
    The gme mmproj was generated from the local HF checkpoint with
    llama.cpp's convert_hf_to_gguf.py --mmproj (F16, 1.3 GB).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
    a3e473c6
Name
Last commit
Last update
..
__init__.py Loading commit data...
admin_data.py Loading commit data...
app.py Loading commit data...
assignment.py Loading commit data...
engine_supervisor.py Loading commit data...
gpu_detect.py Loading commit data...
registry.py Loading commit data...
reqqueue.py Loading commit data...
router.py Loading commit data...
ui_pages.py Loading commit data...