• Stefy Lanza (nextime / spora )'s avatar
    embeddings: native GGUF Qwen2-VL image embeddings via llama.cpp mtmd · a3e473c6
    Stefy Lanza (nextime / spora ) authored
    A GGUF embedder with a configured `mmproj` now loads as a vision-capable
    'llama-vl' backend: the mtmd image tower feeds an embeddings context
    (pooling LAST) under the GME chat prompt — the same scheme as the HF
    qwenvl backend, so both produce the same vector space (~0.89 cosine
    agreement at Q4). Runs natively on whatever the llama.cpp build targets
    (Vulkan on radeon). Text goes through the same prompt+last-token path
    so both modalities share the GME space; PIL-decoded images are passed
    as raw RGB bitmaps (mtmd's stb can't read AVIF). Serialized on the
    per-model lock; cleanup frees the mtmd ctx.
    
    Reverts the front's image-reroute-to-HF-sibling: a model pinned to an
    engine must be served there, not silently moved.
    
    The gme mmproj was generated from the local HF checkpoint with
    llama.cpp's convert_hf_to_gguf.py --mmproj (F16, 1.3 GB).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
    a3e473c6
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...