-
Stefy Lanza (nextime / spora ) authored
A GGUF embedder with a configured `mmproj` now loads as a vision-capable 'llama-vl' backend: the mtmd image tower feeds an embeddings context (pooling LAST) under the GME chat prompt — the same scheme as the HF qwenvl backend, so both produce the same vector space (~0.89 cosine agreement at Q4). Runs natively on whatever the llama.cpp build targets (Vulkan on radeon). Text goes through the same prompt+last-token path so both modalities share the GME space; PIL-decoded images are passed as raw RGB bitmaps (mtmd's stb can't read AVIF). Serialized on the per-model lock; cleanup frees the mtmd ctx. Reverts the front's image-reroute-to-HF-sibling: a model pinned to an engine must be served there, not silently moved. The gme mmproj was generated from the local HF checkpoint with llama.cpp's convert_hf_to_gguf.py --mmproj (F16, 1.3 GB). Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
a3e473c6