• Stefy Lanza (nextime / spora )'s avatar
    record why a Qwen projector loads, injects, and describes every image the same · 5c549fd3
    Stefy Lanza (nextime / spora ) authored
    Qwen3.8-27B served images for months and answered identically for all of them.
    Four plausible explanations were wrong, two of my own inferences were wrong, and
    only an experiment settled it — so the whole thing is written down rather than
    left as a line in a changelog.
    
    Cleared with evidence: the projector pairs correctly (projection_dim 5120 = the
    model's hidden_size), the missing deepstack tensors are missing from unsloth's
    reference conversion too so this family does not use them, clip_graph_qwen3vl IS
    implemented in libmtmd, the base model genuinely is a VLM with vision_config and
    image_token_id 248056, the GGUF tokenizer carries all three vision markers at
    the ids the base config names, the mmproj does load, coderai's vision guard does
    fire, and the 1024-token floor is llama.cpp's own requirement.
    
    Also wrong, and worth recording as a trap: "no mtmd output at request time, so
    the handler is bypassed". The llama.cpp log callback is deliberately swapped to a
    quiet one after the load, so runtime silence proves nothing.
    
    What settled it was two experiments. gemma-4-14b-8k with its own projector reads
    the same two images correctly ('Red square.' / 'Blue circle.') and invents when
    given none — so the pipeline is sound end to end. And the same Qwen model with a
    DIFFERENT publisher's projector fails identically, so it is not one bad artifact.
    The fault is upstream: either llama.cpp's qwen3vl_merger graph is wrong for this
    model, or both converters mislabel it — the base is qwen3_5 while the projector
    declares qwen3vl_merger, so the wrong graph may simply be running.
    
    mmproj is removed from the qwen38 cards. A model that answers confidently and
    wrongly is worse than one that declines, and the listing now says text only.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    5c549fd3
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-acestep.txt Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-latentsync.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-musetalk.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-parler.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements-xtts.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...