• Stefy Lanza (nextime / spora )'s avatar
    fix: config-key shadow dropped mmproj on GGUF vision model reload · 1126bdbc
    Stefy Lanza (nextime / spora ) authored
    A GGUF vision model (e.g. Gemma-4-14B) served correct image descriptions
    on its FIRST load after a restart but hallucinated an identical answer for
    every image on every subsequent load — the image was silently flattened to
    a "[image_url content]" text placeholder and the model never saw pixels.
    
    Root cause was a config-key mismatch that self-polluted the in-memory
    config. On-demand loads arrive as a basename (Gemma-...gguf) while the real
    models.json entry is keyed by full path. record_vram_delta() resolved the
    write target via _config_for_model_key(), which — unlike _config_for_model()
    — did NOT fall back to basename/alias matching, so it returned {} and then
    persisted a NEW basename-keyed entry holding ONLY the measured_* fields (no
    mmproj, no n_ctx). On the next load _config_for_model()'s exact-match hit
    that stripped basename entry FIRST, before the basename loop that would have
    found the real full-path config, so mmproj was dropped, supports_vision went
    False, and the vision projector never loaded.
    
    Fix:
    - Add _resolve_config_key(): returns the actual self.config key for a model
      (exact -> alias -> basename), the single canonical key readers and writers
      must agree on.
    - Route _config_for_model() through it; give _config_for_model_key() the same
      basename/alias fallback so it can no longer return {} for a basename.
    - record_vram_delta()/_persist() now read and persist measured fields under
      the canonical key, merging into the real config instead of spawning a
      stripped shadow entry.
    
    Verified: after restart the 14B loads with "mmproj ... (vision enabled)" on
    every reload and three distinct test images produce three distinct, accurate
    descriptions; the measured-VRAM writeback now logs "(force_vram_update)"
    (real config resolved) instead of the old "(no used_vram_gb)" (empty config).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
    1126bdbc
Name
Last commit
Last update
..
cache Loading commit data...
__init__.py Loading commit data...
acceleration.py Loading commit data...
capabilities.py Loading commit data...
gpu_lock.py Loading commit data...
gpu_query.py Loading commit data...
grammar.py Loading commit data...
hf_loading.py Loading commit data...
manager.py Loading commit data...
parser.py Loading commit data...
peft_compat.py Loading commit data...
pipeline_cache.py Loading commit data...
quant.py Loading commit data...
ram_monitor.py Loading commit data...
templates.py Loading commit data...
thermal.py Loading commit data...
tmp_janitor.py Loading commit data...
tool_call_grammar.gbnf Loading commit data...
turboquant.py Loading commit data...
utils.py Loading commit data...