• Stefy Lanza (nextime / spora )'s avatar
    fix: config-key shadow dropped mmproj on GGUF vision model reload · 1126bdbc
    Stefy Lanza (nextime / spora ) authored
    A GGUF vision model (e.g. Gemma-4-14B) served correct image descriptions
    on its FIRST load after a restart but hallucinated an identical answer for
    every image on every subsequent load — the image was silently flattened to
    a "[image_url content]" text placeholder and the model never saw pixels.
    
    Root cause was a config-key mismatch that self-polluted the in-memory
    config. On-demand loads arrive as a basename (Gemma-...gguf) while the real
    models.json entry is keyed by full path. record_vram_delta() resolved the
    write target via _config_for_model_key(), which — unlike _config_for_model()
    — did NOT fall back to basename/alias matching, so it returned {} and then
    persisted a NEW basename-keyed entry holding ONLY the measured_* fields (no
    mmproj, no n_ctx). On the next load _config_for_model()'s exact-match hit
    that stripped basename entry FIRST, before the basename loop that would have
    found the real full-path config, so mmproj was dropped, supports_vision went
    False, and the vision projector never loaded.
    
    Fix:
    - Add _resolve_config_key(): returns the actual self.config key for a model
      (exact -> alias -> basename), the single canonical key readers and writers
      must agree on.
    - Route _config_for_model() through it; give _config_for_model_key() the same
      basename/alias fallback so it can no longer return {} for a basename.
    - record_vram_delta()/_persist() now read and persist measured fields under
      the canonical key, merging into the real config instead of spawning a
      stripped shadow entry.
    
    Verified: after restart the 14B loads with "mmproj ... (vision enabled)" on
    every reload and three distinct test images produce three distinct, accurate
    descriptions; the measured-VRAM writeback now logs "(force_vram_update)"
    (real config resolved) instead of the old "(no used_vram_gb)" (empty config).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
    1126bdbc
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...