• Stefy Lanza (nextime / spora )'s avatar
    quant: fix settings showing "GPTQModel not installed" when it is installed · e72e33eb
    Stefy Lanza (nextime / spora ) authored
    is_available() reported false in the running engine even though gptqmodel + fast
    kernels import fine in a fresh process. Two causes:
    
    1. capabilities() cached a DEGRADED result (gptqmodel imported but the inner
       gptqmodel.utils.backend BACKEND import transiently came up empty, e.g. when the
       first call landed mid model-load). That empty-backends result stuck for the whole
       process life, so the settings page said "GPTQModel not installed" until restart.
       Now a degraded (available-but-no-backends) result is NOT cached — re-detect next
       call; only a clean positive or a genuine ImportError is cached.
    
    2. is_available() gated on a SPECIFIC fast kernel being detected. GPTQModel always
       has a Triton/torch fallback and picks the kernel at load, so availability now
       gates only on gptqmodel importing; backends stay informational.
    
    Also: /admin/api/quantize-capabilities re-detects live (capabilities(refresh=True))
    so the settings page never serves a stale cache.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    e72e33eb
quant.py 14.9 KB