• Stefy Lanza (nextime / spora )'s avatar
    manager: universal footprint rule — already-quantized (cached) models measured... · 88c1e787
    Stefy Lanza (nextime / spora ) authored
    manager: universal footprint rule — already-quantized (cached) models measured as-is, no quant factor
    
    Per directive: the VRAM estimate must always equal the model's ACTUAL full pipeline
    footprint, by a universal rule rather than per-model tuning. The quant factor exists
    to convert an UNQUANTIZED source's weight size down to its quantized runtime size —
    so it must NOT be applied to weights that are ALREADY quantized on disk.
    
    Add _quantized_cache_gb(): find this model's quantized pipeline cache by MODEL NAME
    (glob '<safe_name>__*', ignoring in-progress '.building' dirs; signature suffix
    drifts with trivial config edits, so name-match is the robust signal) and return its
    real on-disk size. _get_model_used_vram_gb now, right after the forced-measurement
    check, returns that cache size AS-IS (+ reserve) when present — no quant_mult, no
    precision factor. The cache holds already-quantized weights, so its size IS the
    footprint, and it overrides a stale used_vram_gb (e.g. Wan2.2-I2V's 151 GB fp32 disk
    size). Models quantized AT LOAD (no cache yet) still fall through to the factor-based
    estimate. Verified: VACE-Fun now estimates ~24 GB (real 4-bit footprint) instead of
    24.5 x 0.283 = 6.9 GB, so eviction frees the idle image model before the forward.
    
    Reverted the per-model measured_vram_gb pin in models.json — the universal rule
    covers it.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    88c1e787
Name
Last commit
Last update
..
cache Loading commit data...
__init__.py Loading commit data...
acceleration.py Loading commit data...
capabilities.py Loading commit data...
gpu_query.py Loading commit data...
grammar.py Loading commit data...
hf_loading.py Loading commit data...
manager.py Loading commit data...
parser.py Loading commit data...
peft_compat.py Loading commit data...
pipeline_cache.py Loading commit data...
quant.py Loading commit data...
ram_monitor.py Loading commit data...
templates.py Loading commit data...
thermal.py Loading commit data...
tmp_janitor.py Loading commit data...
tool_call_grammar.gbnf Loading commit data...
turboquant.py Loading commit data...
utils.py Loading commit data...