• Stefy Lanza (nextime / spora )'s avatar
    manager: universal footprint rule — already-quantized (cached) models measured... · 88c1e787
    Stefy Lanza (nextime / spora ) authored
    manager: universal footprint rule — already-quantized (cached) models measured as-is, no quant factor
    
    Per directive: the VRAM estimate must always equal the model's ACTUAL full pipeline
    footprint, by a universal rule rather than per-model tuning. The quant factor exists
    to convert an UNQUANTIZED source's weight size down to its quantized runtime size —
    so it must NOT be applied to weights that are ALREADY quantized on disk.
    
    Add _quantized_cache_gb(): find this model's quantized pipeline cache by MODEL NAME
    (glob '<safe_name>__*', ignoring in-progress '.building' dirs; signature suffix
    drifts with trivial config edits, so name-match is the robust signal) and return its
    real on-disk size. _get_model_used_vram_gb now, right after the forced-measurement
    check, returns that cache size AS-IS (+ reserve) when present — no quant_mult, no
    precision factor. The cache holds already-quantized weights, so its size IS the
    footprint, and it overrides a stale used_vram_gb (e.g. Wan2.2-I2V's 151 GB fp32 disk
    size). Models quantized AT LOAD (no cache yet) still fall through to the factor-based
    estimate. Verified: VACE-Fun now estimates ~24 GB (real 4-bit footprint) instead of
    24.5 x 0.283 = 6.9 GB, so eviction frees the idle image model before the forward.
    
    Reverted the per-model measured_vram_gb pin in models.json — the universal rule
    covers it.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    88c1e787
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...