• Stefy Lanza (nextime / spora )'s avatar
    prefer VRAM by default, and let the suite run where production runs · 1ad16f15
    Stefy Lanza (nextime / spora ) authored
    Offload: the default becomes auto-borderline, not auto. `auto` condemns a model
    on its ESTIMATE -- if used_vram_gb exceeds free VRAM it goes straight to CPU
    offload without ever trying the card -- and that estimate is a measurement from
    an earlier load in some cards and a hand-entered round number in others
    (LTX-2.3-PinkCherry declared exactly 24.0 on a 24 GiB card). So a conservative
    figure permanently exiled a model that would have fit.
    
    The cost of being wrong is asymmetric, which is what decides it: guessing "fits"
    wrongly costs one failed attempt and a fallback, while guessing "does not fit"
    wrongly costs a CPU copy of every component and a PCIe trip per request for the
    life of the install -- 28-37 GB of host RAM per video render on a 54 GB box with
    no cgroup limit to stop it. The dropdown is relabelled by intent and reordered,
    because the old labels were the trap: `auto` read as the sensible choice while
    being the pessimistic one.
    
    Tests: three files hardcoded /storage/coderai, so collection FAILED outright
    inside the production image and every test in them had only ever been exercised
    against the host's dependency set -- which is diffusers 0.37.0.dev0 with no
    torchsde and no PyAV, against the image's 0.42.1 with both. The four Krea 2 edit
    tests were silently skipping for want of Krea2Pipeline and now run (15 passed,
    0 skipped in-image). Paths are derived from __file__ instead.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1ad16f15
config.py 172 KB