• Stefy Lanza (nextime / spora )'s avatar
    prefer VRAM by default, and let the suite run where production runs · 1ad16f15
    Stefy Lanza (nextime / spora ) authored
    Offload: the default becomes auto-borderline, not auto. `auto` condemns a model
    on its ESTIMATE -- if used_vram_gb exceeds free VRAM it goes straight to CPU
    offload without ever trying the card -- and that estimate is a measurement from
    an earlier load in some cards and a hand-entered round number in others
    (LTX-2.3-PinkCherry declared exactly 24.0 on a 24 GiB card). So a conservative
    figure permanently exiled a model that would have fit.
    
    The cost of being wrong is asymmetric, which is what decides it: guessing "fits"
    wrongly costs one failed attempt and a fallback, while guessing "does not fit"
    wrongly costs a CPU copy of every component and a PCIe trip per request for the
    life of the install -- 28-37 GB of host RAM per video render on a 54 GB box with
    no cgroup limit to stop it. The dropdown is relabelled by intent and reordered,
    because the old labels were the trap: `auto` read as the sensible choice while
    being the pessimistic one.
    
    Tests: three files hardcoded /storage/coderai, so collection FAILED outright
    inside the production image and every test in them had only ever been exercised
    against the host's dependency set -- which is diffusers 0.37.0.dev0 with no
    torchsde and no PyAV, against the image's 0.42.1 with both. The four Krea 2 edit
    tests were silently skipping for want of Krea2Pipeline and now run (15 passed,
    0 skipped in-image). Paths are derived from __file__ instead.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1ad16f15
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pipelines Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...