• Stefy Lanza (nextime / spora )'s avatar
    runpod: warn loudly before a 150 GB model is put on a rented card · b49b4936
    Stefy Lanza (nextime / spora ) authored
    Everything around pods is tuned for images of ~10 GB and models of a few. The
    native MoE engines are a different shape: DeepSeek-V4 is ~154 GB and they go
    well past that, so a pod spends one to two hours filling its disk before it
    answers anything — on EVERY cold start, since a pod is disposable storage.
    
    The warning names the model, the size, and the fix: attach a network volume so
    the weights are fetched once and every later pod attaches to them instead. It
    appears at provision time, boxed so it cannot be missed in a log, and in the
    test run — which is where someone looks BEFORE committing to a configuration,
    while at provision time the money is already being spent.
    
    It says nothing when a volume is attached, and nothing for ordinary models.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    b49b4936
model_test.py 28.6 KB