• Stefy Lanza (nextime / spora )'s avatar
    runpod: optionally keep pod dependencies on the volume, not in the image · 98c02766
    Stefy Lanza (nextime / spora ) authored
    A capability image is ~7 GB and nearly all of it is torch, pulled again by
    every cold pod. With a network volume those libraries can live there
    instead: `venv_on_volume` boots a slim image — OS, Python and coderai
    only — and runs coderai from a venv on the volume. The first pod builds
    it; every pod after skips both the pull and the install.
    
    Opt-in per pool and per model, never a default, because the trade is real
    and unmeasured: the pull gets much smaller, but importing torch off
    network storage is slower than off local disk, so some of the saving
    comes back at import time. Whether that is a net win depends on how often
    pods cold-start, which is the user's workload, not mine to assume.
    
    Two hazards are handled rather than left to chance. The ready marker is
    written AFTER the install, so a pod that dies mid-build leaves nothing a
    later pod would import from. And a lock directory — mkdir is atomic —
    makes a second pod wait for the first instead of two installs
    interleaving into the same directory.
    
    `venv_on_volume` without a volume is refused with the reason, rather than
    booting a pod that installs 7 GB onto container disk and discards it.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    98c02766
Name
Last commit
Last update
..
archive.html Loading commit data...
base.html Loading commit data...
change_password.html Loading commit data...
chat.html Loading commit data...
dashboard.html Loading commit data...
login.html Loading commit data...
models.html Loading commit data...
runpod.html Loading commit data...
settings.html Loading commit data...
tasks.html Loading commit data...
tokens.html Loading commit data...
users.html Loading commit data...