• Stefy Lanza (nextime / spora )'s avatar
    runpod: optionally keep pod dependencies on the volume, not in the image · 98c02766
    Stefy Lanza (nextime / spora ) authored
    A capability image is ~7 GB and nearly all of it is torch, pulled again by
    every cold pod. With a network volume those libraries can live there
    instead: `venv_on_volume` boots a slim image — OS, Python and coderai
    only — and runs coderai from a venv on the volume. The first pod builds
    it; every pod after skips both the pull and the install.
    
    Opt-in per pool and per model, never a default, because the trade is real
    and unmeasured: the pull gets much smaller, but importing torch off
    network storage is slower than off local disk, so some of the saving
    comes back at import time. Whether that is a net win depends on how often
    pods cold-start, which is the user's workload, not mine to assume.
    
    Two hazards are handled rather than left to chance. The ready marker is
    written AFTER the install, so a pod that dies mid-build leaves nothing a
    later pod would import from. And a lock directory — mkdir is atomic —
    makes a second pod wait for the first instead of two installs
    interleaving into the same directory.
    
    `venv_on_volume` without a volume is refused with the reason, rather than
    booting a pod that installs 7 GB onto container disk and discards it.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    98c02766
Name
Last commit
Last update
..
superpowers Loading commit data...
README.md Loading commit data...
deepseek-ds4.md Loading commit data...
dtype-auto-selection.md Loading commit data...
expressive-tts.md Loading commit data...
frontend-engine-split.md Loading commit data...
gguf-process-isolation.md Loading commit data...
glm-colibri.md Loading commit data...
kimi-k3.md Loading commit data...
ktransformers.md Loading commit data...
minimax-h3.md Loading commit data...
ocr.md Loading commit data...
process-isolation-plans.md Loading commit data...
reference-projects.md Loading commit data...
remote-execution.md Loading commit data...
reverse-proxy-nginx.md Loading commit data...
runpod.md Loading commit data...
tools-as-services.md Loading commit data...
vllm.md Loading commit data...
zimage-lora-training.md Loading commit data...