-
Stefy Lanza (nextime / spora ) authored
A capability image is ~7 GB and nearly all of it is torch, pulled again by every cold pod. With a network volume those libraries can live there instead: `venv_on_volume` boots a slim image — OS, Python and coderai only — and runs coderai from a venv on the volume. The first pod builds it; every pod after skips both the pull and the install. Opt-in per pool and per model, never a default, because the trade is real and unmeasured: the pull gets much smaller, but importing torch off network storage is slower than off local disk, so some of the saving comes back at import time. Whether that is a net win depends on how often pods cold-start, which is the user's workload, not mine to assume. Two hazards are handled rather than left to chance. The ready marker is written AFTER the install, so a pod that dies mid-build leaves nothing a later pod would import from. And a lock directory — mkdir is atomic — makes a second pod wait for the first instead of two installs interleaving into the same directory. `venv_on_volume` without a volume is refused with the reason, rather than booting a pod that installs 7 GB onto container disk and discards it. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
98c02766
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| static | ||
| templates | ||
| __init__.py | ||
| auth.py | ||
| download_worker.py | ||
| routes.py |