• Stefy Lanza (nextime / spora )'s avatar
    runpod: optionally keep pod dependencies on the volume, not in the image · 98c02766
    Stefy Lanza (nextime / spora ) authored
    A capability image is ~7 GB and nearly all of it is torch, pulled again by
    every cold pod. With a network volume those libraries can live there
    instead: `venv_on_volume` boots a slim image — OS, Python and coderai
    only — and runs coderai from a venv on the volume. The first pod builds
    it; every pod after skips both the pull and the install.
    
    Opt-in per pool and per model, never a default, because the trade is real
    and unmeasured: the pull gets much smaller, but importing torch off
    network storage is slower than off local disk, so some of the saving
    comes back at import time. Whether that is a net win depends on how often
    pods cold-start, which is the user's workload, not mine to assume.
    
    Two hazards are handled rather than left to chance. The ready marker is
    written AFTER the install, so a pod that dies mid-build leaves nothing a
    later pod would import from. And a lock directory — mkdir is atomic —
    makes a second pod wait for the first instead of two installs
    interleaving into the same directory.
    
    `venv_on_volume` without a volume is refused with the reason, rather than
    booting a pod that installs 7 GB onto container disk and discards it.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    98c02766
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...