• Stefy Lanza (nextime / spora )'s avatar
    pods: a compiler in the shared capability base, for Triton (v0.2.80) · ec6f5c30
    Stefy Lanza (nextime / spora ) authored
    Sweeping the pod images for the gap the LongCat one had: Triton compiles its
    CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler.
    Thirteen other torch-bearing profiles did not.
    
    The risk is not theoretical and not confined to block-sparse attention.
    core.txt — which every capability image installs — carries bitsandbytes, and
    bitsandbytes reaches Triton's autotuner on import; that is how the NF4
    verification failed first time round, with "libcuda.so cannot found!" and then a
    failed gcc invocation. On a pod the first quantised forward would have died with
    "Failed to find C compiler", exactly as Surya did locally before v0.2.62.
    
    So gcc and libc6-dev go into Dockerfile.capability-base, where every profile
    image inherits them, rather than into thirteen profiles one at a time.
    
    capability-base-light is deliberately left without one: it ships no torch, so it
    has no Triton to build for. That is precisely why the LongCat image installs its
    own — it adds a 3.10 venv with torch and triton to a parent that has neither —
    and there is now a test saying so, so a later cleanup does not "simplify" it
    away.
    
    Also: the quantised text-encoder load now reads torch.cuda.current_device()
    instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so
    0 was right here, but assuming it would not survive an install that does its
    device selection differently.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ec6f5c30
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...