• Stefy Lanza (nextime / spora )'s avatar
    pods: a compiler in the shared capability base, for Triton (v0.2.80) · ec6f5c30
    Stefy Lanza (nextime / spora ) authored
    Sweeping the pod images for the gap the LongCat one had: Triton compiles its
    CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler.
    Thirteen other torch-bearing profiles did not.
    
    The risk is not theoretical and not confined to block-sparse attention.
    core.txt — which every capability image installs — carries bitsandbytes, and
    bitsandbytes reaches Triton's autotuner on import; that is how the NF4
    verification failed first time round, with "libcuda.so cannot found!" and then a
    failed gcc invocation. On a pod the first quantised forward would have died with
    "Failed to find C compiler", exactly as Surya did locally before v0.2.62.
    
    So gcc and libc6-dev go into Dockerfile.capability-base, where every profile
    image inherits them, rather than into thirteen profiles one at a time.
    
    capability-base-light is deliberately left without one: it ships no torch, so it
    has no Triton to build for. That is precisely why the LongCat image installs its
    own — it adds a 3.10 venv with torch and triton to a parent that has neither —
    and there is now a test saying so, so a later cleanup does not "simplify" it
    away.
    
    Also: the quantised text-encoder load now reads torch.cuda.current_device()
    instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so
    0 was right here, but assuming it would not survive an install that does its
    device selection differently.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ec6f5c30
__init__.py 4.21 KB