• Stefy Lanza (nextime / spora )'s avatar
    pods: a compiler in the shared capability base, for Triton (v0.2.80) · ec6f5c30
    Stefy Lanza (nextime / spora ) authored
    Sweeping the pod images for the gap the LongCat one had: Triton compiles its
    CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler.
    Thirteen other torch-bearing profiles did not.
    
    The risk is not theoretical and not confined to block-sparse attention.
    core.txt — which every capability image installs — carries bitsandbytes, and
    bitsandbytes reaches Triton's autotuner on import; that is how the NF4
    verification failed first time round, with "libcuda.so cannot found!" and then a
    failed gcc invocation. On a pod the first quantised forward would have died with
    "Failed to find C compiler", exactly as Surya did locally before v0.2.62.
    
    So gcc and libc6-dev go into Dockerfile.capability-base, where every profile
    image inherits them, rather than into thirteen profiles one at a time.
    
    capability-base-light is deliberately left without one: it ships no torch, so it
    has no Triton to build for. That is precisely why the LongCat image installs its
    own — it adds a 3.10 venv with torch and triton to a parent that has neither —
    and there is now a test saying so, so a later cleanup does not "simplify" it
    away.
    
    Also: the quantised text-encoder load now reads torch.cuda.current_device()
    instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so
    0 was right here, but assuming it would not survive an install that does its
    device selection differently.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ec6f5c30
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...