-
Stefy Lanza (nextime / spora ) authored
Sweeping the pod images for the gap the LongCat one had: Triton compiles its CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler. Thirteen other torch-bearing profiles did not. The risk is not theoretical and not confined to block-sparse attention. core.txt — which every capability image installs — carries bitsandbytes, and bitsandbytes reaches Triton's autotuner on import; that is how the NF4 verification failed first time round, with "libcuda.so cannot found!" and then a failed gcc invocation. On a pod the first quantised forward would have died with "Failed to find C compiler", exactly as Surya did locally before v0.2.62. So gcc and libc6-dev go into Dockerfile.capability-base, where every profile image inherits them, rather than into thirteen profiles one at a time. capability-base-light is deliberately left without one: it ships no torch, so it has no Triton to build for. That is precisely why the LongCat image installs its own — it adds a 3.10 venv with torch and triton to a parent that has neither — and there is now a test saying so, so a later cleanup does not "simplify" it away. Also: the quantised text-encoder load now reads torch.cuda.current_device() instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so 0 was right here, but assuming it would not survive an install that does its device selection differently. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
ec6f5c30