pods: a compiler in the shared capability base, for Triton (v0.2.80)

Sweeping the pod images for the gap the LongCat one had: Triton compiles its
CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler.
Thirteen other torch-bearing profiles did not.

The risk is not theoretical and not confined to block-sparse attention.
core.txt — which every capability image installs — carries bitsandbytes, and
bitsandbytes reaches Triton's autotuner on import; that is how the NF4
verification failed first time round, with "libcuda.so cannot found!" and then a
failed gcc invocation. On a pod the first quantised forward would have died with
"Failed to find C compiler", exactly as Surya did locally before v0.2.62.

So gcc and libc6-dev go into Dockerfile.capability-base, where every profile
image inherits them, rather than into thirteen profiles one at a time.

capability-base-light is deliberately left without one: it ships no torch, so it
has no Triton to build for. That is precisely why the LongCat image installs its
own — it adds a 3.10 venv with torch and triton to a parent that has neither —
and there is now a test saying so, so a later cleanup does not "simplify" it
away.

Also: the quantised text-encoder load now reads torch.cuda.current_device()
instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so
0 was right here, but assuming it would not survive an install that does its
device selection differently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.2.79"
+__version__ = "0.2.80"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

packaging/runpod/Dockerfile.capability-base

@@ -24,8 +24,16 @@ ENV PYTHONUNBUFFERED=1 \
 
 # ffmpeg for every audio/video path; git because a few ML packages still want it
 # at install time; libgl/libglib for opencv.
+# gcc and libc6-dev are RUNTIME requirements, not build ones. Triton compiles its CUDA
+# driver module the first time anything uses a Triton kernel, and this base carries
+# torch AND bitsandbytes (core.txt) — bitsandbytes reaches Triton's autotuner on import.
+# Without them the first quantised forward dies with "Failed to find C compiler", the
+# way Surya did locally before v0.2.62. Linking also needs libcuda.so, which the nvidia
+# container runtime provides on a pod.
 RUN apt-get update && apt-get install -y --no-install-recommends \
         ffmpeg git curl ca-certificates libgl1 libglib2.0-0 \
+        gcc \
+        libc6-dev \
     && rm -rf /var/lib/apt/lists/*
 
 # Installing NPP is not enough: torch preloads its own nvidia libs and NPP is

tests/test_image_has_c_compiler.py

@@ -29,7 +29,11 @@ DOCKERFILES = [ROOT / "packaging" / "linux" / "Dockerfile.oci",
                # The LongCat pod image too: block-sparse attention is pure Triton, so a
                # pod without a compiler dies on the first denoising step. Its bases
                # (capability-base-light) carry no compiler, so it must add its own.
-               ROOT / "packaging" / "runpod" / "Dockerfile.capability-video-longcat"]
+               ROOT / "packaging" / "runpod" / "Dockerfile.capability-video-longcat",
+               # The shared capability base: it carries torch and, via core.txt,
+               # bitsandbytes — which reaches Triton's autotuner on import. Every
+               # profile image built FROM it inherits the gap otherwise.
+               ROOT / "packaging" / "runpod" / "Dockerfile.capability-base"]
 RUNPOD_VLLM_APT = ROOT / "packaging" / "runpod" / "profiles" / "vllm.apt"
 
 
@@ -69,3 +73,17 @@ def test_the_runpod_vllm_profile_agrees():
     pkgs = [l.strip() for l in RUNPOD_VLLM_APT.read_text(encoding="utf-8").splitlines()
             if l.strip() and not l.strip().startswith("#")]
     assert "gcc" in pkgs
+
+def test_the_light_base_is_deliberately_left_without_one():
+    """capability-base-light ships no torch on purpose, so it has no Triton to build
+    for. Anything that adds its own torch there — the LongCat 3.10 venv — must bring
+    its own compiler, which is why Dockerfile.capability-video-longcat installs one
+    even though its parent does not.
+    """
+    light = (ROOT / "packaging" / "runpod" /
+             "Dockerfile.capability-base-light").read_text(encoding="utf-8")
+    assert "torch" not in light.split("FROM")[-1].lower() or "No torch here" in light
+    longcat = (ROOT / "packaging" / "runpod" /
+               "Dockerfile.capability-video-longcat").read_text(encoding="utf-8")
+    assert re.search(r"^\s+gcc\s*\\?\s*$", longcat, re.M), \
+        "the light base gives it nothing, so it must install its own"

tools/longcat_service.py

@@ -123,7 +123,12 @@ def load_pipeline(checkpoint_dir: str, source_dir: str, dtype: str = "bfloat16",
         if _te_cfg is not None:
             # device_map is required for a quantised load: bitsandbytes quantises as the
             # weights land on the card.
-            te_kw.update(quantization_config=_te_cfg, device_map={"": 0})
+            # The worker masks the card with CUDA_VISIBLE_DEVICES, so the selected
+            # GPU is index 0 in this process — but read the current device rather than
+            # assume it, so this agrees with pipe.to("cuda") on any install.
+            te_kw.update(quantization_config=_te_cfg,
+                         device_map={"": torch.cuda.current_device()
+                                     if torch.cuda.is_available() else 0})
             log(f"text encoder: {te_quant} "
                 f"(~{11.0 * LC.component_quant_fraction(te_quant):.1f} GB instead of "
                 f"~11 GB at bf16 — the embedding table does not quantise) "