runpod: vLLM pods need ninja too, the dep after g++

Triton and torch.inductor JIT-compile vLLM's kernels at run time and need a full
build toolchain for the stubs they emit. Each piece was found on a live olmOCR
pod, one layer at a time, because from outside the failure is identical every
time: vLLM dies before registering and the pod answers 502.

    gcc          the C compiler
    g++          the launcher Triton builds is C++, so gcc alone is not enough
    ninja-build  inductor shells out to ninja for its extensions

Only olmOCR hits this, and the reason is worth recording: it is FP8, so vLLM's
quantisation kernels go down the Triton path. surya-2 is dense bf16 and served
fine on images that had none of these, which is why "the image lacks a compiler"
looked for a while like it could not be the problem.

Applied to vllm.apt AND vllm-ocr.apt, which is copied from it.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
parent 81235f2e
# Triton JIT-compiles vLLM's kernels at run time and needs a compiler for its
# launcher stubs; without one the engine core dies before it registers.
# Triton and torch.inductor JIT-compile vLLM's kernels at run time, and need a
# full build toolchain for the launcher stubs they emit; without it the engine
# core dies before it registers and the pod answers 502.
#
# BOTH compilers, not just gcc. The launcher Triton builds is C++, so it invokes
# g++ — with only gcc present the image looks right, vLLM starts, and then the
# engine dies and the pod answers 502. Found the hard way on an olmOCR pod: the
# TLS problem was fixed, the pod became reachable, and the next layer underneath
# was a missing g++.
# Found one layer at a time, each on a real olmOCR pod:
# gcc — the C compiler
# g++ — the launcher Triton builds is C++, so gcc alone is not enough
# ninja-build — inductor shells out to ninja to build its extensions
# libc6-dev is the headers both compilers need.
#
# Kept (not purged) because the compilation happens at RUN time on the pod, not
# at build time.
gcc
g++
ninja-build
libc6-dev
# Triton JIT-compiles vLLM's kernels at run time and needs a compiler for its
# launcher stubs; without one the engine core dies before it registers.
# Triton and torch.inductor JIT-compile vLLM's kernels at run time, and need a
# full build toolchain for the launcher stubs they emit; without it the engine
# core dies before it registers and the pod answers 502.
#
# BOTH compilers, not just gcc. The launcher Triton builds is C++, so it invokes
# g++ — with only gcc present the image looks right, vLLM starts, and then the
# engine dies and the pod answers 502. Found the hard way on an olmOCR pod: the
# TLS problem was fixed, the pod became reachable, and the next layer underneath
# was a missing g++.
# Found one layer at a time, each on a real olmOCR pod:
# gcc — the C compiler
# g++ — the launcher Triton builds is C++, so gcc alone is not enough
# ninja-build — inductor shells out to ninja to build its extensions
# libc6-dev is the headers both compilers need.
#
# Kept (not purged) because the compilation happens at RUN time on the pod, not
# at build time.
gcc
g++
ninja-build
libc6-dev
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment