-
Stefy Lanza (nextime / spora ) authored
Triton and torch.inductor JIT-compile vLLM's kernels at run time and need a full build toolchain for the stubs they emit. Each piece was found on a live olmOCR pod, one layer at a time, because from outside the failure is identical every time: vLLM dies before registering and the pod answers 502. gcc the C compiler g++ the launcher Triton builds is C++, so gcc alone is not enough ninja-build inductor shells out to ninja for its extensions Only olmOCR hits this, and the reason is worth recording: it is FP8, so vLLM's quantisation kernels go down the Triton path. surya-2 is dense bf16 and served fine on images that had none of these, which is why "the image lacks a compiler" looked for a while like it could not be the problem. Applied to vllm.apt AND vllm-ocr.apt, which is copied from it. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
4025664d