• Stefy Lanza (nextime / spora )'s avatar
    runpod: vLLM pods need ninja too, the dep after g++ · 4025664d
    Stefy Lanza (nextime / spora ) authored
    Triton and torch.inductor JIT-compile vLLM's kernels at run time and need a full
    build toolchain for the stubs they emit. Each piece was found on a live olmOCR
    pod, one layer at a time, because from outside the failure is identical every
    time: vLLM dies before registering and the pod answers 502.
    
        gcc          the C compiler
        g++          the launcher Triton builds is C++, so gcc alone is not enough
        ninja-build  inductor shells out to ninja for its extensions
    
    Only olmOCR hits this, and the reason is worth recording: it is FP8, so vLLM's
    quantisation kernels go down the Triton path. surya-2 is dense bf16 and served
    fine on images that had none of these, which is why "the image lacks a compiler"
    looked for a while like it could not be the problem.
    
    Applied to vllm.apt AND vllm-ocr.apt, which is copied from it.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    4025664d
Name
Last commit
Last update
..
audio.build-deps Loading commit data...
audio.nodeps Loading commit data...
audio.txt Loading commit data...
core-light.txt Loading commit data...
core.txt Loading commit data...
embeddings.txt Loading commit data...
engines-kt.apt Loading commit data...
engines-kt.light Loading commit data...
engines-kt.txt Loading commit data...
engines-kt.venv-sglang.check Loading commit data...
engines-kt.venv-sglang.txt Loading commit data...
engines-kt.venv-sglang.uv Loading commit data...
engines.dockerfile Loading commit data...
engines.light Loading commit data...
engines.txt Loading commit data...
faceswap.txt Loading commit data...
images.txt Loading commit data...
llama.dockerfile Loading commit data...
llama.light Loading commit data...
llama.txt Loading commit data...
ocr-paddle.light Loading commit data...
ocr-paddle.txt Loading commit data...
ocr-paddle.venv-paddleocr.txt Loading commit data...
ocr.txt Loading commit data...
ocr.venv-surya.check Loading commit data...
ocr.venv-surya.txt Loading commit data...
speaker.light Loading commit data...
speaker.txt Loading commit data...
speaker.venv-pyannote.check Loading commit data...
speaker.venv-pyannote.txt Loading commit data...
stt-crisper.light Loading commit data...
stt-crisper.txt Loading commit data...
stt-crisper.venv-transformers.txt Loading commit data...
stt-nemo.light Loading commit data...
stt-nemo.txt Loading commit data...
stt-nemo.venv-nemo.txt Loading commit data...
stt.txt Loading commit data...
text.txt Loading commit data...
tts-xtts.light Loading commit data...
tts-xtts.txt Loading commit data...
tts-xtts.venv-TTS.check Loading commit data...
tts-xtts.venv-TTS.txt Loading commit data...
tts.txt Loading commit data...
video-longcat.apt Loading commit data...
video-longcat.dockerfile Loading commit data...
video-longcat.light Loading commit data...
video-longcat.txt Loading commit data...
video.txt Loading commit data...
vllm-ocr.apt Loading commit data...
vllm-ocr.light Loading commit data...
vllm-ocr.txt Loading commit data...
vllm-ocr.venv-vllm.check Loading commit data...
vllm-ocr.venv-vllm.txt Loading commit data...
vllm-ocr.venv-vllm.uv Loading commit data...
vllm.apt Loading commit data...
vllm.light Loading commit data...
vllm.txt Loading commit data...
vllm.venv-vllm.check Loading commit data...
vllm.venv-vllm.txt Loading commit data...
vllm.venv-vllm.uv Loading commit data...
voice.build-deps Loading commit data...
voice.txt Loading commit data...