# The version the local vLLM node runs (requirements-vllm.txt) — but NOT the
# PyPI wheel: that one is built against CUDA 13 (torch 2.13+cu130), and a pod
# whose host driver is CUDA 12.8 dies at engine start with "The NVIDIA driver
# on your system is too old (found version 12080)". vLLM publishes a +cu129
# variant on its own index, with torch's cu129 wheels; CUDA 12.x minor-version
# compatibility runs a 12.9 runtime on a 12.8 driver. Resolved with uv (the
# .uv marker): pip cannot cope with the pin set. The venv is ~10.5 GB (torch's
# CUDA libraries, of which only cusolver is unused at import) and the image
# ~12 GB; if RunPod refuses it, run this model with venv_on_volume instead.
--extra-index-url https://wheels.vllm.ai/0.27.1/cu129
--extra-index-url https://download.pytorch.org/whl/cu129
vllm==0.27.1+cu129
torch==2.13.0+cu129
torchvision==0.28.0+cu129
torchaudio==2.11.0+cu129
# torchcodec is a vLLM requirement (>=0.14) and PyPI's newest, 0.17.0, is built
# against CUDA 13: it dlopens libnvrtc.so.13 while this venv has nvrtc 12.9, so
# `import vllm.entrypoints.openai.api_server` dies even though `import vllm`
# works. Pinned to the newest +cu129 build on torch's index. The same CUDA
# 13-vs-12 trap as the torch pins above, one layer further down.
torchcodec==0.16.0+cu129

# FlashInfer's PREBUILT kernels, so nothing is compiled on a rented pod.
#
# FP8 KV cache (--kv-cache-dtype fp8) routes attention through FlashInfer on
# Ampere, and FlashInfer JIT-compiles its prefill kernel on first use: it calls
# nvcc, which no image here ships (the CUDA runtime is present, the toolkit is
# not, and PyPI's nvidia-cuda-nvcc-cu12 wheel contains only ptxas). Measured on
# a pod 2026-10-22: the engine accepted the dtype, resolved it to
# torch.float8_e4m3fn, then died in flashinfer/jit/cpp_ext.py with
# "Could not find nvcc and default cuda_home='/usr/local/cuda' doesn't exist".
#
# flashinfer-jit-cache is the upstream package of precompiled modules for this
# exact flashinfer + CUDA pair: 959 of them, 320 with an e4m3 KV cache, head
# dims 64/128/256. `jit/env.py` prefers it over compiling, so the engine finds
# the module and never looks for a compiler. e5m2 is NOT in it, which is why
# `fp8` (= e4m3fn) is the dtype to ask for.
#
# It costs ~2.9 GiB in the image, and a pod pulls the image on every cold
# start. That is the trade for doubling the cache in tokens: FP16 KV on an A40
# gave 212,321 tokens for a 27B hybrid (64 KiB/token), so a 128k window fitted
# 1.62 times; FP8 should fit it about 3.2 times.
--extra-index-url https://flashinfer.ai/whl/cu129
flashinfer-jit-cache==0.6.16.post3+cu129
