-
Stefy Lanza (nextime / spora ) authored
The fleet image and the OCR image were coupled in three places, not one: the profile-parity test, and -- less visibly -- the OCR image's own build check, which read the FLEET's vLLM pin and failed the build on a mismatch. That one meant the OCR image could not be built on a different vLLM at all. They may diverge now. What replaces the coupling is the thing that actually failed last time: not that the images differed, but that nothing said so, and someone had to already suspect it. So each profile must PIN vllm and the torch stack with `==` (no floating version, so two builds of one image cannot differ either), each build check holds its own venv to its OWN pin -- which still catches the resolver substituting a version nobody asked for, a failure the cross-image check could never see -- and a choice the engine would otherwise make by version is named by the image that depends on it. That needed a mechanism, so profiles gain `<profile>.env`: KEY=VALUE baked into the image and exported by boot.sh, which PRINTS each variable it applies and lets the pod's own environment win. The OCR profile uses it to pin VLLM_ATTENTION_BACKEND=FLASH_ATTN -- vLLM's own default on Ampere, named rather than inherited, because FlashInfer is the one backend that image cannot use (its kernels compile on first use and it ships no CUDA toolkit) and which backend gets selected is a property of the vLLM VERSION. A future bump can now change throughput, never whether the pod starts. Also takes one install's model names out of three shipped files. The measurement they illustrate is what matters -- a registration overwritten, a backend fallen through to "cpu" -- not whose models they were. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
34612ec8