• Stefy Lanza (nextime / spora )'s avatar
    let the two vLLM images evolve apart, visibly · 34612ec8
    Stefy Lanza (nextime / spora ) authored
    The fleet image and the OCR image were coupled in three places, not one: the
    profile-parity test, and -- less visibly -- the OCR image's own build check,
    which read the FLEET's vLLM pin and failed the build on a mismatch. That one
    meant the OCR image could not be built on a different vLLM at all.
    
    They may diverge now. What replaces the coupling is the thing that actually
    failed last time: not that the images differed, but that nothing said so, and
    someone had to already suspect it. So each profile must PIN vllm and the torch
    stack with `==` (no floating version, so two builds of one image cannot differ
    either), each build check holds its own venv to its OWN pin -- which still
    catches the resolver substituting a version nobody asked for, a failure the
    cross-image check could never see -- and a choice the engine would otherwise
    make by version is named by the image that depends on it.
    
    That needed a mechanism, so profiles gain `<profile>.env`: KEY=VALUE baked into
    the image and exported by boot.sh, which PRINTS each variable it applies and
    lets the pod's own environment win. The OCR profile uses it to pin
    VLLM_ATTENTION_BACKEND=FLASH_ATTN -- vLLM's own default on Ampere, named rather
    than inherited, because FlashInfer is the one backend that image cannot use
    (its kernels compile on first use and it ships no CUDA toolkit) and which
    backend gets selected is a property of the vLLM VERSION. A future bump can now
    change throughput, never whether the pod starts.
    
    Also takes one install's model names out of three shipped files. The
    measurement they illustrate is what matters -- a registration overwritten, a
    backend fallen through to "cpu" -- not whose models they were.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    34612ec8
test_teach_carries_the_engine.py 3.69 KB