• Stefy Lanza (nextime / spora )'s avatar
    runpod: a request may narrow which cards a lease accepts (v0.3.54) · 3467f293
    Stefy Lanza (nextime / spora ) authored
    Per model, like every other pod setting, and bounded so that it can only ever
    NARROW what the operator configured.
    
    A pool with a preference and allow_other_gpus: true takes another card when the
    preferred one is not free -- usually right, occasionally exactly wrong. Observed
    2026-10-13: gpu_types [NVIDIA A40] fell back to an L40S at $1.09/hr against the
    A40's $0.59, correct per the configuration and useless to a caller measuring
    throughput, since a number taken on a different card is not comparable with the
    one it is being compared to. The only way to get the A40 was to edit the model's
    configuration -- changing it for everybody, permanently, for one request.
    
    POST /v1/runpod/scale now takes `gpu_types` (list or comma string, in preference
    order) and `allow_other_gpus`. The rule is one-directional:
    
      - a card the operator did not list is DROPPED and named in `notes`;
      - narrowing to only cards the operator forbids is a 400 that names what IS
        allowed -- never a silent fall back to the default, which would be the
        opposite of what the caller asked for;
      - allow_other_gpus may be turned OFF by a request, never ON;
      - max_hourly_usd, min_vram_gb, cloud_types and max_pods are the operator's
        alone and are not reachable from a request at all.
    
    Held on the LEASE, not the pool: an "A40 only" cannot outlive the 30 minutes it
    asked for, releasing the lease releases its constraints, and provisioning reads
    the narrowing per pod so a lapsed lease stops constraining the very next one.
    There is deliberately no path from a request to a model's configuration.
    
    Reported back in the answer and on /v1/runpod/status as `client_gpu_narrow`,
    because a constraint silently dropped is worse than one refused -- the caller
    would read its numbers as having come from the card it asked for -- and a pod on
    an unexpected card should have a visible cause, not only a bill.
    
    Tests: 32 new, including that the cost ceiling and the VRAM floor still refuse a
    pinned card, that the operator's limits are not even mentioned in the narrowing
    code, and that None (not False) is the default for both fields -- False would
    silently turn every caller's fallback off.
    
    Also fixed a source-window test that anchored on `_rank_gpus`'s full argument
    list and broke the moment a parameter was added; it now anchors on the
    assignment, since the arguments are not what it is testing.
    
    Full suite: 25 failures, all pre-existing and identical to baseline.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    3467f293
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...