• Stefy Lanza (nextime / spora )'s avatar
    runpod: a request may narrow which cards a lease accepts (v0.3.54) · 3467f293
    Stefy Lanza (nextime / spora ) authored
    Per model, like every other pod setting, and bounded so that it can only ever
    NARROW what the operator configured.
    
    A pool with a preference and allow_other_gpus: true takes another card when the
    preferred one is not free -- usually right, occasionally exactly wrong. Observed
    2026-10-13: gpu_types [NVIDIA A40] fell back to an L40S at $1.09/hr against the
    A40's $0.59, correct per the configuration and useless to a caller measuring
    throughput, since a number taken on a different card is not comparable with the
    one it is being compared to. The only way to get the A40 was to edit the model's
    configuration -- changing it for everybody, permanently, for one request.
    
    POST /v1/runpod/scale now takes `gpu_types` (list or comma string, in preference
    order) and `allow_other_gpus`. The rule is one-directional:
    
      - a card the operator did not list is DROPPED and named in `notes`;
      - narrowing to only cards the operator forbids is a 400 that names what IS
        allowed -- never a silent fall back to the default, which would be the
        opposite of what the caller asked for;
      - allow_other_gpus may be turned OFF by a request, never ON;
      - max_hourly_usd, min_vram_gb, cloud_types and max_pods are the operator's
        alone and are not reachable from a request at all.
    
    Held on the LEASE, not the pool: an "A40 only" cannot outlive the 30 minutes it
    asked for, releasing the lease releases its constraints, and provisioning reads
    the narrowing per pod so a lapsed lease stops constraining the very next one.
    There is deliberately no path from a request to a model's configuration.
    
    Reported back in the answer and on /v1/runpod/status as `client_gpu_narrow`,
    because a constraint silently dropped is worse than one refused -- the caller
    would read its numbers as having come from the card it asked for -- and a pod on
    an unexpected card should have a visible cause, not only a bill.
    
    Tests: 32 new, including that the cost ceiling and the VRAM floor still refuse a
    pinned card, that the operator's limits are not even mentioned in the narrowing
    code, and that None (not False) is the default for both fields -- False would
    silently turn every caller's fallback off.
    
    Also fixed a source-window test that anchored on `_rank_gpus`'s full argument
    list and broke the moment a parameter was added; it now anchors on the
    assignment, since the arguments are not what it is testing.
    
    Full suite: 25 failures, all pre-existing and identical to baseline.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    3467f293
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...