-
Stefy Lanza (nextime / spora ) authored
Per model, like every other pod setting, and bounded so that it can only ever NARROW what the operator configured. A pool with a preference and allow_other_gpus: true takes another card when the preferred one is not free -- usually right, occasionally exactly wrong. Observed 2026-10-13: gpu_types [NVIDIA A40] fell back to an L40S at $1.09/hr against the A40's $0.59, correct per the configuration and useless to a caller measuring throughput, since a number taken on a different card is not comparable with the one it is being compared to. The only way to get the A40 was to edit the model's configuration -- changing it for everybody, permanently, for one request. POST /v1/runpod/scale now takes `gpu_types` (list or comma string, in preference order) and `allow_other_gpus`. The rule is one-directional: - a card the operator did not list is DROPPED and named in `notes`; - narrowing to only cards the operator forbids is a 400 that names what IS allowed -- never a silent fall back to the default, which would be the opposite of what the caller asked for; - allow_other_gpus may be turned OFF by a request, never ON; - max_hourly_usd, min_vram_gb, cloud_types and max_pods are the operator's alone and are not reachable from a request at all. Held on the LEASE, not the pool: an "A40 only" cannot outlive the 30 minutes it asked for, releasing the lease releases its constraints, and provisioning reads the narrowing per pod so a lapsed lease stops constraining the very next one. There is deliberately no path from a request to a model's configuration. Reported back in the answer and on /v1/runpod/status as `client_gpu_narrow`, because a constraint silently dropped is worse than one refused -- the caller would read its numbers as having come from the card it asked for -- and a pod on an unexpected card should have a visible cause, not only a bill. Tests: 32 new, including that the cost ceiling and the VRAM floor still refuse a pinned card, that the operator's limits are not even mentioned in the narrowing code, and that None (not False) is the default for both fields -- False would silently turn every caller's fallback off. Also fixed a source-window test that anchored on `_rank_gpus`'s full argument list and broke the moment a parameter was added; it now anchors on the assignment, since the arguments are not what it is testing. Full suite: 25 failures, all pre-existing and identical to baseline. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
3467f293