runpod: a request may narrow which cards a lease accepts (v0.3.54)

Per model, like every other pod setting, and bounded so that it can only ever
NARROW what the operator configured.

A pool with a preference and allow_other_gpus: true takes another card when the
preferred one is not free -- usually right, occasionally exactly wrong. Observed
2026-10-13: gpu_types [NVIDIA A40] fell back to an L40S at $1.09/hr against the
A40's $0.59, correct per the configuration and useless to a caller measuring
throughput, since a number taken on a different card is not comparable with the
one it is being compared to. The only way to get the A40 was to edit the model's
configuration -- changing it for everybody, permanently, for one request.

POST /v1/runpod/scale now takes `gpu_types` (list or comma string, in preference
order) and `allow_other_gpus`. The rule is one-directional:

  - a card the operator did not list is DROPPED and named in `notes`;
  - narrowing to only cards the operator forbids is a 400 that names what IS
    allowed -- never a silent fall back to the default, which would be the
    opposite of what the caller asked for;
  - allow_other_gpus may be turned OFF by a request, never ON;
  - max_hourly_usd, min_vram_gb, cloud_types and max_pods are the operator's
    alone and are not reachable from a request at all.

Held on the LEASE, not the pool: an "A40 only" cannot outlive the 30 minutes it
asked for, releasing the lease releases its constraints, and provisioning reads
the narrowing per pod so a lapsed lease stops constraining the very next one.
There is deliberately no path from a request to a model's configuration.

Reported back in the answer and on /v1/runpod/status as `client_gpu_narrow`,
because a constraint silently dropped is worse than one refused -- the caller
would read its numbers as having come from the card it asked for -- and a pod on
an unexpected card should have a visible cause, not only a bill.

Tests: 32 new, including that the cost ceiling and the VRAM floor still refuse a
pinned card, that the operator's limits are not even mentioned in the narrowing
code, and that None (not False) is the default for both fields -- False would
silently turn every caller's fallback off.

Also fixed a source-window test that anchored on `_rank_gpus`'s full argument
list and broke the moment a parameter was added; it now anchors on the
assignment, since the arguments are not what it is testing.

Full suite: 25 failures, all pre-existing and identical to baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.53"
+__version__ = "0.3.54"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/app.py

@@ -23,7 +23,7 @@ import logging
 import os
 import time
 from contextlib import asynccontextmanager
-from typing import List
+from typing import List, Optional, Union
 
 from fastapi import FastAPI, HTTPException, Request
 from fastapi.responses import FileResponse, JSONResponse
@@ -902,6 +902,21 @@ class RunpodScaleRequest(BaseModel):
                                        "waiting out scale_down_after_s. Never goes "
                                        "below the model's own min_pods, and a pod "
                                        "with requests on it drains first.")
+    gpu_types: Optional[Union[str, List[str]]] = Field(
+        None,
+        description="Cards THIS lease will accept, in preference order (a list, "
+                    "or a comma-separated string). May only NARROW the model's "
+                    "configured gpu_types: a card the operator did not list is "
+                    "ignored and reported back in `notes`, and narrowing to "
+                    "nothing allowed is a 400. Omit to use the model's own "
+                    "preference.")
+    allow_other_gpus: Optional[bool] = Field(
+        None,
+        description="false makes this lease wait for one of its gpu_types "
+                    "instead of taking whatever else fits. May only be turned "
+                    "OFF by a request: asking for true where the operator set "
+                    "false is ignored and reported. Omit to use the model's own "
+                    "setting.")
     model_config = ConfigDict(extra="allow")
 
 
@@ -922,6 +937,14 @@ async def runpod_scale(body: RunpodScaleRequest, request: Request):
     still cannot scale through the operator's own ``min_pods``, and a pod with
     requests on it drains rather than dying under them.
 
+    ``gpu_types`` and ``allow_other_gpus`` let a lease say which cards it will
+    accept. A caller measuring throughput knows something the operator cannot --
+    that a number taken on a different card is not comparable with yesterday's,
+    and that waiting for the right one beats paying a fallback's price for a
+    result it cannot use. It may only ever NARROW the model's own choice, never
+    widen it, and the narrowing dies with the lease: there is deliberately no way
+    for a request to change a model's configuration.
+
     Deliberately NOT admin-scoped: the gate is per model
     (``allow_client_scale``), the number is clamped to ``max_pods``, and the
     lease expires (``client_scale_ttl_s``, default 30 min) so a client that dies
@@ -930,10 +953,15 @@ async def runpod_scale(body: RunpodScaleRequest, request: Request):
     """
     from codai.api import runpod_worker
     try:
-        return runpod_worker.set_client_pods(body.model, body.pods,
-                                            teardown=bool(body.teardown))
+        return runpod_worker.set_client_pods(
+            body.model, body.pods, teardown=bool(body.teardown),
+            gpu_types=body.gpu_types, allow_other_gpus=body.allow_other_gpus)
     except LookupError as exc:
         raise HTTPException(status_code=404, detail=str(exc))
+    except ValueError as exc:
+        # A narrowing that leaves nothing allowed: the caller's mistake, and it
+        # must not quietly fall back to the operator's default.
+        raise HTTPException(status_code=400, detail=str(exc))
     except PermissionError as exc:
         raise HTTPException(status_code=403, detail=str(exc))
     except Exception as exc:

docs/orchestrator-runpod-guide.md

@@ -774,6 +774,57 @@ curl -fsS -X POST http://127.0.0.1:8776/v1/runpod/scale \
  "max_pods": 3, "expires_in_s": 1800, "clamped": false}
 ```
 
+#### Choosing the card for one lease
+
+A pool configured with a preference and `allow_other_gpus: true` will take
+another card when the preferred one is not free — which is usually what you
+want, and occasionally exactly what you do not. A caller measuring throughput
+knows something the operator cannot: that a number taken on a different card is
+not comparable with yesterday's, and that waiting for the right card beats
+paying a fallback's price for a result it cannot use.
+
+So a lease may say which cards it accepts:
+
+```bash
+curl -fsS -X POST http://127.0.0.1:8776/v1/runpod/scale \
+     -H "Authorization: Bearer sk-coderai-…" -H "Content-Type: application/json" \
+     -d '{"model": "qwen38-awq", "pods": 1,
+          "gpu_types": ["NVIDIA A40"], "allow_other_gpus": false}'
+```
+
+```json
+{"model": "qwen38-awq", "requested": 1, "granted": 1, "max_pods": 3,
+ "expires_in_s": 1800, "clamped": false,
+ "gpu_types": ["NVIDIA A40"], "allow_other_gpus": false}
+```
+
+The rule is one-directional: **a request may narrow the operator's choice, never
+widen it.**
+
+| asked | result |
+|---|---|
+| a card in the model's `gpu_types` | honoured, in the order the caller listed |
+| a card the model does **not** list | dropped, and named in `notes` |
+| only cards the model forbids | **400**, naming what is allowed — never a silent fall back to the default |
+| `allow_other_gpus: false` | honoured: this lease waits rather than falling back |
+| `allow_other_gpus: true` where the model says false | ignored, and said so in `notes` |
+| `max_hourly_usd`, `min_vram_gb`, `cloud_types`, `max_pods` | not reachable from a request at all |
+
+Two properties worth knowing:
+
+- **It dies with the lease.** The narrowing is held on the lease, not the pool,
+  so an "A40 only" cannot outlive the thirty minutes it asked for, and the pool
+  returns to your preference by itself. There is deliberately **no way for a
+  request to change a model's configuration** — that stays yours.
+- **A dropped constraint is reported, never silent.** Otherwise a caller reads
+  its numbers as having come from the card it asked for. `/v1/runpod/status`
+  shows any narrowing in force as `client_gpu_narrow`, so a pod on an unexpected
+  card has a visible cause and not only a bill.
+
+It is per model, like every other pod setting: the gate is the same
+`allow_client_scale`, and the bounds are that model's own `gpu_types`,
+`allow_other_gpus` and cost ceiling.
+
 It is a **lease**, bounded three ways, because a client that can raise the floor
 can raise the bill:
 

tests/test_runpod_placement.py

@@ -160,7 +160,10 @@ def test_candidates_are_expanded_across_regions_for_the_capacity_fallback():
     """The existing retry walks `ranked` on a capacity miss; pairing each card
     with each region makes it walk regions too, with no new control flow."""
     src = (ROOT / "codai/api/runpod_worker.py").read_text()
-    blk = src.split("ranked = _rank_gpus(client, self.mcfg, self.account)")[1][:700]
+    # Anchored on the assignment alone: the call's ARGUMENTS are not what this
+    # test is about, and spelling them out made it fail the day a parameter was
+    # added and wrapped the line.
+    blk = src.split("ranked = _rank_gpus(")[1][:900]
     assert 'dict(sel, _dc=dc) for sel in ranked for dc in dcs' in blk
     assert "if len(dcs) > 1:" in blk