a request could ask for a warm floor but not for a ceiling (v0.4.33)

/v1/runpod/scale set a FLOOR. "Give me 8 pods" meant "hold at least 8
ready", and the autoscaler stayed free to grow the pool to the configured
max_pods for as long as work kept arriving. A client running a staged
batch -- 8 pods for the first stage, 32 for the next -- had no way to
bound a stage at all, and nothing in the reply said so.

Observed 2026-10-18: a run that asked for 8 pods was at 13 and climbing
toward 128. Every pod past the plan is ~700s of paid boot producing
nothing, which is why the realised cost per page came out 16.5x the
measured baseline while every component behaved exactly as written.

A lease may now also carry `max_pods`, under the same rule `gpu_types`
already follows there: it may only ever NARROW the operator's ceiling, a
larger number is clamped and reported in `notes`, and it dies with the
lease. `pods: 0` with a `max_pods` is a cap with no floor -- what a
client with a long tail of work wants; `pods: 0` alone releases both.

The ceiling has to be read by EVERY path that decides whether to rent,
or it is a race between the grower that honours it and the one that does
not: _pods_for_demand, _grow_reason, _pods_needed and the warm-floor
top-up all now read effective_max_pods(). A leased floor is clamped to
it too, since a floor above a ceiling is a fleet that oscillates for as
long as the lease lasts. A configured max_pods of 0 still means zero --
that is the operator having turned renting off, not a lease.

22 tests, including a guard that no growth path reads mcfg.max_pods
directly, so a path added later cannot silently escape every cap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.32"
+__version__ = "0.4.33"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/app.py

@@ -962,6 +962,15 @@ class RunpodScaleRequest(BaseModel):
     pods: int = Field(..., ge=0,
                       description="Warm pods to hold, clamped to the model's max_pods. "
                                   "0 releases the lease.")
+    max_pods: Optional[int] = Field(
+        None, ge=0,
+        description="Cap the fleet at this many pods for the life of this "
+                    "lease. May only NARROW the model's configured max_pods: a "
+                    "larger number is clamped to it and reported back in "
+                    "`notes`. `pods` is a FLOOR and says nothing about how "
+                    "large the pool may grow, so a staged batch that wants "
+                    "8 pods for its first stage sets both. 0 drops this "
+                    "lease's own cap; omit to leave it as it is.")
     teardown: bool = Field(False,
                            description="Give the surplus pods back NOW rather than "
                                        "waiting out scale_down_after_s. Never goes "
@@ -1034,8 +1043,19 @@ async def runpod_scale(body: RunpodScaleRequest, request: Request):
 
     Deliberately NOT admin-scoped: the gate is per model
     (``allow_client_scale``), the number is clamped to ``max_pods``, and the
-    lease expires (``client_scale_ttl_s``, default 30 min) so a client that dies
-    holding four A40s stops paying for them by itself. The account's $/hr and
+    ``max_pods`` is the other half of that, and the half that bounds what a
+    batch may cost. ``pods`` is a FLOOR: it says what to hold ready, and the
+    autoscaler is still free to grow the pool to the configured ``max_pods``
+    for as long as work keeps arriving. A client running a staged batch — 8
+    pods for the first stage, 32 for the next — needs to say how large the
+    fleet may BECOME, which it does by sending both. Like ``gpu_types`` it may
+    only ever narrow the operator's own number and it dies with the lease.
+    ``pods: 0`` with a ``max_pods`` keeps the cap and holds nothing warm, which
+    is what a client with a long tail of work wants; ``pods: 0`` alone releases
+    the whole lease.
+
+    The lease expires (``client_scale_ttl_s``, default 30 min) so a client that
+    dies holding four A40s stops paying for them by itself. The account's $/hr and
     spend caps still refuse to provision regardless of what is asked here.
     """
     from codai.api import runpod_worker
@@ -1051,7 +1071,8 @@ async def runpod_scale(body: RunpodScaleRequest, request: Request):
     task = asyncio.create_task(asyncio.to_thread(
         runpod_worker.set_client_pods,
         body.model, body.pods, teardown=bool(body.teardown),
-        gpu_types=body.gpu_types, allow_other_gpus=body.allow_other_gpus))
+        gpu_types=body.gpu_types, allow_other_gpus=body.allow_other_gpus,
+        max_pods=body.max_pods))
 
     def _done(t):
         try:

tests/test_runpod_client_scale.py

@@ -377,7 +377,9 @@ def test_the_demand_path_already_had_the_guard_and_keeps_it():
     # meant the fleet could grow by one pod per boot however much was queued.
     # The guard is still here, it just reads how many rather than whether.
     assert "self._booting >= parallel" in blk
-    assert "in_hand >= self.mcfg.max_pods" in blk
+    # Reads the EFFECTIVE ceiling since v0.4.33: the configured max_pods
+    # narrowed by any client lease. Same guard, one indirection on.
+    assert "in_hand >= self.effective_max_pods()" in blk
 
 
 def test_the_answer_names_the_model_the_client_asked_about():

tests/test_runpod_schedule_region_spend.py

@@ -136,6 +136,14 @@ def test_effective_min_pods_drops_to_zero_outside_the_window(monkeypatch):
         _client_floor = 0
         _client_floor_until = 0.0
         _client_floor_logged = -1
+        # The floor is also clamped by the CEILING in force, which a lease may
+        # narrow (/v1/runpod/scale max_pods). No lease here either: this is the
+        # schedule.
+        effective_max_pods = rw.RunpodPodPool.effective_max_pods
+        client_cap = rw.RunpodPodPool.client_cap
+        _client_cap = 0
+        _client_cap_until = 0.0
+        _client_cap_logged = -1
 
     pool = _Pool()
     monkeypatch.setattr(rw, "schedule_state", lambda cfg, now=None: {"in_window": True})

tests/test_runpod_warm_floor_respects_max_pods.py

@@ -105,7 +105,7 @@ def test_worker_uses_two_distinct_counts():
     src = open("codai/api/runpod_worker.py").read()
     assert "billing_in_hand = len(self.pods) + self._booting" in src
     assert "healthy_in_hand = len([p for p in self.pods if p.healthy]) + self._booting" in src
-    assert "max(0, int(self.mcfg.max_pods or 0) - billing_in_hand)" in src, \
+    assert "max(0, self.effective_max_pods() - billing_in_hand)" in src, \
         "the max_pods cap must be tested against pods that bill"
     assert "want = min(keep_min - healthy_in_hand," in src, \
         "the deficit must still be tested against pods that can serve"