the growth line named a trigger that had not fired (v0.3.100)

A pool with one healthy pod rented a second one second after the first became
ready, and said:

    'datalab-to/surya-ocr-2': growing to meet 1 pod(s) of demand — 1 up, 1 booting

It needs one pod. It has one pod. It is renting another. Read plainly that is a
double-rent bug, and it took a real investigation to establish that the fleet
was behaving exactly as configured.

`_should_grow` has two independent triggers:

  DEMAND       (in-flight + waiting) / scale_up_inflight_per_pod, rounded up —
               a measured number, and the one that may rent several at once.
  SATURATION   every serving pod is at its in-flight threshold, so the NEXT
               request has nowhere good to go. Anticipatory, single-flight.

What happened was saturation: 16 pages landed on the one pod the moment it went
ready, 16 at a threshold of 16 is exactly one pod's worth, so demand was already
MET and that branch did not fire at all. But the caller computed
`_pods_for_demand()` itself, after the fact, and printed it whichever branch had
answered — so the line always claimed demand, and in the saturation case
reported a number that contradicted the action it was describing.

The decision now carries its own reason. `_grow_reason()` returns the sentence
of the trigger that fired ('' for no), `_should_grow` is `bool()` of it — the
bool contract is unchanged, callers and tests compare with `is True` — and
acquire() prints what it was given instead of recomputing a figure of its own.
Saturation now says so, and says what it is NOT:

    growing — every pod is at its 16-in-flight threshold (least loaded carries
    16) — renting one more for the next burst, not for work already queued
    [1 up, 1 booting]

Nothing about when a pod is rented changes; only what the log says about why.

10 tests, including the live line as a regression: a saturation rent must not
contain the word "demand". Suite: 3142 passed, the same 13 known-stale failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.99"
+__version__ = "0.3.100"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/runpod_worker.py

@@ -3651,7 +3651,20 @@ class RunpodPodPool:
             return self._cancel_epoch != epoch
 
     def _should_grow(self, healthy: list) -> bool:
-        """True when real concurrency justifies another pod.
+        """True when real concurrency justifies another pod."""
+        return bool(self._grow_reason(healthy))
+
+    def _grow_reason(self, healthy: list) -> str:
+        """WHY another pod is justified, in the words of the trigger that fired,
+        or '' for no.
+
+        Two independent triggers, and saying which one fired is the whole point
+        of this returning a sentence. The caller used to print the DEMAND
+        number whichever trigger fired, so a pool growing because its one pod
+        was saturated announced "growing to meet 1 pod(s) of demand — 1 up" —
+        which reads as a pool renting a second card to meet demand it has
+        already met, i.e. as a bug. Observed on a live orchestrator and it cost
+        a real investigation to find that the fleet was behaving correctly.
 
         The pool used to grow only when it had NO healthy pod, so every request
         beyond the first piled onto pod #1 and max_pods never came into play.
@@ -3670,10 +3683,10 @@ class RunpodPodPool:
         """
         in_hand = len(self.pods) + self._booting
         if in_hand >= self.mcfg.max_pods:
-            return False
+            return ""
         parallel = max(1, int(getattr(self.mcfg, "scale_up_max_parallel_boots", 1) or 1))
         if self._booting >= parallel:
-            return False
+            return ""
         # A draining pod is on its way out and takes no new work, so counting it
         # as capacity would answer "no need to grow" with the one pod that cannot
         # help — and counting its emptying in-flight as the minimum would say the
@@ -3683,12 +3696,18 @@ class RunpodPodPool:
             # Nothing can serve a request. One on the way is enough to wait for;
             # a second here would be two cards for the same first request, which
             # is how a warm floor of one became two.
-            return self._booting == 0
+            if self._booting:
+                return ""
+            return "nothing can serve a request yet — renting the first pod"
         # Demand we can measure: what the pods carry plus what is blocked. This
         # is the part that may grow SEVERAL pods at once, because the number is
         # real rather than a guess about the future.
-        if in_hand < self._pods_for_demand(serving):
-            return True
+        want = self._pods_for_demand(serving)
+        if in_hand < want:
+            threshold = max(1, int(self.mcfg.scale_up_inflight_per_pod or 1))
+            load = sum(p.inflight for p in serving) + max(0, self._waiting)
+            return (f"{load} page(s) in flight or waiting at {threshold} per pod "
+                    f"needs {want} pod(s), and {in_hand} are in hand")
         # And the original, anticipatory trigger: every pod is at its in-flight
         # threshold, so the next request has nowhere good to go. Kept because
         # `max_inflight_per_pod` defaults to unlimited, so requests are often
@@ -3697,8 +3716,17 @@ class RunpodPodPool:
         # unlike the demand above this is a prediction, and a prediction should
         # not rent four cards.
         threshold = max(1, int(self.mcfg.scale_up_inflight_per_pod or 1))
-        return (self._booting == 0
-                and min(p.inflight for p in serving) >= threshold)
+        if self._booting:
+            return ""
+        least = min(p.inflight for p in serving)
+        if least >= threshold:
+            # NOT a demand figure. Demand is already met — one pod is exactly
+            # right for `threshold` in flight — and this rents ahead of the
+            # next request rather than for one that exists.
+            return (f"every pod is at its {threshold}-in-flight threshold "
+                    f"(least loaded carries {least}) — renting one more for "
+                    f"the next burst, not for work already queued")
+        return ""
 
     def _grow_in_background(self):
         """Provision another pod without making the triggering request wait for it.
@@ -3750,10 +3778,13 @@ class RunpodPodPool:
             with self._cv:
                 healthy = [p for p in self.pods if p.healthy]
                 chosen = self._pick(healthy, affinity) if healthy else None
-                if self._should_grow(healthy):
+                # Asked ONCE, and the answer carries its own reason. Deciding
+                # here and describing it separately below is what let the
+                # message name a trigger that had not fired.
+                why = self._grow_reason(healthy)
+                if why:
                     self._booting += 1
                     grow_now = True
-                    want = self._pods_for_demand([p for p in healthy if not p.draining])
                 if chosen is not None:
                     chosen.inflight += 1
                     chosen.last_used = time.time()
@@ -3761,9 +3792,9 @@ class RunpodPodPool:
                     url = chosen.url
             if chosen is not None:
                 if grow_now:
-                    print(f"[runpod] {self.model_key!r}: growing to meet "
-                          f"{want} pod(s) of demand — {len(self.pods)} up, "
-                          f"{self._booting} booting", flush=True)
+                    print(f"[runpod] {self.model_key!r}: growing — {why} "
+                          f"[{len(self.pods)} up, {self._booting} booting]",
+                          flush=True)
                     self._grow_in_background()      # absorbs the NEXT request
                 return chosen, url
 

tests/test_runpod_grow_reason.py

+"""The growth line must name the trigger that actually fired.
+
+Observed 2026-10-15. A pool with one pod rented a second one second after the
+first went ready, and announced:
+
+    'datalab-to/surya-ocr-2': growing to meet 1 pod(s) of demand — 1 up, 1 booting
+
+Read plainly that says: it needs one pod, it has one pod, and it is renting
+another. It looks exactly like a double-rent bug, and it cost a real
+investigation to establish that the fleet was correct — the DEMAND trigger had
+not fired at all (one pod is exactly right for 16 in flight at 16 per pod); the
+SATURATION trigger had, which rents ahead of the next burst. The caller printed
+the demand figure whichever trigger fired.
+
+Two triggers that mean different things must not share one sentence.
+"""
+
+import types
+
+import pytest
+
+from codai.api import runpod_worker as rw
+
+
+def _pool(max_pods=8, threshold=16, parallel=1):
+    from codai.api.runpod_worker import RunpodPodPool, parse_model_runpod
+
+    class _Acct:
+        enabled = True
+
+    return RunpodPodPool("m", _Acct(), parse_model_runpod({
+        "max_pods": max_pods,
+        "scale_up_inflight_per_pod": threshold,
+        "scale_up_max_parallel_boots": parallel,
+    }), "m")
+
+
+def _pod(inflight=0, healthy=True, draining=False):
+    return types.SimpleNamespace(inflight=inflight, healthy=healthy,
+                                 draining=draining, pod_id="p", url="u",
+                                 last_used=0.0)
+
+
+# ------------------------------------------------------------- saturation #
+def test_saturation_does_not_claim_to_be_meeting_demand():
+    """The exact live line. Demand is MET; this is anticipatory."""
+    p = _pool(threshold=16)
+    busy = _pod(inflight=16)
+    p.pods = [busy]
+    why = p._grow_reason([busy])
+    assert why
+    assert "demand" not in why.lower(), why
+    assert "16-in-flight threshold" in why
+    assert "next burst" in why
+
+
+def test_saturation_says_it_is_not_for_queued_work():
+    """The distinction that was missing: nothing is waiting for this pod."""
+    p = _pool(threshold=16)
+    busy = _pod(inflight=16)
+    p.pods = [busy]
+    assert "not for work already queued" in p._grow_reason([busy])
+
+
+# ----------------------------------------------------------------- demand #
+def test_demand_reports_the_load_and_the_shortfall():
+    p = _pool(threshold=16)
+    busy = [_pod(inflight=16), _pod(inflight=16)]
+    p.pods = list(busy)
+    p._waiting = 20
+    why = p._grow_reason(busy)
+    assert "52 page(s) in flight or waiting" in why   # 16 + 16 + 20
+    assert "needs 4 pod(s)" in why
+    assert "2 are in hand" in why
+
+
+def test_the_first_pod_says_nothing_can_serve_yet():
+    p = _pool()
+    assert "nothing can serve a request yet" in p._grow_reason([])
+
+
+# -------------------------------------------------------------- refusals #
+def test_no_reason_is_an_empty_string_not_a_sentence():
+    """`_should_grow` is `bool(reason)`, so a chatty 'no' would rent a pod."""
+    p = _pool(threshold=16)
+    quiet = _pod(inflight=1)
+    p.pods = [quiet]
+    assert p._grow_reason([quiet]) == ""
+
+
+@pytest.mark.parametrize("setup,label", [
+    (lambda p: setattr(p.mcfg, "max_pods", 1), "at max_pods"),
+    (lambda p: setattr(p, "_booting", 1), "already booting"),
+])
+def test_the_guards_still_refuse(setup, label):
+    p = _pool(threshold=16)
+    busy = _pod(inflight=99)
+    p.pods = [busy]
+    setup(p)
+    assert p._grow_reason([busy]) == "", label
+
+
+def test_a_booting_pod_blocks_the_anticipatory_rent():
+    """Single-flight: a prediction must not rent a second card while the first
+    prediction is still being filled."""
+    p = _pool(threshold=16, parallel=4)
+    busy = _pod(inflight=16)
+    p.pods = [busy]
+    p._booting = 1
+    assert p._grow_reason([busy]) == ""
+
+
+# -------------------------------------------- the bool contract is unchanged #
+def test_should_grow_still_returns_a_real_bool():
+    """Callers and tests compare with `is True` / `is False`."""
+    p = _pool(threshold=16)
+    busy = _pod(inflight=16)
+    p.pods = [busy]
+    assert p._should_grow([busy]) is True
+    quiet = _pod(inflight=0)
+    p.pods = [quiet]
+    assert p._should_grow([quiet]) is False
+
+
+def test_the_caller_prints_the_reason_it_was_given():
+    """It must not recompute a figure of its own — that was the bug."""
+    import pathlib
+    src = pathlib.Path(rw.__file__).read_text()
+    blk = src.split("def acquire")[1].split("def release")[0]
+    assert "growing — {why}" in blk
+    assert "pod(s) of demand" not in blk