runpod: a keep_warm pod that is disabled, unnamed or out of hours (v0.2.94)

Configuring the Digesta catalogue on the production orchestrator turned up three
faults in warm_configured_pools, all of which spend money:

  * `enabled: false` was not consulted. A catalogue entry parked for later with
    keep_warm still set rented a GPU at every boot.
  * the entry was named `alias or path`. A RunPod-only entry has neither — no
    weights live on that machine, and an alias only exists when a second name is
    wanted — so it warmed under the key "" and the pool was never the one that
    went on to serve requests. `id` is now the fallback, and an entry with no
    name at all says so instead of warming nothing quietly.
  * the schedule was ignored. ensure_ready() is the "a request is waiting" path
    and provisions unconditionally; at boot nothing is waiting, so a restart at
    02:00 on a Sunday rented exactly the pod the window exists to avoid. It now
    checks effective_min_pods first, which also means a client lease that
    outlives a restart is honoured rather than dropped.

Found on the host, not by reading: the warm pod the configuration asked for
never appeared.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.2.93"
+__version__ = "0.2.94"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/runpod_worker.py

@@ -2796,9 +2796,32 @@ def warm_configured_pools() -> int:
             block = entry.get("runpod")
             if not isinstance(block, dict) or not _as_bool(block.get("keep_warm"), False):
                 continue
-            name = entry.get("alias") or entry.get("path") or ""
+            # A disabled model must not rent a GPU. Nothing else consults
+            # `enabled` here, so a catalogue entry parked for later with
+            # keep_warm still set would have started billing at every boot.
+            if entry.get("enabled") is False:
+                continue
+            # `id` is how a RunPod-only entry names itself: it has no local
+            # `path` (there are no weights on this machine) and an `alias` only
+            # when a second name is wanted. Without it such an entry warmed
+            # under the key "" and the pool was never the one serving requests.
+            name = entry.get("alias") or entry.get("path") or entry.get("id") or ""
+            if not name:
+                print("[runpod] a keep_warm entry has no id/alias/path — skipped",
+                      flush=True)
+                continue
             try:
                 pool = get_model_pod_pool(name, entry, block)
+                # ensure_ready() provisions unconditionally — it is the "a
+                # request is waiting" path. At boot there is no request, so a
+                # scheduled model must respect its window or a restart at 02:00
+                # would rent the pod the schedule exists to avoid.
+                if pool.effective_min_pods() < 1:
+                    st = schedule_state(pool.mcfg)
+                    print(f"[runpod] not warming {name!r} yet: outside its window "
+                          f"({st.get('window')}) — a request still starts a pod",
+                          flush=True)
+                    continue
                 pool.ensure_ready()
                 warmed += 1
                 print(f"[runpod] keeping a pod warm for {name!r}", flush=True)

tests/test_runpod_client_scale.py

@@ -280,3 +280,43 @@ def test_the_endpoint_rejects_a_negative_body_before_it_reaches_the_pool():
     with pytest.raises(ValidationError):
         RunpodScaleRequest(model="qwen38-awq", pods=-1)
     assert RunpodScaleRequest(model="qwen38-awq", pods=0).pods == 0
+
+
+# ------------------------------------- warming at boot, which pays real money
+
+def test_a_disabled_model_is_never_warmed():
+    """A catalogue entry parked for later with keep_warm still set would have
+    rented a GPU at every boot. Nothing else here consults `enabled`."""
+    src = __import__("pathlib").Path(rw.__file__).read_text()
+    blk = src.split("def warm_configured_pools")[1].split("def _all_pools_hourly_rate")[0]
+    assert 'entry.get("enabled") is False' in blk
+
+
+def test_a_runpod_only_entry_is_found_by_its_id():
+    """Such an entry has no local `path` — there are no weights on this machine
+    — and an `alias` only when a second name is wanted. Keyed on alias-or-path
+    alone it warmed under "" and the pool was never the one serving requests."""
+    src = __import__("pathlib").Path(rw.__file__).read_text()
+    blk = src.split("def warm_configured_pools")[1].split("def _all_pools_hourly_rate")[0]
+    assert 'entry.get("id")' in blk
+    assert 'print("[runpod] a keep_warm entry has no id/alias/path' in blk
+
+
+def test_boot_warming_respects_the_schedule():
+    """ensure_ready() is the "a request is waiting" path and provisions
+    unconditionally. At boot nothing is waiting, so a restart at 02:00 must not
+    rent the pod the schedule exists to avoid."""
+    src = __import__("pathlib").Path(rw.__file__).read_text()
+    blk = src.split("def warm_configured_pools")[1].split("def _all_pools_hourly_rate")[0]
+    i_guard = blk.find("effective_min_pods() < 1")
+    i_ready = blk.find("pool.ensure_ready()")
+    assert i_guard != -1 and i_ready != -1 and i_guard < i_ready
+
+
+def test_a_client_lease_can_warm_at_boot_even_out_of_hours():
+    """The guard reads effective_min_pods, which already maxes the lease in, so
+    a lease that outlives a restart is honoured rather than dropped."""
+    p = _pool(min_pods=1, max_pods=3, schedule_enabled=True,
+              schedule_start="08:00", schedule_end="20:00")
+    p.set_client_floor(2)
+    assert p.effective_min_pods() >= 2