the gate warning compared requests against pages (v0.4.26)

The diagnostic added in 0.4.25 tested `gate * 4 < ocr.max_concurrency`. Those
are different units -- a request against a page -- and for the 4096-page budget
this install runs it demanded a gate of 1024: memory-unsafe, and four times
more than the derivation in the same function will ever choose. It therefore
fired against the 256 that derivation had just picked, telling the operator to
raise a value coderai itself had chosen as correct:

  [queue] max_parallel_requests=256 max_size=2048
  [queue] WARNING: ... admits only 256 request(s) ... Raise
          server.max_parallel_requests ...

The floor and the cap are now named once, as _GATE_FLOOR and _GATE_CAP, and
both the derivation and the warning read them, so the two rules cannot
disagree again -- a separate literal in each place is how they came to.

The warning now tests the gate's own size rather than its ratio to a page
budget, which is also the only thing worth saying: a handful of admitted
requests cannot feed a pod pool whatever the page budget is. Since an unwritten
key is already derived above the floor, it now reaches only an explicitly small
value, and says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.25"
+__version__ = "0.4.26"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/main.py

@@ -1424,6 +1424,15 @@ def main():
     print(f"[queue] max_parallel_requests={queue_manager.max_parallel_requests} "
           f"max_size={queue_manager.max_size}", flush=True)
 
+    # Floor and cap for the OCR admission gate, named once so the derivation
+    # below and the warning after it cannot disagree -- the first version of
+    # this pair did, and warned about a value it had just chosen.
+    #
+    # The floor is what a pod pool needs to be fed at all. The cap is memory:
+    # an admitted request holds its whole upload plus the per-page PDFs split
+    # from it, so 256 slots x a 32-file call of p95-sized scans is ~59 GB.
+    _GATE_FLOOR, _GATE_CAP = 16, 256
+
     # An OCR install must not have to know this setting exists.
     #
     # The gate of 2 in flight / 6 queued is right for one local card answering
@@ -1456,7 +1465,7 @@ def main():
                 and "max_parallel_requests" not in _explicit and not _mp):
             # ~8 pages per document is what real corpora run at; a request
             # usually carries one document.
-            _derived = max(16, min(256, _page_budget // 8))
+            _derived = max(_GATE_FLOOR, min(_GATE_CAP, _page_budget // 8))
             if _derived > queue_manager.max_parallel_requests:
                 queue_manager.max_parallel_requests = _derived
                 config.server.max_parallel_requests = _derived
@@ -1475,40 +1484,33 @@ def main():
     except Exception:
         pass            # a derived default must never stop a server booting
 
-    # Say so when this gate is the ceiling, not the GPU.
+    # Say so when an EXPLICIT gate is small enough to be the ceiling.
     #
-    # The admission gate defaults to 2 in flight and 6 queued, which is right
-    # for one local card answering chat. It is catastrophic for an OCR install
-    # that dispatches pages to rented pods: the pool is sized in pages
-    # (ocr.max_concurrency, max_inflight_per_pod) and the pod is handed its own
-    # much larger limit through env, while the front in front of all of it keeps
-    # admitting two requests. A fleet then idles behind a two-lane gate and
-    # every diagnostic points at the pods.
+    # Only reachable when the operator wrote the key, because an unwritten one
+    # was already derived above. The test is the gate's absolute size, not its
+    # ratio to ocr.max_concurrency: those are different units -- a request
+    # against a page -- and comparing them demanded a gate of 1024 for a
+    # 4096-page budget, which is memory-unsafe and more than the derivation
+    # itself will ever choose. A first version of this check did exactly that
+    # and fired against the 256 this same file had just derived.
     #
-    # This was observed costing roughly an order of magnitude: one A40 ran at
-    # `Running: 1` with `Waiting: 0` -- the pod asking for work and not getting
-    # it -- while the orchestrator reported pages in flight. The bare line above
-    # was printed at every boot throughout and read past, because a number
-    # without a consequence attached does not look like a problem. Hence a
-    # diagnostic that names what to change.
+    # What is worth saying is that a handful of admitted requests cannot feed a
+    # pod pool, whatever the page budget says: one A40 at a gate of 2 sat at
+    # `Running: 1, Waiting: 0` and read 219 pages in 1210s, the same as reading
+    # them one at a time.
     try:
         _ocr = getattr(config, "ocr", None)
         _ocr_on = bool(getattr(_ocr, "enabled", False))
-        _pages = int(getattr(_ocr, "max_concurrency", 0) or 0)
         _gate = int(queue_manager.max_parallel_requests or 0)
-        # A request carries at least one page, so a gate below the page budget
-        # the operator asked for cannot deliver it. Four is slack for the common
-        # case where one request carries several documents.
-        if _ocr_on and _pages > 0 and _gate * 4 < _pages:
-            print(f"[queue] WARNING: OCR is enabled with ocr.max_concurrency="
-                  f"{_pages} pages, but this server admits only {_gate} request(s) "
-                  f"at a time (server.max_parallel_requests) with "
-                  f"{queue_manager.max_size} queued. The admission gate, not the "
-                  f"GPU or the pod pool, will be the throughput ceiling: pods sit "
-                  f"at 'Running: 1, Waiting: 0' while work queues here. Raise "
-                  f"server.max_parallel_requests (and server.queue_max_size) to "
-                  f"at least the number of documents you intend to have in "
-                  f"flight.", flush=True)
+        if _ocr_on and _gate < _GATE_FLOOR:
+            print(f"[queue] WARNING: OCR is enabled but this server admits only "
+                  f"{_gate} request(s) at a time (server.max_parallel_requests) "
+                  f"with {queue_manager.max_size} queued, which is set "
+                  f"explicitly. The admission gate, not the GPU or the pod pool, "
+                  f"will be the throughput ceiling: pods sit at 'Running: 1, "
+                  f"Waiting: 0' while work queues here. Unset it to have it "
+                  f"derived, or raise it to the number of documents you intend "
+                  f"to have in flight.", flush=True)
     except Exception:
         pass            # a diagnostic must never be the reason a server fails to start
 

tests/test_admission_gate_derived.py

@@ -108,11 +108,15 @@ def test_queue_is_capped():
 
 def test_main_applies_the_same_derivation():
     src = open("codai/main.py").read()
-    assert 'max(16, min(256, _page_budget // 8))' in src
+    # Named constants, shared with the warning: a separate literal in each
+    # place is how the two rules came to disagree and warn about a value the
+    # derivation had just chosen.
+    assert '_GATE_FLOOR, _GATE_CAP = 16, 256' in src
+    assert 'max(_GATE_FLOOR, min(_GATE_CAP, _page_budget // 8))' in src
     assert '"max_parallel_requests" not in _explicit' in src
     assert 'not _mp' in src, "env override must still win"
     block = src[src.index("An OCR install must not have to know"):]
-    block = block[:block.index("# Say so when this gate is the ceiling")]
+    block = block[:block.index("# Say so when an EXPLICIT gate")]
     assert "except Exception:" in block, "must never stop a server booting"
 
 

tests/test_admission_gate_warning.py

-"""The admission gate must announce itself when it is the ceiling.
-
-server.max_parallel_requests defaults to 2 in flight and 6 queued. That is
-right for one local card answering chat, and catastrophic for an OCR install
-dispatching pages to rented pods: the pool is sized in pages, the pod is handed
-a much larger limit through env, and the front in front of all of it keeps
-admitting two requests. A fleet idles behind a two-lane gate while every
-diagnostic points at the pods.
-
-The bare "[queue] max_parallel_requests=2" line was printed at every boot while
-exactly this went undiagnosed for days -- a number with no consequence attached
-does not read as a problem. These tests pin the warning that does.
-
-The condition is extracted here as `gate_is_the_ceiling` so it can be tested
-without booting a server; main.py applies the same rule.
+"""The admission-gate warning must fire on the trap and stay silent otherwise.
+
+Context: server.max_parallel_requests defaults to 2 in flight / 6 queued,
+right for one local card answering chat and the throughput ceiling for an OCR
+install dispatching pages to rented pods. One A40 at a gate of 2 sat at
+`Running: 1, Waiting: 0` and read 219 pages in 1210s -- the same as reading
+them one at a time -- while ocr.max_concurrency said 4096.
+
+The first version of this check compared the gate to ocr.max_concurrency
+(gate * 4 < pages). Those are different units, a request against a page, and
+for a 4096-page budget it demanded a gate of 1024 -- memory-unsafe, and more
+than the derivation itself will ever choose. It duly fired against the 256 that
+the same file had just derived. These tests pin the corrected rule and, above
+all, that the warning never contradicts the derivation.
 """
 import pytest
 
+GATE_FLOOR, GATE_CAP = 16, 256
 
-def gate_is_the_ceiling(ocr_enabled, ocr_max_concurrency, gate):
+
+def warn(ocr_enabled, gate):
     """Mirror of the check in codai/main.py."""
-    if not ocr_enabled:
-        return False
-    if not ocr_max_concurrency or ocr_max_concurrency <= 0:
-        return False
-    return gate * 4 < ocr_max_concurrency
+    return bool(ocr_enabled) and int(gate or 0) < GATE_FLOOR
+
+
+def derive(page_budget):
+    """Mirror of the derivation in codai/main.py."""
+    return max(GATE_FLOOR, min(GATE_CAP, page_budget // 8))
+
+
+# --- the trap it exists for ---
+
+def test_the_shipped_default_with_ocr_enabled_is_flagged():
+    assert warn(True, 2) is True
 
 
-def test_the_default_gate_with_ocr_enabled_is_flagged():
-    """The exact shipped default, with OCR on: 2 requests vs 4096 pages."""
-    assert gate_is_the_ceiling(True, 4096, 2) is True
+def test_a_gate_of_one_is_flagged():
+    assert warn(True, 1) is True
 
 
-def test_the_observed_production_case_is_flagged():
-    """What Aruba actually ran: ocr.max_concurrency 4096, gate 2."""
-    assert gate_is_the_ceiling(True, 4096, 2) is True
+# --- the regression that shipped: never contradict the derivation ---
 
+@pytest.mark.parametrize("page_budget", [8, 128, 1024, 2048, 4096, 99999])
+def test_a_derived_gate_is_never_warned_about(page_budget):
+    """The bug: 256 was derived from a 4096-page budget and then warned about."""
+    assert warn(True, derive(page_budget)) is False
 
-def test_a_raised_gate_is_not_flagged():
-    """64 requests x ~9 pages each covers a 4096-page budget's useful range."""
-    assert gate_is_the_ceiling(True, 4096, 2048) is False
 
+def test_the_observed_production_values_are_silent():
+    """ocr.max_concurrency=4096 with the gate at 256 is correct, not a problem."""
+    assert warn(True, 256) is False
+
+
+def test_a_large_page_budget_does_not_by_itself_trigger_a_warning():
+    """The page budget is not the unit; only the gate's own size is."""
+    assert warn(True, GATE_FLOOR) is False
+
+
+# --- not everyone's problem ---
 
 def test_ocr_disabled_is_never_flagged():
-    """A chat-only install is exactly who the default of 2 is for."""
-    assert gate_is_the_ceiling(False, 4096, 2) is False
+    assert warn(False, 2) is False
+    assert warn(False, 1) is False
+
 
+def test_the_boundary_is_the_shared_floor():
+    assert warn(True, GATE_FLOOR - 1) is True
+    assert warn(True, GATE_FLOOR) is False
 
-def test_no_declared_page_budget_is_not_flagged():
-    """Nothing to compare against: say nothing rather than guess."""
-    assert gate_is_the_ceiling(True, 0, 2) is False
-    assert gate_is_the_ceiling(True, None, 2) is False
 
+# --- the rules above must be the rules main.py applies ---
 
-@pytest.mark.parametrize("pages,gate,expect", [
-    (8, 2, False),        # gate*4 == pages: covered, no warning
-    (9, 2, True),         # one page beyond what the gate can cover
-    (64, 16, False),
-    (64, 15, True),
-])
-def test_the_boundary_is_four_pages_per_admitted_request(pages, gate, expect):
-    """Four is slack for one request carrying several documents; past it the
-    gate provably cannot deliver the page budget the operator asked for."""
-    assert gate_is_the_ceiling(True, pages, gate) is expect
+def test_main_uses_one_shared_floor_for_both_rules():
+    """A separate constant in each place is how they came to disagree."""
+    src = open("codai/main.py").read()
+    assert "_GATE_FLOOR, _GATE_CAP = 16, 256" in src
+    assert "_gate < _GATE_FLOOR" in src, "warning must use the shared floor"
+    assert "max(_GATE_FLOOR, min(_GATE_CAP, _page_budget // 8))" in src, \
+        "derivation must use the shared floor and cap"
+
+
+def test_main_no_longer_compares_requests_against_pages():
+    """The unit error that caused the false positive must not come back."""
+    src = open("codai/main.py").read()
+    assert "_gate * 4 < _pages" not in src
 
 
-def test_main_applies_the_same_rule():
-    """The rule above is not a second implementation drifting from the first."""
-    import re
+def test_the_warning_is_exception_guarded():
     src = open("codai/main.py").read()
-    assert "_gate * 4 < _pages" in src, "main.py no longer uses this condition"
-    # and it must be guarded so a diagnostic cannot stop a server booting
-    block = src[src.index("Say so when this gate is the ceiling"):]
-    block = block[:block.index("# RunPod:")]
+    block = src[src.index("Say so when an EXPLICIT gate is small enough"):]
     assert "except Exception:" in block
-    assert re.search(r"server\.max_parallel_requests", block)