size the summarizer by the window that refused us (v0.4.75)

Both of these surfaced the moment a config said 131072 while the pod was still
serving the 32768 it had been launched with -- a disagreement that is normal for
a few minutes after an edit, and which compaction has to survive.

**The summarizer was sized from the configured window.** It chunks the history
it is about to summarize to fit the summarizing model's context, and that number
came straight from the backend's configured value: 131072 against a pod serving
32768, so every summarization request overflowed too. The slicer then caught the
failure, left the oversized message exactly as it was, and compaction reported
that it could not help. It now prefers a window an engine has already refused
against, like the plan itself does.

**The head's size cap was waived for its first message.** The intent was to keep
at least one anchoring turn, but it meant a conversation opening with a pasted
45,000-token document kept that document verbatim as the head, dropped the rest
of the history, and overflowed anyway -- the one shape where compaction matters
most. An opening turn too big to keep belongs to the slicer, which summarizes
its middle.

Also says on the model page what the RunPod "Max context" field actually is:
the window the pod is LAUNCHED with, which the general context field above it
does not reach, which takes effect on the next pod rather than the running one,
and which the first model to create a shared pool sets for every model on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.74"
+__version__ = "0.4.75"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/templates/models.html

@@ -869,6 +869,7 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
         <div class="form-row" style="margin:.5rem 0 0">
           <label class="form-label">Max context <span class="muted">(tokens, optional)</span></label>
           <input type="number" id="cfg-rp-ctx" class="form-input" min="0" placeholder="0 = default">
+          <span class="form-hint" style="font-size:11px">The window the pod is <b>launched</b> with (<code>--max-model-len</code>), and the one this model reports to clients and sizes auto-compaction from. The general context field above does not reach a pod. It takes effect on the <b>next pod</b>: a running one keeps the window it started with, and a pod that is re-adopted keeps it too. With a shared pool, the first model to create the pool sets this for every model on it — so set the same value on all of them.</span>
         </div>
 
         <!-- Serverless-only -->

codai/api/text.py

@@ -516,7 +516,14 @@ def _split_for_compaction(messages, target, strategy, head_messages=1,
                 if len(head) >= want:
                     break
                 t = _estimate_tokens([m])
-                if head and used + t > cap:
+                # The cap binds the FIRST message too. It used to be waived for
+                # it ("keep at least one anchor"), so a conversation whose
+                # opening turn is a pasted 45,000-token document kept that
+                # document verbatim as the head, dropped everything else, and
+                # overflowed anyway -- the one shape where compaction most needs
+                # to work. An opening turn too big to keep belongs to the
+                # slicer, which summarizes its middle, not to the head.
+                if used + t > cap:
                     break
                 head.append(m)
                 used += t
@@ -1052,6 +1059,21 @@ def _resolve_compact_manager(plan):
         compact_n_ctx = compact_manager.get_context_size() if compact_manager else 0
     except Exception:
         compact_n_ctx = 0
+    # A window an engine has already refused against beats the configured one
+    # HERE TOO, and for a sharper reason than in the plan: this number sizes the
+    # summarizer's own chunks, so trusting a configured 131072 against a pod
+    # serving 32768 makes every summarization request overflow -- and the slicer
+    # then leaves the oversized message exactly as it was and reports that
+    # compaction could not help. Measured 2026-10-22, the moment a config said
+    # 131072 before the pod had been relaunched with it.
+    try:
+        from codai.api import context_overflow as _co
+        _served = _co.learned_limit(compact_name) or _co.learned_limit(
+            plan.get("model") or "")
+        if _served and (not compact_n_ctx or _served < compact_n_ctx):
+            compact_n_ctx = int(_served)
+    except Exception:
+        pass
     return compact_manager, compact_name, (compact_n_ctx or plan.get("n_ctx") or 4096)
 
 

tests/test_context_compaction.py

@@ -403,3 +403,38 @@ def test_the_summarizer_is_given_only_the_messages_being_dropped():
     assert f"{last_kept} " not in handed
     # ...and something from the middle is.
     assert "msg20" in handed or "msg15" in handed
+
+
+# ------------------------- one message bigger than the whole target ---------- #
+
+def test_a_huge_opening_turn_is_not_kept_verbatim_as_the_head():
+    """THE BUG: the head's size cap was waived for its FIRST message, so a
+    conversation that opens with a pasted 45,000-token document kept that
+    document whole, dropped the rest of the history, and overflowed anyway --
+    the exact shape compaction exists for. Measured 2026-10-22.
+    """
+    huge = {"role": "user", "content": "DOC " + ("x" * 180_000)}
+    msgs = [{"role": "system", "content": "s"}, huge]
+    msgs += [{"role": "user" if i % 2 == 0 else "assistant",
+              "content": f"m{i} " + "y" * 400} for i in range(10)]
+    _, head, middle, tail = _split_for_compaction(
+        msgs, target=12_000, strategy="summarize", head_messages=2)
+    assert huge not in head
+    assert huge in middle              # it goes to the summarizer/slicer instead
+    assert tail
+
+
+def test_the_summarizer_is_sized_by_the_window_that_refused_us():
+    """THE SECOND BUG, and the sharper one: this number sizes the summarizer's
+    own chunks. With a configured 131072 against a pod serving 32768, every
+    summarization request overflowed too -- so the slicer left the oversized
+    message exactly as it was and compaction reported that it could not help.
+    """
+    from codai.api.text import _resolve_compact_manager
+    plan = _plan([], n_ctx=131072, current_manager=_FakeManager(131072),
+                 model="qwen38")
+    _mgr, _name, n = _resolve_compact_manager(plan)
+    assert n == 131072                          # nothing refused us yet
+    co.remember_limit("qwen38", 32768)           # ...then a pod does
+    _mgr, _name, n = _resolve_compact_manager(plan)
+    assert n == 32768