ocr: surya says when a page was still looping, and its detector is tunable (v0.3.64)

Raised from the Digesta side: the one high-leverage way to make surya fall
through less often is to make its repeat-detector and retry tunable. Half right,
and the other half turned out to be the bigger lever.

Checking the installed surya rather than the description: `_should_retry` stops
at `max_retries` and RETURNS the looping text — surya does not return empty. The
empty page comes later, because looping markup parses to no blocks. So the
symptom is real and the mechanism is not what it looked like.

**The lever is that surya never said it looped.** chandra has reported
`repeat_detected` since it shipped, with the text returned anyway because a
looping transcription usually has a correct beginning. surya reported nothing,
so a short or empty page was indistinguishable from an unreadable scan — and
those need opposite handling: one is a cascade candidate, the other means the
document is bad. surya now flags it at the moment the verdict is made (the
retry hook's last call for the page), with `repeat_chars`, the length of the
reply that was still looping: "looped and produced four thousand characters" has
a usable beginning, "looped and produced nothing" does not.

**And the detector is now retunable** — `ocr.surya_repeat_max_repeats`,
`surya_repeat_window`, `surya_repeat_scaling`, 0 on each leaving surya's own (4,
500, 3.0). Hooked like `_should_retry` already was, since the vLLM backend
passes neither. A false positive is expensive: the page is re-run several times
at rising temperature and then parsed from a reply nobody wanted, usually
yielding less text than the first one had. Operator settings, deliberately not
per-request — they describe the corpus (tables, dotted leaders and form fields
repeat short strings legitimately), not the page. An explicit argument from
surya's own code still wins, so its `cut_from_end` call keeps working.

11 tests in tests/test_surya_loop_reported.py, including that a loop surya then
rescues is not recorded as a verdict, and that the thresholds must be in place
BEFORE the hook is installed (it patches once per process).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.63"
+__version__ = "0.3.64"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/routes.py

@@ -4155,6 +4155,9 @@ def build_settings_dict(c, gpu_cards):
             "surya_max_retries": getattr(c.ocr, "surya_max_retries", 1),
             "surya_max_tokens": getattr(c.ocr, "surya_max_tokens", 0),
             "surya_max_continuations": getattr(c.ocr, "surya_max_continuations", 2),
+            "surya_repeat_max_repeats": getattr(c.ocr, "surya_repeat_max_repeats", 0),
+            "surya_repeat_window": getattr(c.ocr, "surya_repeat_window", 0),
+            "surya_repeat_scaling": getattr(c.ocr, "surya_repeat_scaling", 0.0),
             "surya_context_tokens": getattr(c.ocr, "surya_context_tokens", 0),
             "surya_langs": c.ocr.surya_langs,
             "surya_token_trace": c.ocr.surya_token_trace,
@@ -4999,6 +5002,16 @@ async def api_save_settings(request: Request, username: str = Depends(require_ad
             o.surya_max_continuations = _i(
                 "surya_max_continuations",
                 getattr(o, "surya_max_continuations", 2), 0)
+        if "surya_repeat_max_repeats" in d:
+            o.surya_repeat_max_repeats = _i(
+                "surya_repeat_max_repeats",
+                getattr(o, "surya_repeat_max_repeats", 0), 0)
+        if "surya_repeat_window" in d:
+            o.surya_repeat_window = _i(
+                "surya_repeat_window", getattr(o, "surya_repeat_window", 0), 0)
+        if "surya_repeat_scaling" in d:
+            o.surya_repeat_scaling = _f(
+                "surya_repeat_scaling", getattr(o, "surya_repeat_scaling", 0.0), 0.0)
         if "surya_context_tokens" in d:
             o.surya_context_tokens = _i(
                 "surya_context_tokens",

codai/admin/templates/settings.html

@@ -1214,6 +1214,9 @@
       <div class="form-group" style="flex:1;min-width:140px"><label class="form-label" title="Cap on the tokens Surya may generate for ONE page. 0 uses Surya's own 2048, which its vLLM backend never overrides. Surya-ocr-2 emits block markup (a category and a bbox per block) BEFORE the text, so a reply stopped at the cap is incomplete markup and parses to a few blocks or to none — the page then arrives as a short fragment or as no text at all, which looks exactly like a page the model could not read. A dense page can legitimately need several thousand tokens. model_attempts.truncated in a page's result tells you when this is what went wrong.">Max tokens / page</label><input type="number" id="s-ocr-surya-max-tokens" class="form-input" min="0" placeholder="0 = 2048"></div>
       <div class="form-group" style="flex:1;min-width:140px"><label class="form-label" title="How many times a page whose reply hit the token cap may be RESUMED from where it stopped. Raising Max tokens only moves the cliff — a dense enough page overflows any fixed number, and incomplete block markup parses to a fragment or to nothing. Resumption sends the text so far back as a prefilled assistant turn so the model carries on the exact string. 2 by default, because returning a page known to be incomplete is a correctness failure, not a speed trade-off; 0 restores the old behaviour. A page still unfinished afterwards is marked model_attempts.complete = false so a caller can refuse it.">Resume cut-off pages</label><input type="number" id="s-ocr-surya-max-continuations" class="form-input" min="0" placeholder="2"></div>
       <div class="form-group" style="flex:1;min-width:150px"><label class="form-label" title="The serving model's context window, in tokens. 0 asks the server for its own max_model_len (vLLM reports it on /v1/models) — leave it at 0 unless the server will not say, or is served with a shorter --max-model-len than the checkpoint advertises. This is the ceiling resumption really runs into: every resumption re-sends the text so far as INPUT, so once the window is nearly full, asking for another whole Max tokens is not a bigger segment, it is an HTTP 400 — and a page that was nearly whole gets reported as a page that could not be resumed at all. Knowing the number lets such a page stop one segment early, keep what it already has and say so. It also means a SMALLER Max tokens fits more text in the same window, because the last segment wastes less of it.">Context window</label><input type="number" id="s-ocr-surya-context-tokens" class="form-input" min="0" placeholder="0 = ask the server"></div>
+      <div class="form-group" style="flex:1;min-width:150px"><label class="form-label" title="Surya's repetition detector, retuned. 0 leaves its own thresholds (4 repeats, a 500-character window, scaling 3.0). A page judged degenerate is re-run at rising temperature and then parsed from whatever came back, so a FALSE positive costs several full generations and usually returns less text than the first reply had — and a page still looping when Surya gives up now arrives flagged repeat_detected, which is the signal to route it to another engine rather than read it as an unreadable scan. Whether a repetition is degeneration depends on the documents: tables, dotted leaders and form fields repeat short strings legitimately. Raise this to be more forgiving, lower it to be stricter.">Repeat threshold</label><input type="number" id="s-ocr-surya-repeat-max-repeats" class="form-input" min="0" placeholder="0 = surya's 4"></div>
+      <div class="form-group" style="flex:1;min-width:140px"><label class="form-label" title="How far back the repetition detector looks, in characters. 0 = surya's 500.">Repeat window</label><input type="number" id="s-ocr-surya-repeat-window" class="form-input" min="0" placeholder="0 = 500"></div>
+      <div class="form-group" style="flex:1;min-width:140px"><label class="form-label" title="How much more forgiving the detector is of SHORT repeated sequences: the threshold is multiplied by (1 + scaling / sequence length), so a one-character run needs far more repetitions than a sentence does. 0 = surya's 3.0.">Repeat scaling</label><input type="number" step="0.1" id="s-ocr-surya-repeat-scaling" class="form-input" min="0" placeholder="0 = 3.0"></div>
       <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer" title="Local mode only. On, Surya runs on this machine's GPU or refuses to load — it will not quietly fall back to the CPU, where a page takes minutes. Off, the CPU is used deliberately."><input type="checkbox" id="s-ocr-surya-use-gpu" checked><span>Require GPU (Local)</span></label>
       <div class="form-group" style="flex:1;min-width:120px"><label class="form-label">Langs</label><input type="text" id="s-ocr-surya-langs" class="form-input" placeholder="it"></div>
       <label style="display:flex;align-items:center;gap:.4rem;cursor:pointer" title="Return the model's own per-token bytes wherever a page came back unreadable. A diagnostic — leave off unless you are chasing lost characters."><input type="checkbox" id="s-ocr-surya-token-trace"><span>Trace unreadable tokens</span></label>
@@ -2225,6 +2228,9 @@ async function loadSettings(){
     document.getElementById('s-ocr-surya-max-tokens').value = ocr.surya_max_tokens ?? 0;
     document.getElementById('s-ocr-surya-max-continuations').value = ocr.surya_max_continuations ?? 2;
     document.getElementById('s-ocr-surya-context-tokens').value = ocr.surya_context_tokens ?? 0;
+    document.getElementById('s-ocr-surya-repeat-max-repeats').value = ocr.surya_repeat_max_repeats ?? 0;
+    document.getElementById('s-ocr-surya-repeat-window').value = ocr.surya_repeat_window ?? 0;
+    document.getElementById('s-ocr-surya-repeat-scaling').value = ocr.surya_repeat_scaling ?? 0;
     document.getElementById('s-ocr-surya-use-gpu').checked = ocr.surya_use_gpu !== false;
     document.getElementById('s-ocr-surya-langs').value = ocr.surya_langs ?? 'it';
     document.getElementById('s-ocr-surya-token-trace').checked = !!ocr.surya_token_trace;
@@ -2523,6 +2529,9 @@ async function saveSettings(silent){
       // 0 is meaningful (resumption off), so an explicit NaN check.
       surya_max_continuations: (function(v){ v = parseInt(v); return isNaN(v) ? 2 : v; })(
         document.getElementById('s-ocr-surya-max-continuations').value),
+      surya_repeat_max_repeats: parseInt(document.getElementById('s-ocr-surya-repeat-max-repeats').value) || 0,
+      surya_repeat_window: parseInt(document.getElementById('s-ocr-surya-repeat-window').value) || 0,
+      surya_repeat_scaling: parseFloat(document.getElementById('s-ocr-surya-repeat-scaling').value) || 0,
       surya_context_tokens: parseInt(document.getElementById('s-ocr-surya-context-tokens').value) || 0,
       surya_langs: document.getElementById('s-ocr-surya-langs').value.trim() || 'it',
       surya_token_trace: document.getElementById('s-ocr-surya-token-trace').checked,

codai/config.py

@@ -1102,6 +1102,22 @@ class OcrConfig:
     #: stops making progress stops early regardless, so a generous budget costs
     #: nothing on a page that cannot use it. 0 disables resumption.
     surya_max_continuations: int = 8
+    #: surya's repetition detector, retuned. 0 on all three leaves its own
+    #: defaults (4 repeats, a 500-character window, scaling 3.0).
+    #:
+    #: A page it judges degenerate is re-run up to `surya_max_retries` times at
+    #: rising temperature and then parsed from whatever came back, so a FALSE
+    #: positive costs several full generations and usually returns less text
+    #: than the first reply had. Whether a repetition is degeneration depends on
+    #: the documents — a corpus of tables, dotted leaders or form fields repeats
+    #: short strings legitimately — which is why this is an operator setting and
+    #: not a per-request one: it describes the corpus, not the page.
+    #:
+    #: Raise `surya_repeat_max_repeats` to be more forgiving (more repetitions
+    #: before a reply counts as looping), lower it to be stricter.
+    surya_repeat_max_repeats: int = 0
+    surya_repeat_window: int = 0
+    surya_repeat_scaling: float = 0.0
     #: The serving model's context window, or 0 to ask the server for its own
     #: `max_model_len` (vLLM reports it on `/v1/models`).
     #:
@@ -1899,6 +1915,12 @@ class ConfigManager:
                 "surya_max_tokens": getattr(self.config.ocr, "surya_max_tokens", 0),
                 "surya_max_continuations": getattr(self.config.ocr,
                                                    "surya_max_continuations", 2),
+                "surya_repeat_max_repeats": getattr(
+                    self.config.ocr, "surya_repeat_max_repeats", 0),
+                "surya_repeat_window": getattr(self.config.ocr,
+                                               "surya_repeat_window", 0),
+                "surya_repeat_scaling": getattr(self.config.ocr,
+                                                "surya_repeat_scaling", 0.0),
                 "surya_langs": self.config.ocr.surya_langs,
                 "surya_token_trace": self.config.ocr.surya_token_trace,
                 "surya_repair_accents": self.config.ocr.surya_repair_accents,

codai/ocr/surya.py

@@ -83,7 +83,16 @@ class SuryaEngine(SubprocessOcrEngine):
                     self.cfg, "surya_max_continuations", 2) or 0)),
                 # 0 = ask the server for its own max_model_len.
                 "context_tokens": max(0, int(getattr(
-                    self.cfg, "surya_context_tokens", 0) or 0))}
+                    self.cfg, "surya_context_tokens", 0) or 0)),
+                # The repetition detector's thresholds, when retuned. 0 leaves
+                # surya's own, which is what every install gets until someone
+                # measures their corpus and decides otherwise.
+                "repeat_max_repeats": max(0, int(getattr(
+                    self.cfg, "surya_repeat_max_repeats", 0) or 0)),
+                "repeat_window": max(0, int(getattr(
+                    self.cfg, "surya_repeat_window", 0) or 0)),
+                "repeat_scaling": max(0.0, float(getattr(
+                    self.cfg, "surya_repeat_scaling", 0.0) or 0.0))}
 
     #: Modes where the recognition VLM lives in a model server we only talk to.
     SERVED_MODES = ("model", "vllm", "llamacpp", "server")

codai/ocr/workers/ocr_worker.py

@@ -87,6 +87,13 @@ _PROMPT_TOKENS_GUESS = 4096
 #: the log, and the number is the same every time.
 _CAP_NOTED = [False]
 
+#: Thresholds for surya's own repetition detector, when the operator has set
+#: them. Empty = surya's defaults (4 repeats, a 500-character window, scaling
+#: 3.0). A corpus property rather than a per-document one: a page of tables and
+#: dotted leaders legitimately repeats short strings, and whether that counts as
+#: degeneration depends on the documents, not on the request.
+_REPEAT_OPTS: dict = {}
+
 #: Below this there is no point asking for more: a segment this short cannot
 #: close an open block, so it would spend a round to add nothing. It is a floor
 #: on the room LEFT, never on the cap -- a deployment whose cap is smaller than
@@ -217,6 +224,8 @@ def _trace_begin():
     st["still_truncated"] = False
     st.pop("no_resume", None)
     st.pop("budget", None)
+    st.pop("looping", None)
+    st.pop("loop_chars", None)
     return st["calls"], st["repairs"]
 
 
@@ -318,6 +327,26 @@ def _no_resume(reason: str) -> None:
         pass
 
 
+def _note_loop(raw: str) -> None:
+    """Record that this page's generation ended still repeating itself.
+
+    Only when surya has stopped retrying: a loop it then rescues is not a
+    property of the page. The character count goes with it because "looped and
+    produced four thousand characters of garbage" and "looped and produced
+    nothing" need different handling — the first has a usable beginning.
+    """
+    try:
+        from surya.inference.backends import openai_client as _oc
+        repeats = getattr(_oc, "detect_repeat_token", None)
+        if repeats is None or not repeats(raw):
+            return
+    except Exception:
+        return
+    st = _state()
+    st["looping"] = True
+    st["loop_chars"] = len(raw or "")
+
+
 def _page_override(name: str, default):
     """A per-request value for the page this THREAD is reading, else the default.
 
@@ -1163,6 +1192,18 @@ class SuryaWorker:
         #: does not report one, or when it serves a shorter window than the
         #: checkpoint advertises.
         _CONTEXT_TOKENS[0] = max(0, int(opts.get("context_tokens", 0) or 0))
+        _REPEAT_OPTS.clear()
+        for key, name in (("repeat_max_repeats", "base_max_repeats"),
+                          ("repeat_window", "window_size"),
+                          ("repeat_scaling", "scaling_factor")):
+            val = opts.get(key)
+            try:
+                num = float(val or 0)
+            except (TypeError, ValueError):
+                num = 0.0
+            if num > 0:
+                _REPEAT_OPTS[name] = (num if name == "scaling_factor"
+                                      else int(num))
         #: The page's token logprobs and accent repairs live in _TRACE, per
         #: thread — see _trace_begin(). They were per-worker-object until lanes
         #: put several pages in one process.
@@ -1337,6 +1378,22 @@ class SuryaWorker:
         # spend is logged. `max_retries` is not reachable from surya's settings:
         # the vLLM backend never passes it, so the default was the only value
         # available. Hooking the decision is the only seam there is.
+        # The detector itself, when the operator has retuned it. surya's
+        # thresholds are sensible defaults, not universal truths: a corpus of
+        # tables and dotted leaders repeats short strings legitimately, and a
+        # page wrongly judged degenerate is re-run three times at rising
+        # temperature and then parsed from a reply nobody wanted.
+        detector = getattr(_oc, "detect_repeat_token", None)
+        if detector is not None and _REPEAT_OPTS:
+            def _tuned_detector(text, *a, **kw):
+                for name, value in _REPEAT_OPTS.items():
+                    kw.setdefault(name, value)
+                return detector(text, *a, **kw)
+            _oc.detect_repeat_token = _tuned_detector
+            print(f"[surya] repetition detector: "
+                  + ", ".join(f"{k}={v}" for k, v in sorted(_REPEAT_OPTS.items())),
+                  file=sys.stderr, flush=True)
+
         inner_retry = getattr(_oc, "_should_retry", None)
         if inner_retry is not None:
             def _wrapped_retry(result, retries, max_retries, *a, **kw):
@@ -1349,6 +1406,15 @@ class SuryaWorker:
                           f"{retries + 1}/{max_retries} (temperature rises; a "
                           f"retried page costs another full generation)",
                           file=sys.stderr, flush=True)
+                else:
+                    # The last word on this page. If it STILL repeats, surya has
+                    # given up and the looping reply is what the page will be
+                    # parsed from — which usually yields a few blocks or none,
+                    # and then reads downstream exactly like a page the model
+                    # could not see. chandra has said "repeat_detected" since it
+                    # shipped; surya said nothing, so a cascade had to guess
+                    # from an empty page. Recorded here, where the verdict is.
+                    _note_loop(getattr(result, "raw", "") or "")
                 return again
             _oc._should_retry = _wrapped_retry
 
@@ -1477,6 +1543,14 @@ class SuryaWorker:
         # was slow and a page that was generated two, three or four times over,
         # and without it the only visible symptom is a batch that produces far
         # fewer completions than the GPU's throughput says it should.
+        # A page whose generation was still looping when surya gave up. The
+        # text below it is parsed from that reply, so it is usually short or
+        # empty — and this flag is the difference between "route this one to
+        # another engine" and "the scan is unreadable", which are opposite
+        # conclusions from the same blank page.
+        if _state().get("looping"):
+            page["repeat_detected"] = True
+            page["repeat_chars"] = int(_state().get("loop_chars") or 0)
         tries = _attempts()
         toks = _attempt_token_counts()
         cap = int(_page_override("max_tokens", _MAX_TOKENS[0])

docs/ocr.md

@@ -253,6 +253,48 @@ model actually saw as the page's `width`/`height`, with the size you rasterised
 and de-normalising them against the pre-scaling size would put every box on a scaled page
 in the wrong place.
 
+### A page that was still looping when the engine gave up
+
+A VLM that loses its place repeats the last thing it saw until it runs out of
+tokens. Both VLM engines retry such a page at rising temperature — chandra's own
+client does, and surya's — and when the retries run out the looping reply is
+what the page is parsed from. That parses to a few blocks or to none, so the
+page arrives short or empty.
+
+**Short or empty is also what an unreadable scan looks like, and the two need
+opposite handling**: a looping page is a cascade candidate (hand it to another
+engine), an unreadable one means the document itself is bad. chandra has flagged
+this since it shipped; surya said nothing, so a cascade had to infer it from
+emptiness. Both now report it:
+
+```json
+"repeat_detected": true,
+"repeat_chars": 4096
+```
+
+`repeat_chars` is the length of the reply that was still looping — "looped and
+produced four thousand characters" has a usable beginning, "looped and produced
+nothing" does not.
+
+#### Retuning the detector (`ocr.surya_repeat_*`)
+
+surya's thresholds — 4 repetitions, a 500-character window, scaling 3.0 — are
+defaults, not universal truths. A corpus of tables, dotted leaders or form
+fields repeats short strings legitimately, and a page wrongly judged degenerate
+is re-run up to `surya_max_retries` times at rising temperature and then parsed
+from a reply nobody wanted: a false positive costs several full generations and
+usually returns LESS text than the first reply had.
+
+| setting | 0 means | raise it to |
+|---|---|---|
+| `surya_repeat_max_repeats` | surya's 4 | be more forgiving — more repetitions before a reply counts as looping |
+| `surya_repeat_window` | surya's 500 characters | look further back |
+| `surya_repeat_scaling` | surya's 3.0 | be more forgiving of SHORT repeated sequences (the threshold is multiplied by `1 + scaling / sequence length`) |
+
+Operator settings, deliberately not per-request: they describe the corpus, not
+the page. An explicit argument from surya's own code still wins, so
+`cut_from_end` keeps working.
+
 ### Repetition loops
 
 A VLM that loses its place on a page does not fail. It repeats the last thing it saw until

tests/test_surya_loop_reported.py

+"""A page surya gave up on must SAY it was looping, and the detector must be tunable.
+
+A VLM that loses its place repeats the last thing it saw until it runs out of
+tokens. surya retries such a page at rising temperature and, when the retries
+run out, returns the looping reply anyway — which parses to a few blocks or to
+none. Downstream that is a short or empty page, and it reads exactly like a scan
+the model could not see. They are opposite conclusions: one should be routed to
+another engine, the other means the document itself is bad.
+
+chandra has flagged this since it shipped (`repeat_detected`). surya said
+nothing, so a cascade had to infer it from emptiness — which is why dense pages
+"fell through" with no explanation.
+
+The detector's thresholds are an operator setting, not a per-request one: a
+corpus of tables, dotted leaders or form fields repeats short strings
+legitimately, and whether that is degeneration depends on the documents. A false
+positive is expensive — several full generations, and usually less text than the
+first reply already had.
+"""
+import pathlib
+import sys
+
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT))
+sys.path.insert(0, str(ROOT / "tests"))
+
+from test_surya_retry_budget import (_fake_surya, _worker,           # noqa: E402
+                                     _worker_module, _Result)
+
+
+def _hooked(monkeypatch, **cfg):
+    mod = _worker_module()
+    oc = _fake_surya(monkeypatch, mod)
+    mod._REPEAT_OPTS.clear()
+    w = _worker(mod, token_trace=True, **cfg)
+    w._install_trace_hook()
+    mod._trace_begin()
+    return mod, oc, w
+
+
+# --- the flag ---------------------------------------------------------------
+
+def test_a_page_still_looping_when_surya_gives_up_is_flagged(monkeypatch):
+    mod, oc, _w = _hooked(monkeypatch)
+    oc.detect_repeat_token = lambda text, **kw: "loop" in (text or "")
+    res = _Result(raw="loop loop loop")
+    # retries exhausted: surya's own decision function says no more
+    oc._should_retry(res, 3, 3)
+    assert mod._state()["looping"] is True
+    assert mod._state()["loop_chars"] == len("loop loop loop")
+
+
+def test_a_page_that_stopped_looping_is_not_flagged(monkeypatch):
+    mod, oc, _w = _hooked(monkeypatch)
+    oc.detect_repeat_token = lambda text, **kw: "loop" in (text or "")
+    oc._should_retry(_Result(raw="clean text"), 3, 3)
+    assert "looping" not in mod._state()
+
+
+def test_a_loop_that_will_be_retried_is_not_yet_a_verdict(monkeypatch):
+    """A loop surya then rescues is not a property of the page."""
+    mod, oc, _w = _hooked(monkeypatch)
+    oc.detect_repeat_token = lambda text, **kw: True
+    oc._should_retry(_Result(raw="loop"), 0, 3)      # will retry
+    assert "looping" not in mod._state()
+
+
+def test_the_character_count_separates_garbage_from_nothing(monkeypatch):
+    """"Looped and produced 4,000 characters" has a usable beginning;
+    "looped and produced nothing" does not."""
+    mod, oc, _w = _hooked(monkeypatch)
+    oc.detect_repeat_token = lambda text, **kw: True
+    oc._should_retry(_Result(raw="x" * 4000), 3, 3)
+    assert mod._state()["loop_chars"] == 4000
+
+
+def test_a_new_page_does_not_inherit_the_verdict(monkeypatch):
+    mod, oc, _w = _hooked(monkeypatch)
+    oc.detect_repeat_token = lambda text, **kw: True
+    oc._should_retry(_Result(raw="loop"), 3, 3)
+    mod._trace_begin()
+    assert "looping" not in mod._state() and "loop_chars" not in mod._state()
+
+
+def test_the_flag_reaches_the_page_dict():
+    """The worker's page dict is copied wholesale into OcrPage.meta, so this is
+    what a caller actually sees — the same key chandra reports."""
+    src = (ROOT / "codai/ocr/workers/ocr_worker.py").read_text()
+    assert 'page["repeat_detected"] = True' in src
+    assert 'page["repeat_chars"]' in src
+
+
+# --- the tunable detector ---------------------------------------------------
+
+def _detector_probe(monkeypatch, **cfg):
+    """A fresh surya whose detector records the arguments it is called with.
+
+    Set up BEFORE the hook is installed: the patch happens once per process
+    (`_coderai_traced`), so a detector swapped in afterwards is never wrapped.
+    """
+    mod = _worker_module()
+    oc = _fake_surya(monkeypatch, mod)
+    mod._REPEAT_OPTS.clear()
+    seen = {}
+    oc.detect_repeat_token = lambda text, **kw: seen.update(kw) or False
+    _worker(mod, token_trace=True, **cfg)._install_trace_hook()
+    return mod, oc, seen
+
+
+def test_the_thresholds_are_applied_when_set(monkeypatch):
+    _mod, oc, seen = _detector_probe(monkeypatch, repeat_max_repeats=8,
+                                     repeat_window=200, repeat_scaling=1.5)
+    oc.detect_repeat_token("anything")
+    assert seen == {"base_max_repeats": 8, "window_size": 200,
+                    "scaling_factor": 1.5}
+
+
+def test_unset_thresholds_leave_suryas_own_alone(monkeypatch):
+    _mod, oc, seen = _detector_probe(monkeypatch)
+    oc.detect_repeat_token("anything")
+    # Not even a keyword: surya's signature defaults are the whole point of 0.
+    assert seen == {}
+
+
+def test_an_explicit_argument_still_wins(monkeypatch):
+    """`_should_retry` passes cut_from_end itself; our defaults must not fight
+    the caller's own arguments."""
+    _mod, oc, seen = _detector_probe(monkeypatch, repeat_max_repeats=8)
+    oc.detect_repeat_token("anything", base_max_repeats=2, cut_from_end=50)
+    assert seen["base_max_repeats"] == 2 and seen["cut_from_end"] == 50
+
+
+def test_rubbish_thresholds_are_ignored_rather_than_crashing(monkeypatch):
+    mod = _worker_module()
+    _fake_surya(monkeypatch, mod)
+    mod._REPEAT_OPTS.clear()
+    _worker(mod, token_trace=True, repeat_max_repeats="lots", repeat_window=-5)
+    assert mod._REPEAT_OPTS == {}
+
+
+def test_the_settings_travel_from_the_engine_to_the_worker():
+    """They are sent at LOAD, like the other hook settings — the detector is
+    patched before the first page, not per request."""
+    src = (ROOT / "codai/ocr/surya.py").read_text()
+    for key in ("repeat_max_repeats", "repeat_window", "repeat_scaling"):
+        assert f'"{key}"' in src