-
Stefy Lanza (nextime / spora ) authored
Raised from the Digesta side: the one high-leverage way to make surya fall through less often is to make its repeat-detector and retry tunable. Half right, and the other half turned out to be the bigger lever. Checking the installed surya rather than the description: `_should_retry` stops at `max_retries` and RETURNS the looping text — surya does not return empty. The empty page comes later, because looping markup parses to no blocks. So the symptom is real and the mechanism is not what it looked like. **The lever is that surya never said it looped.** chandra has reported `repeat_detected` since it shipped, with the text returned anyway because a looping transcription usually has a correct beginning. surya reported nothing, so a short or empty page was indistinguishable from an unreadable scan — and those need opposite handling: one is a cascade candidate, the other means the document is bad. surya now flags it at the moment the verdict is made (the retry hook's last call for the page), with `repeat_chars`, the length of the reply that was still looping: "looped and produced four thousand characters" has a usable beginning, "looped and produced nothing" does not. **And the detector is now retunable** — `ocr.surya_repeat_max_repeats`, `surya_repeat_window`, `surya_repeat_scaling`, 0 on each leaving surya's own (4, 500, 3.0). Hooked like `_should_retry` already was, since the vLLM backend passes neither. A false positive is expensive: the page is re-run several times at rising temperature and then parsed from a reply nobody wanted, usually yielding less text than the first one had. Operator settings, deliberately not per-request — they describe the corpus (tables, dotted leaders and form fields repeat short strings legitimately), not the page. An explicit argument from surya's own code still wins, so its `cut_from_end` call keeps working. 11 tests in tests/test_surya_loop_reported.py, including that a loop surya then rescues is not recorded as a verdict, and that the thresholds must be in place BEFORE the hook is installed (it patches once per process). Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
0e334bae