-
Stefy Lanza (nextime / spora ) authored
From last night's run: ~97% of a 1,000-document batch came out truncated, with no error anywhere. The engine answered 200 every time. Its decode had fallen into a loop, run to the token cap emitting the same fragment, and the resumer correctly refused to continue a reply that was already repeating itself -- so the page was flagged, returned, and indexed. `fallback_engines` never fired, because it only fires on an exception. The same pages read cleanly on another engine (38,055 characters against a truncated stub), so the recovery was there the whole time and nothing reached for it. `ocr.retry_degraded` (default "auto") re-reads ONLY the degraded pages on another engine and keeps the replacement only if it is not degraded in turn. Bounded by construction: a clean document costs nothing, a ruined one costs at most a second pass over its ruined pages, and `retry_degraded_max_pages` caps even that. The page keeps its own index and carries `degraded_retry` saying who re-read it and why; the response reports `recovered_pages`. Degraded means looping or truncated -- the two the engine itself attests. NOT a short page, a blank page or an unreadable one: those have their own meaning and re-reading them spends a card to be told the same thing. Which leaves the other half of the report: olmOCR returned 4,046 characters where chandra read 30,983 on the same page, ending cleanly well short of the cap, flagging nothing. The only evidence an early stop leaves is the company it keeps, so `retry_short_pages_below` compares a page with the DOCUMENT'S OWN median rather than a character count -- a threshold in characters would need re-tuning for every language, DPI and layout. Default 0, OFF, and deliberately: this signal is inferred where the others are attested, and a genuine title or signature page looks identical. A short page is also only replaced by a LONGER one. Off on pods: only the orchestrator can reach another engine's pods, so a pod would re-read on the engine that just produced the flag. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
731ae0e4