-
Stefy Lanza (nextime / spora ) authored
Suggested by a third party and worth keeping: run layout detection, count the blocks, and pick full-page or block mode from the count. Recorded rather than built, because our own evidence says the repetition it would prevent came from the 3072-token cap (532 of 594 continuations were segment 2, and the loops appear AT the truncation boundary, where a continuation re-sends the page its own output), and v0.4.22 raises that budget. The argument that survives the budget fix is GPU utilisation: one full-page generation is a single sequence and cannot be batched with itself, which is consistent with pods idling at 1.4% KV cache and with an A100 PCIe delivering only 1.14x an A40. SURYA_INFERENCE_BACKEND=vllm is already set, so the batching it would rely on is in place. The doc states the three measurements that decide it. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
23faf1a1