• Stefy Lanza (nextime / spora )'s avatar
    a control plane that rented pods to deliver a teardown (v0.3.87) · 7d0f24aa
    Stefy Lanza (nextime / spora ) authored
    Seven items from a production field report. Three of them were one cause.
    
    /v1/runpod is NOT in the remote gateway's local list. It carries a `model`
    field like an inference request, so the gateway read it as one, resolved the
    model to its pod, and FORWARDED the control-plane call there. Asking for six
    pods back rented a seventh to ask on; the pod has no runpod configuration of
    its own, so it answered
    
        404 {"detail": "'datalab-to/surya-ocr-2' has no runpod block"}
    
    which reads exactly like the orchestrator refusing a name it should know. The
    same call blocked for minutes while a pod booted to receive it. An operator
    could not turn off $3.54/hr from the API. Reproduced here while testing the
    fix: two teardown requests rented two A40s, which is the clearest possible
    statement of the bug.
    
    surya answered 200 with empty pages for a whole document. Three failures in a
    row, each of which had to be fixed:
    
      * its client catches the HTTP error itself and returns a generation with no
        tokens, so the lease check in recognize_image — which only ran in `except`
        — never ran, and the pod address was never re-resolved;
      * the repetition detector was handed that empty string, found no progress in
        it and re-ran the page at a rising temperature, four full paid generations
        per page, `tokens: [0,0,0,0]`;
      * and the page came back as read. We had already measured the ink and said
        there IS something on it; the judgement never reached the response, so a
        client treating an empty read as an unreadable scan consumed roughly a
        thousand documents from its queue and wrote them off.
    
    A page with ink and no text is now `unread`: it goes to the fallback engine
    like any other degraded page, it is listed in `unread_pages`, and a document
    whose every inked page came back empty is no longer a 200 (`ocr.unread_status`,
    502 by default; a partially read document stays 200 with the list).
    
    `bge-m3` and `BAAI/bge-m3` each rented their own A40 — the pool key was
    whichever name the request carried. Keyed on the model ENTRY now, so every name
    for one set of weights lands on one pool.
    
    A client read timeout cannot bound an OCR call: it times each READ, so a
    response that dribbles never trips it and one stayed open nineteen minutes
    against timeout=420. `ocr.request_timeout_s` is the server-side stopwatch.
    
    Chunking: `it-legal` accepts a decorated heading (`**DISPOSITIVO**`) where the
    implementation it ports does not, which splits one more section and leaves a
    short trailing chunk. That was true and undocumented — the table printed the
    strict pattern. Documented now, with `it-legal-strict` named as the
    byte-identical one for re-chunking an existing index. `min_chunk_chars` folds a
    runt chunk into its predecessor, off by default because merging moves a
    boundary, and never silently: the absorbed section label is reported, since
    `conclusioni` is what a party asked for and `dispositivo` is what the court
    decided.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    7d0f24aa
ocr_worker.py 79.9 KB