• Stefy Lanza (nextime / spora )'s avatar
    a dead socket is not a pod refusing the pipeline (v0.4.6) · a6fc0266
    Stefy Lanza (nextime / spora ) authored
    A pooled keep-alive connection that a pod closes while idle fails on its
    next use:
    
      pod OCR at https://.../v1/ocr is unreachable: ('Connection aborted.',
      RemoteDisconnected('Remote end closed connection without response'))
    
    _post_page_to_pod turned every transport failure into OcrError(503), 503 is
    in _POD_CANNOT_RUN_PIPELINE, so one dead TCP connection was read as "these
    pods cannot run the OCR pipeline" and set the whole pool aside for five
    minutes. It happened twelve times in four hours on a production run: the
    GPU-less orchestrator rasterised every page of every document for those
    windows and climbed to 40 GB at ~0.8 GB/min, while sixteen rented A40s sat
    idle and willing.
    
    An ANSWER may condemn an image; a socket may not. So:
    
    * PodUnreachable, a distinct error for "nothing came back", which never sets
      the pool aside. A pod that ANSWERS 404/501/503 still does.
    * One page is retried on a fresh connection (nothing was sent, so there is no
      side effect) and then on another pod — but a pod is dropped from the pool
      only after it fails to answer /healthz. Writing a pod off because an idle
      connection was reused has cost seven healthy cards before.
    * A timeout is NOT retried: the pod may be ninety seconds into a 130-second
      read, and sending the page again doubles the work on that card.
    * An active cooldown is no longer extended. It used to be, silently: every
      further refusal pushed the expiry out while only the first one printed, so
      a five-minute window could last twenty with nothing in the log.
    
    Two more ways to end up local, both of which were SILENT:
    
    * The gate compared the request's engine against the three VLM engine names.
      /v1/ocr has always also accepted the MODEL a caller knows in place of the
      engine driving it — resolve_engine maps the Surya-2 checkpoint to 'surya'
      and the catalogue advertises those ids as aliases — so a caller naming the
      model got no pod-side pipeline and no message. The switch has to be read
      the way a READ reads it, for the same reason it has to be read the way a
      LAUNCH reads it.
    * Serve mode not 'model', an empty model id, an engine with no pod pipeline,
      a standing refusal: each returned "" or None without a word. The log of the
      run that prompted this holds 8,278 successful pod reads and nothing at all
      explaining the thousands of documents in between. Every one of them now
      says which way the decision went, rate limited rather than said once,
      because a cooldown is minutes and a run is hours.
    
    And, since reading a page here means rasterising here, ocr.pod_pipeline:
    
      prefer  (default) wait up to ocr.pod_pipeline_wait_s for a pod, then read
              it here — a pool with nothing free is usually one still booting.
              A timed-out wait is not paid again for five minutes, so the ceiling
              cannot become a per-page tax.
      auto    read it here at once (0.4.1-0.4.5 behaviour).
      require never rasterise on this box: 503, saying why.
    
    Tests: 32 new (tests/test_ocr_pod_pipeline_policy.py) covering the alias, all
    four silent routes, the three policies and the transport/answer line.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    a6fc0266
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...