• Stefy Lanza (nextime / spora )'s avatar
    a page forward is a wait on a socket, not a thread (v0.4.9) · b044ed11
    Stefy Lanza (nextime / spora ) authored
    0.4.8 moved the forward off the default 32-thread pool and the measured result
    on the live orchestrator was 33 -> 483 pages in flight, 1.65-2.2 -> 7.99
    pages/s, and a control-plane request that used to time out at 40 s answering in
    12 ms. But a thread is still the wrong unit for the wait: ~60 s per page
    against a real corpus means 15 documents/second at ~3.5 pages each is ~3,200
    outstanding forwards, against a pod-pool ceiling of 2048. Threads cannot get
    there, and should not try.
    
    So the hot path is a coroutine. `pod_http` gains `apost`/`aget` on httpx --
    already in the stack (codai/cluster/nodes.py, codai/main.py), so no new
    dependency -- with everything the sync path learned kept intact:
    
    * the pinned TLS context, unchanged: a direct-TCP pod serves HTTPS with a
      certificate signed by THIS install's CA at an ip the certificate cannot name,
      so the CA is trusted and the hostname check is off. NOT verify=False, which
      would accept anybody;
    * one client per event loop AND per pinned-ness, because a client belongs to
      the loop its sockets were opened on and the two trust settings must never
      share one;
    * connection limits from the same `_pool_size()` the sync session uses;
    * a proxied pod (http://, or https:// to a name) stays an ordinary request.
    
    `_apost_page_to_pod` and `_apod_still_alive` mirror the sync twins, and both
    twins now share ONE request builder and ONE response reader -- two copies of
    that field list is how an engine name, a dpi or a token budget comes to apply
    on one transport and not the other. Every distinction 0.4.2-0.4.6 established
    is preserved and tested on the new path: an ANSWER (even 503) is a verdict
    about the pod and belongs on the refusal path, nothing-came-back is
    PodUnreachable, a dead pooled socket is retried on a fresh one, and a timeout
    is never retried -- resending a page the pod is still working on is a second
    card doing the same work and a double bill.
    
    `ocr.pod_http_async` (default on, admin API + settings control) puts the
    forward back on the pod thread pool. That path is kept working deliberately,
    not as dead code: this is a new transport on the hottest path in the
    subsystem, and an operator who hits something odd should change one setting
    rather than wait for a release.
    
    Four existing test files had to be repaired, and the reason is worth keeping:
    they stubbed only the SYNC twin, so the moment the async default landed they
    stopped testing their own stubs and began dialling 10.0.0.1 for real -- 300 s
    hangs where there should have been failures. Their helpers now stub both twins
    via `_stub_page_post`, and the policy tests say `pod_http_async: False` out
    loud because they stub `pod_http.post` specifically. A test that quietly
    measures something else is worse than one that breaks, and this change
    produced four of them.
    
    21 new transport tests. 3345 passed, 13 pre-existing failures (broker, studio,
    manual client) untouched.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    b044ed11
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...