-
Stefy Lanza (nextime / spora ) authored
0.4.8 moved the forward off the default 32-thread pool and the measured result on the live orchestrator was 33 -> 483 pages in flight, 1.65-2.2 -> 7.99 pages/s, and a control-plane request that used to time out at 40 s answering in 12 ms. But a thread is still the wrong unit for the wait: ~60 s per page against a real corpus means 15 documents/second at ~3.5 pages each is ~3,200 outstanding forwards, against a pod-pool ceiling of 2048. Threads cannot get there, and should not try. So the hot path is a coroutine. `pod_http` gains `apost`/`aget` on httpx -- already in the stack (codai/cluster/nodes.py, codai/main.py), so no new dependency -- with everything the sync path learned kept intact: * the pinned TLS context, unchanged: a direct-TCP pod serves HTTPS with a certificate signed by THIS install's CA at an ip the certificate cannot name, so the CA is trusted and the hostname check is off. NOT verify=False, which would accept anybody; * one client per event loop AND per pinned-ness, because a client belongs to the loop its sockets were opened on and the two trust settings must never share one; * connection limits from the same `_pool_size()` the sync session uses; * a proxied pod (http://, or https:// to a name) stays an ordinary request. `_apost_page_to_pod` and `_apod_still_alive` mirror the sync twins, and both twins now share ONE request builder and ONE response reader -- two copies of that field list is how an engine name, a dpi or a token budget comes to apply on one transport and not the other. Every distinction 0.4.2-0.4.6 established is preserved and tested on the new path: an ANSWER (even 503) is a verdict about the pod and belongs on the refusal path, nothing-came-back is PodUnreachable, a dead pooled socket is retried on a fresh one, and a timeout is never retried -- resending a page the pod is still working on is a second card doing the same work and a double bill. `ocr.pod_http_async` (default on, admin API + settings control) puts the forward back on the pod thread pool. That path is kept working deliberately, not as dead code: this is a new transport on the hottest path in the subsystem, and an operator who hits something odd should change one setting rather than wait for a release. Four existing test files had to be repaired, and the reason is worth keeping: they stubbed only the SYNC twin, so the moment the async default landed they stopped testing their own stubs and began dialling 10.0.0.1 for real -- 300 s hangs where there should have been failures. Their helpers now stub both twins via `_stub_page_post`, and the policy tests say `pod_http_async: False` out loud because they stub `pod_http.post` specifically. A test that quietly measures something else is worse than one that breaks, and this change produced four of them. 21 new transport tests. 3345 passed, 13 pre-existing failures (broker, studio, manual client) untouched. Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
b044ed11