• Stefy Lanza (nextime / spora )'s avatar
    a page forward is a wait on a socket, not a thread (v0.4.9) · b044ed11
    Stefy Lanza (nextime / spora ) authored
    0.4.8 moved the forward off the default 32-thread pool and the measured result
    on the live orchestrator was 33 -> 483 pages in flight, 1.65-2.2 -> 7.99
    pages/s, and a control-plane request that used to time out at 40 s answering in
    12 ms. But a thread is still the wrong unit for the wait: ~60 s per page
    against a real corpus means 15 documents/second at ~3.5 pages each is ~3,200
    outstanding forwards, against a pod-pool ceiling of 2048. Threads cannot get
    there, and should not try.
    
    So the hot path is a coroutine. `pod_http` gains `apost`/`aget` on httpx --
    already in the stack (codai/cluster/nodes.py, codai/main.py), so no new
    dependency -- with everything the sync path learned kept intact:
    
    * the pinned TLS context, unchanged: a direct-TCP pod serves HTTPS with a
      certificate signed by THIS install's CA at an ip the certificate cannot name,
      so the CA is trusted and the hostname check is off. NOT verify=False, which
      would accept anybody;
    * one client per event loop AND per pinned-ness, because a client belongs to
      the loop its sockets were opened on and the two trust settings must never
      share one;
    * connection limits from the same `_pool_size()` the sync session uses;
    * a proxied pod (http://, or https:// to a name) stays an ordinary request.
    
    `_apost_page_to_pod` and `_apod_still_alive` mirror the sync twins, and both
    twins now share ONE request builder and ONE response reader -- two copies of
    that field list is how an engine name, a dpi or a token budget comes to apply
    on one transport and not the other. Every distinction 0.4.2-0.4.6 established
    is preserved and tested on the new path: an ANSWER (even 503) is a verdict
    about the pod and belongs on the refusal path, nothing-came-back is
    PodUnreachable, a dead pooled socket is retried on a fresh one, and a timeout
    is never retried -- resending a page the pod is still working on is a second
    card doing the same work and a double bill.
    
    `ocr.pod_http_async` (default on, admin API + settings control) puts the
    forward back on the pod thread pool. That path is kept working deliberately,
    not as dead code: this is a new transport on the hottest path in the
    subsystem, and an operator who hits something odd should change one setting
    rather than wait for a release.
    
    Four existing test files had to be repaired, and the reason is worth keeping:
    they stubbed only the SYNC twin, so the moment the async default landed they
    stopped testing their own stubs and began dialling 10.0.0.1 for real -- 300 s
    hangs where there should have been failures. Their helpers now stub both twins
    via `_stub_page_post`, and the policy tests say `pod_http_async: False` out
    loud because they stub `pod_http.post` specifically. A test that quietly
    measures something else is worse than one that breaks, and this change
    produced four of them.
    
    21 new transport tests. 3345 passed, 13 pre-existing failures (broker, studio,
    manual client) untouched.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    b044ed11
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
audiocraft_worker.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
chunking.py Loading commit data...
colibri_worker.py Loading commit data...
compose.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
host_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lipsync.py Loading commit data...
log.py Loading commit data...
longcat_worker.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
model_test.py Loading commit data...
models_transfer.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
pod_http.py Loading commit data...
pod_tls.py Loading commit data...
podlog.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
thread_pool.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
uploads.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...
xtts_worker.py Loading commit data...