• Stefy Lanza (nextime / spora )'s avatar
    ten connections is a browser's number, not a fan-out's (v0.3.101) · a90b93a2
    Stefy Lanza (nextime / spora ) authored
    One shared session serves every pod, and `requests`' stock adapter keeps ten
    connections per host. `ocr.max_concurrency` defaults to 64 and is 128 on the
    install this came from, so from the ELEVENTH concurrent page onward urllib3
    opened a connection, used it, found the pool full on release and threw it
    away:
    
        urllib3.connectionpool: Connection pool is full, discarding connection:
        194.68.245.54. Connection pool size: 10
    
    74 of those in five minutes — about fourteen a minute against one pod — while
    that run's health checks were timing out. Every discarded connection means the
    next page pays a full TLS handshake to a machine in another country: several
    round trips of latency each, and handshake CPU on the orchestrator at precisely
    the moment it is busiest. It does not CAUSE a slow run, but it multiplies one,
    and it is the part that is ours.
    
    The pool is now sized from the concurrency the install is configured for:
    `ocr.max_concurrency`, floored at 64 and capped at 256. Read at session
    creation rather than at import, because the config is not loaded when this
    module is imported — and because an operator raising max_concurrency should not
    have to know a connection pool exists. An idle connection costs a socket and
    nothing else, which is the cheap side of this trade.
    
    The adapter also carries the floor as its own default, so a `_PinnedAdapter`
    built anywhere cannot silently fall back to ten.
    
    Routing is deliberately unchanged: only https-to-a-bare-IP uses the pinned
    session, everything else still goes through `requests` untouched, so callers
    and tests that patch `requests.get` keep working.
    
    Not a fix for the slowness that exposed it. That host was running MariaDB at
    534% CPU, four Postgres parallel workers, and a backup that had been streaming
    over SFTP for three hours, against coderai's own 8.8% of one core out of
    sixteen — the orchestrator was waiting for a CPU, not using one.
    
    9 tests. Suite: 3151 passed, the same 13 known-stale failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    a90b93a2
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...