• Stefy Lanza (nextime / spora )'s avatar
    ten connections is a browser's number, not a fan-out's (v0.3.101) · a90b93a2
    Stefy Lanza (nextime / spora ) authored
    One shared session serves every pod, and `requests`' stock adapter keeps ten
    connections per host. `ocr.max_concurrency` defaults to 64 and is 128 on the
    install this came from, so from the ELEVENTH concurrent page onward urllib3
    opened a connection, used it, found the pool full on release and threw it
    away:
    
        urllib3.connectionpool: Connection pool is full, discarding connection:
        194.68.245.54. Connection pool size: 10
    
    74 of those in five minutes — about fourteen a minute against one pod — while
    that run's health checks were timing out. Every discarded connection means the
    next page pays a full TLS handshake to a machine in another country: several
    round trips of latency each, and handshake CPU on the orchestrator at precisely
    the moment it is busiest. It does not CAUSE a slow run, but it multiplies one,
    and it is the part that is ours.
    
    The pool is now sized from the concurrency the install is configured for:
    `ocr.max_concurrency`, floored at 64 and capped at 256. Read at session
    creation rather than at import, because the config is not loaded when this
    module is imported — and because an operator raising max_concurrency should not
    have to know a connection pool exists. An idle connection costs a socket and
    nothing else, which is the cheap side of this trade.
    
    The adapter also carries the floor as its own default, so a `_PinnedAdapter`
    built anywhere cannot silently fall back to ten.
    
    Routing is deliberately unchanged: only https-to-a-bare-IP uses the pinned
    session, everything else still goes through `requests` untouched, so callers
    and tests that patch `requests.get` keep working.
    
    Not a fix for the slowness that exposed it. That host was running MariaDB at
    534% CPU, four Postgres parallel workers, and a backup that had been streaming
    over SFTP for three hours, against coderai's own 8.8% of one core out of
    sixteen — the orchestrator was waiting for a CPU, not using one.
    
    9 tests. Suite: 3151 passed, the same 13 known-stale failures.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
    a90b93a2
__init__.py 4.21 KB