• Stefy Lanza (nextime / spora )'s avatar
    Stop upstream 429s from disabling a codex provider · f9d15eee
    Stefy Lanza (nextime / spora ) authored
    A 429 from the ChatGPT backend is per-quota-bucket, not a provider
    fault: x-codex-active-limit names the bucket that refused, and the
    account keeps serving from another one meanwhile. In one hour of
    production traffic codex_think returned 1888 x 200 interleaved with
    1282 x 429. Every 429 was recorded as a provider failure, so three in a
    row tripped the three-strikes cooldown and aisbf spent five minutes at a
    time rejecting requests itself -- including the majority the upstream
    would have answered. One client fired 32 requests during a cooldown and
    got 32 x 503 without a single one reaching OpenAI.
    
    - codex: raise RateLimitError on a 429 instead of raise_for_status(),
      without disabling the provider, on both the streaming and
      non-streaming OAuth paths.
    - handlers: _should_record_failure() now excludes upstream rate limits
      as well as CoderAI warm-up, and a quota refusal is forwarded to the
      client as 429 rather than a generic 500.
    
    Send ChatGPT-Account-ID again. The header was only set from
    tokens.account_id, which is null in every credentials file the login
    flow writes; the real value is in the id_token's chatgpt_account_id
    claim. Without it the backend picks a workspace itself, so an account
    belonging to several workspaces cannot be steered at the configured one.
    
    Make rate_limit actually do something. The spacing timestamp lived on
    the handler, but get_provider_handler() builds a fresh handler per
    request, so it was always 0 on arrival and no wait was ever applied --
    a configured rate_limit was silently inert. Move the timestamps to a
    process-wide registry guarded by a per-slot lock, without which N
    concurrent requests all read the same stale timestamp and burst
    together. Verified: 4 concurrent requests at 0.5s spacing now take
    1.50s, previously 0.00s.
    
    Bump version to 0.99.90.
    Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
    f9d15eee
setup.py 16.4 KB