Transparent rotation failover for streaming provider errors; bump to 0.99.70
Streaming rotation requests returned the provider's lazy async generator to
the client before the upstream HTTP call was made, so an immediate provider
error (e.g. HTTP 400 "model not supported") surfaced mid-stream after the
200 OK headers were already sent — leaking the error to the client instead
of failing over.
- Prime streaming responses (_prime_stream) inside the rotation retry loop so
immediate provider errors are raised within the try/except and trigger
failover to the next provider by weight, transparently to the client.
- Add _is_non_retryable_error to skip pointless same-provider retries for
client errors (400/401/403/404/405/422) and move straight to the next
provider; 5xx/timeouts/network remain retryable.
Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com>
Showing
This diff is collapsed.
No preview for this file type
Please
register
or
sign in
to comment