Transparent rotation failover for streaming provider errors; bump to 0.99.70

Streaming rotation requests returned the provider's lazy async generator to
the client before the upstream HTTP call was made, so an immediate provider
error (e.g. HTTP 400 "model not supported") surfaced mid-stream after the
200 OK headers were already sent — leaking the error to the client instead
of failing over.

- Prime streaming responses (_prime_stream) inside the rotation retry loop so
  immediate provider errors are raised within the try/except and trigger
  failover to the next provider by weight, transparently to the client.
- Add _is_non_retryable_error to skip pointless same-provider retries for
  client errors (400/401/403/404/405/422) and move straight to the next
  provider; 5xx/timeouts/network remain retryable.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
parent bd7bfb97
...@@ -55,7 +55,7 @@ from .auth.qwen import QwenOAuth2 ...@@ -55,7 +55,7 @@ from .auth.qwen import QwenOAuth2
from .handlers import RequestHandler, RotationHandler, AutoselectHandler from .handlers import RequestHandler, RotationHandler, AutoselectHandler
from .utils import count_messages_tokens, split_messages_into_chunks, get_max_request_tokens_for_model from .utils import count_messages_tokens, split_messages_into_chunks, get_max_request_tokens_for_model
__version__ = "0.99.69" __version__ = "0.99.70"
__all__ = [ __all__ = [
# Config # Config
"config", "config",
......
This diff is collapsed.
...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" ...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "aisbf" name = "aisbf"
version = "0.99.69" version = "0.99.70"
description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations" description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations"
readme = "README.md" readme = "README.md"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"
......
...@@ -106,7 +106,7 @@ class InstallCommand(_install): ...@@ -106,7 +106,7 @@ class InstallCommand(_install):
setup( setup(
name="aisbf", name="aisbf",
version="0.99.69", version="0.99.70",
author="AISBF Contributors", author="AISBF Contributors",
author_email="stefy@nexlab.net", author_email="stefy@nexlab.net",
description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations", description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment