• Stefy Lanza (nextime / spora )'s avatar
    Don't let request-scoped errors trip the provider failure cooldown · fae38ea0
    Stefy Lanza (nextime / spora ) authored
    A model that intermittently lacks an image endpoint on kilo returns 404
    "No endpoints found that support image input". The kilo handler counted every
    exception (that 404, a 400 for an undecodable image, a 429) toward the
    three-strike cooldown, so a few request-scoped errors disabled kilo-spora for
    ALL traffic for 5 minutes — during which a high-rate client piled up thousands
    of 503 "in failure cooldown" rejections.
    
    Add a shared classifier in providers/base.py: is_request_scoped_error /
    is_request_scoped_status / extract_http_status treat 400/404/413/415/422/429
    (and the wrapped streaming-error message form) as request-scoped — about the
    specific request, not provider health. Genuine faults (5xx, timeouts, network)
    still count.
    
    Wire it into both places that record failures:
    - KiloProviderHandler.handle_request and _handle_streaming_request skip
      record_failure() for request-scoped errors (429 still routes through
      handle_429_error for adaptive backoff).
    - handlers._should_record_failure excludes them for every provider.
    
    The error still propagates to the caller; it just no longer disables the whole
    provider. Only a genuinely unhealthy provider now enters the cooldown.
    Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
    fae38ea0
Name
Last commit
Last update
aisbf Loading commit data...
config Loading commit data...
docs Loading commit data...
static Loading commit data...
templates Loading commit data...
tests Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
API_EXAMPLES.md Loading commit data...
BUILD_DEPLOY.md Loading commit data...
CHANGELOG.md Loading commit data...
DEBUG_GUIDE.md Loading commit data...
DOCUMENTATION.md Loading commit data...
ENDPOINTS.md Loading commit data...
KILO_OAUTH2.md Loading commit data...
LICENSE.txt Loading commit data...
MANIFEST.in Loading commit data...
MISSING_ENDPOINTS.md Loading commit data...
PYPI.md Loading commit data...
QUICK_START_PAYMENT.md Loading commit data...
README.md Loading commit data...
RUNPOD_IMPLEMENTATION_PLAN.md Loading commit data...
SUBSCRIPTION_PRICING_FEATURE.md Loading commit data...
TODO.md Loading commit data...
TORRC_SETUP.md Loading commit data...
aisbf.sh Loading commit data...
build.sh Loading commit data...
chatgpt-api-implementation-guide.md Loading commit data...
clean.sh Loading commit data...
cli.py Loading commit data...
codex-oauth-implementation-guide.md Loading commit data...
main.py Loading commit data...
main.py.bak Loading commit data...
minio-archive-plan.md Loading commit data...
pyproject.toml Loading commit data...
requirements.txt Loading commit data...
screenshot.png Loading commit data...
setup.py Loading commit data...
start_proxy.sh Loading commit data...
verify_release.sh Loading commit data...