• Stefy Lanza (nextime / spora )'s avatar
    Per-provider client rate limit, dashboard-editable (v0.99.95) · 502c7e74
    Stefy Lanza (nextime / spora ) authored
    Adds a configurable global cap (all clients combined) on requests forwarded to a
    given provider, so a hammering client can be throttled per provider — e.g. cap
    kilo-spora so the real-estate app can't exhaust kilo's per-IP free-model quota
    (which was cascading into 429s for other kilo providers like kilo-stefy and
    breaking the lisa rotation).
    
    - config.py: ProviderConfig gains client_rate_limit_rpm / client_rate_limit_rph
      (0 = unlimited). Persists via the existing provider-save path.
    - handlers.py: a sliding-window limiter keyed provider:<id>:<window>, enforced at
      all six client entry points (chat, streaming, transcription, TTS, image,
      embeddings) right before the availability gate — excess requests get HTTP 429 +
      Retry-After without building the handler or calling upstream.
    - providers.html / user_providers.html: a "Rate Limiting" section with per-minute
      and per-hour inputs (0 = unlimited) in each provider's edit form.
    
    Counts direct requests to a provider; rotation-internal dispatch is not counted
    in this version.
    Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
    502c7e74
Name
Last commit
Last update
aisbf Loading commit data...
config Loading commit data...
docs Loading commit data...
static Loading commit data...
templates Loading commit data...
tests Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
API_EXAMPLES.md Loading commit data...
BUILD_DEPLOY.md Loading commit data...
CHANGELOG.md Loading commit data...
DEBUG_GUIDE.md Loading commit data...
DOCUMENTATION.md Loading commit data...
ENDPOINTS.md Loading commit data...
KILO_OAUTH2.md Loading commit data...
LICENSE.txt Loading commit data...
MANIFEST.in Loading commit data...
MISSING_ENDPOINTS.md Loading commit data...
PYPI.md Loading commit data...
QUICK_START_PAYMENT.md Loading commit data...
README.md Loading commit data...
RUNPOD_IMPLEMENTATION_PLAN.md Loading commit data...
SUBSCRIPTION_PRICING_FEATURE.md Loading commit data...
TODO.md Loading commit data...
TORRC_SETUP.md Loading commit data...
aisbf.sh Loading commit data...
build.sh Loading commit data...
chatgpt-api-implementation-guide.md Loading commit data...
clean.sh Loading commit data...
cli.py Loading commit data...
codex-oauth-implementation-guide.md Loading commit data...
main.py Loading commit data...
main.py.bak Loading commit data...
minio-archive-plan.md Loading commit data...
pyproject.toml Loading commit data...
requirements.txt Loading commit data...
screenshot.png Loading commit data...
setup.py Loading commit data...
start_proxy.sh Loading commit data...
verify_release.sh Loading commit data...