• Stefy Lanza (nextime / spora )'s avatar
    Per-provider client rate limit, dashboard-editable (v0.99.95) · 502c7e74
    Stefy Lanza (nextime / spora ) authored
    Adds a configurable global cap (all clients combined) on requests forwarded to a
    given provider, so a hammering client can be throttled per provider — e.g. cap
    kilo-spora so the real-estate app can't exhaust kilo's per-IP free-model quota
    (which was cascading into 429s for other kilo providers like kilo-stefy and
    breaking the lisa rotation).
    
    - config.py: ProviderConfig gains client_rate_limit_rpm / client_rate_limit_rph
      (0 = unlimited). Persists via the existing provider-save path.
    - handlers.py: a sliding-window limiter keyed provider:<id>:<window>, enforced at
      all six client entry points (chat, streaming, transcription, TTS, image,
      embeddings) right before the availability gate — excess requests get HTTP 429 +
      Retry-After without building the handler or calling upstream.
    - providers.html / user_providers.html: a "Rate Limiting" section with per-minute
      and per-hour inputs (0 = unlimited) in each provider's edit form.
    
    Counts direct requests to a provider; rotation-internal dispatch is not counted
    in this version.
    Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
    502c7e74
setup.py 16.4 KB