• Stefy Lanza (nextime / spora )'s avatar
    Resolve max_tokens per-model + defaults across rotations and autoselect; bump to 0.99.82 · 5dff11f4
    Stefy Lanza (nextime / spora ) authored
    Extend max output token resolution so per-model and default values are
    honored at every layer when the client omits max_tokens.
    
    Effective priority (client value always wins if present):
      autoselect per-model -> rotation per-model -> provider per-model
      -> provider default -> rotation default -> autoselect default
    
    - rotation path: consult the selected provider's config (new
      RotationHandler._get_provider_config) so a provider default_max_tokens
      applies even though rotations can't configure max_tokens themselves
    - AutoselectModelInfo.max_tokens: new per-model override field
    - AutoselectHandler._apply_autoselect_max_tokens applied in both the
      streaming and non-streaming dispatch: per-model override set directly
      (highest), autoselect default threaded via _autoselect_default_max_tokens
      as a lowest-priority fallback
    - rotation and both RequestHandler injections consume the threaded
      autoselect default, covering autoselect->rotation and
      autoselect->provider/model dispatch routes
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    5dff11f4
Name
Last commit
Last update
aisbf Loading commit data...
config Loading commit data...
docs Loading commit data...
static Loading commit data...
templates Loading commit data...
tests Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
API_EXAMPLES.md Loading commit data...
BUILD_DEPLOY.md Loading commit data...
CHANGELOG.md Loading commit data...
DEBUG_GUIDE.md Loading commit data...
DOCUMENTATION.md Loading commit data...
ENDPOINTS.md Loading commit data...
KILO_OAUTH2.md Loading commit data...
LICENSE.txt Loading commit data...
MANIFEST.in Loading commit data...
MISSING_ENDPOINTS.md Loading commit data...
PYPI.md Loading commit data...
QUICK_START_PAYMENT.md Loading commit data...
README.md Loading commit data...
RUNPOD_IMPLEMENTATION_PLAN.md Loading commit data...
SUBSCRIPTION_PRICING_FEATURE.md Loading commit data...
TODO.md Loading commit data...
TORRC_SETUP.md Loading commit data...
aisbf.sh Loading commit data...
build.sh Loading commit data...
chatgpt-api-implementation-guide.md Loading commit data...
clean.sh Loading commit data...
cli.py Loading commit data...
codex-oauth-implementation-guide.md Loading commit data...
main.py Loading commit data...
main.py.bak Loading commit data...
minio-archive-plan.md Loading commit data...
pyproject.toml Loading commit data...
requirements.txt Loading commit data...
screenshot.png Loading commit data...
setup.py Loading commit data...
start_proxy.sh Loading commit data...
verify_release.sh Loading commit data...