- 26 Jul, 2026 13 commits
-
-
Stefy Lanza (nextime / spora ) authored
The dispatch loop skipped a provider via is_rate_limited() (which includes the manual dashboard flag) but did NOT add it to tried_models, so the next retry re-selected the SAME highest-weight provider every time. A manually-flagged / rate-limited top-weight member (e.g. claude_think in rotation lisa) thus consumed all max_retries and returned 429, never failing over to a working member (kilo-stefy). - Use _provider_in_availability_cooldown() instead of is_rate_limited() so the manual flag (rotation-membership only) does not block dispatch, consistent with the scan. - Mark the skipped model as tried so failover actually advances to the next provider. - Guard the dispatch handler build in try/except too (same bad-cred 500 class). rotation/lisa now returns 200 via kilo-stefy instead of 429/500. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
The per-rotation scan called get_provider_handler() unguarded. A rotation member with invalid/expired credentials (e.g. codex) raises ValueError during credential validation, which propagated uncaught and returned HTTP 500 for the whole rotation (hermes bot hitting rotation/lisa). Wrap the build in try/except and skip the provider on failure, so the rotation falls through to a working member. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
/dashboard/api/provider/{id}/usage returned HTTP 500 whenever a usage fetch failed — most commonly when a provider's OAuth credentials are expired/invalid (e.g. the codex_* providers). The Providers dashboard polls every provider's usage on a timer, so this spammed 500s (visible in nginx/access logs) for any stale-credential provider. A monitoring poll failing is an expected soft condition, not a server error. Return 200 with usage:None and an 'unavailable' flag so the UI can render an offline state; genuine data still returns normally. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Extends the per-provider limit with a per-second cap (client_rate_limit_rps) alongside the existing per-minute/per-hour. A per-second bound is the most effective defense against sub-second retry storms (the shape of the estate app's hammering). Enforcement now checks 1s / 60s / 3600s windows; dashboard provider forms gain a "per second" input. 0 = unlimited on every tier. kilo-spora configured to rps=2, rph=200 to cap its load on kilo's per-IP free quota (the cascade that was 429-ing kilo-stefy / breaking the lisa rotation). Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Adds a configurable global cap (all clients combined) on requests forwarded to a given provider, so a hammering client can be throttled per provider — e.g. cap kilo-spora so the real-estate app can't exhaust kilo's per-IP free-model quota (which was cascading into 429s for other kilo providers like kilo-stefy and breaking the lisa rotation). - config.py: ProviderConfig gains client_rate_limit_rpm / client_rate_limit_rph (0 = unlimited). Persists via the existing provider-save path. - handlers.py: a sliding-window limiter keyed provider:<id>:<window>, enforced at all six client entry points (chat, streaming, transcription, TTS, image, embeddings) right before the availability gate — excess requests get HTTP 429 + Retry-After without building the handler or calling upstream. - providers.html / user_providers.html: a "Rate Limiting" section with per-minute and per-hour inputs (0 = unlimited) in each provider's edit form. Counts direct requests to a provider; rotation-internal dispatch is not counted in this version. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Under --debug, httpcore emits a connection-close DEBUG trace from the OpenAI client's __del__ finalizer. Under load, that finalizer runs during a GC pause that can land mid-flush of aisbf.log, re-entering the same RotatingFileHandler and raising "RuntimeError: reentrant call inside BufferedWriter". This cascades and blocks the asyncio event loop: uvicorn keeps the port bound (LISTEN, Recv-Q backing up) but stops accepting connections, so the web interface hangs. Pin httpcore/httpx/openai to WARNING regardless of debug mode so a finalizer can never emit a DEBUG/INFO record into the reentrant path. Also removes a large share of the log volume. aisbf's own --debug logging is unaffected. Verified in production: 0 reentrant errors and 0 httpcore DEBUG lines under live load after restart; server stable and responsive. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Ships the accumulated fixes to production: media-aware response cache key, manual-disable is rotation-only (direct calls always allowed), per-rotation provider disable via enabled:false, and request-scoped errors (404 no-vision- endpoint / 400 bad-image / 429) no longer tripping the provider failure cooldown. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
- 25 Jul, 2026 2 commits
-
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
A model that intermittently lacks an image endpoint on kilo returns 404 "No endpoints found that support image input". The kilo handler counted every exception (that 404, a 400 for an undecodable image, a 429) toward the three-strike cooldown, so a few request-scoped errors disabled kilo-spora for ALL traffic for 5 minutes — during which a high-rate client piled up thousands of 503 "in failure cooldown" rejections. Add a shared classifier in providers/base.py: is_request_scoped_error / is_request_scoped_status / extract_http_status treat 400/404/413/415/422/429 (and the wrapped streaming-error message form) as request-scoped — about the specific request, not provider health. Genuine faults (5xx, timeouts, network) still count. Wire it into both places that record failures: - KiloProviderHandler.handle_request and _handle_streaming_request skip record_failure() for request-scoped errors (429 still routes through handle_429_error for adaptive backoff). - handlers._should_record_failure excludes them for every provider. The error still propagates to the caller; it just no longer disables the whole provider. Only a genuinely unhealthy provider now enters the cooldown. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
- 24 Jul, 2026 4 commits
-
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Previously the Disable button on a rotation flipped a single global provider_manual_disabled flag: disabling a provider in one rotation removed it from every rotation, from autoselect, and (before this series) from direct API calls too. That is not what "disable in this rotation" should mean. Disabling is now membership in a single rotation, expressed as `enabled: false` on that provider's entry in the rotation config. handlers.py: the rotation scan no longer consults the global manual flag. It skips an entry when `provider.get('enabled') is False` (cheap, before building the handler), and otherwise skips only on a genuine auto-disable cooldown (failure threshold / usage limit) via the new _provider_in_availability_cooldown helper. is_provider_disabled_cheap is no longer used here (import dropped). Autoselect inherits this since it dispatches through handle_rotation_request. rotations.html / user_rotations.html: the per-provider toggle flips `enabled` on that rotation entry and persists just that rotation (full-config POST for admin, single-rotation POST for DB users), updating only that entry's button instead of every rotation sharing the provider. The global manual-status fetch is gone. The legacy global manual-disable endpoints remain but are now inert for rotation and direct-call gating. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Two independent fixes surfaced while diagnosing kilo-spora vision requests returning 503 and (once enabled) risking stale image answers. handlers.py: _raise_if_provider_unavailable no longer rejects a direct, provider-named request just because the provider is manually disabled. The dashboard disable toggle is meant to remove a provider from rotation/autoselect (those scans use is_provider_disabled_cheap) — it must not take the provider offline across the whole API. Direct calls still 503 on genuine unavailability (failure cooldown or usage-limit cooldown). cache.py: _generate_cache_key hashed only the text parts of multimodal messages, dropping image_url/video_url/audio_url. A caller that sends a fixed prompt with a varying image (e.g. the real-estate OUTDOOR/INDOOR classifier) collapsed every image to one cache key, so the first image's answer was served for all subsequent images. Fold each media reference into the message hash so different attachments produce different keys. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
- 22 Jul, 2026 6 commits
-
-
Stefy Lanza (nextime / spora ) authored
Synthesize a valid SSE stream when a dict response hits the streaming path.
-
Stefy Lanza (nextime / spora ) authored
The chunked-request handler always returns a complete chat.completion dict ("Streaming is not supported for chunked rotation requests"), but stream=true callers still wrap it in the SSE stream generator, which iterated the dict and emitted its KEYS as string chunks (data: "id", data: "choices", ...). Clients crashed with "'str' object has no attribute 'choices'". Previously masked: chunked requests died earlier on the null-content TypeError, so this path was unreachable. When the streaming wrapper receives a plain dict, emit one well-formed chat.completion.chunk instead - delta carrying content and/or tool_calls (with the per-entry index streaming requires), the real finish_reason, usage attached - followed by data: [DONE]. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Stop chunked requests crashing on tool-call responses with null content.
-
Stefy Lanza (nextime / spora ) authored
Tool-call completions carry content=None (the key exists, so a .get default never applies). Both chunked request paths concatenated it directly ("" += None -> TypeError), recording a provider failure on every attempt even though upstream returned 200 - rotation 'lisa' exhausted codex_think and returned 429 despite available usage. Coalesce null content to '' and pass tool_calls through to the combined response with finish_reason "tool_calls" (previously they were silently discarded and agents received an empty completion). Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Stop event-loop blocking that trips broker WebSocket keepalive timeouts.
-
Stefy Lanza (nextime / spora ) authored
Under load the CoderAI broker WebSocket was being torn down with "1011 keepalive ping timeout" and reconnecting (surfaced as 502s by the fronting proxy). Root cause: synchronous work run directly on the asyncio event loop starves the WebSocket keepalive so PONGs miss the 20s window. Remove the blockers: - analytics: dispatch the blocking token_usage / per-user INSERTs to a worker thread (fire-and-forget when a loop is running, inline otherwise) instead of calling them inline from async request handlers. - database: add a MySQL connection pool (ping+reconnect, direct-connect fallback) and return connections to the pool on with-block exit instead of opening a fresh TCP+auth connection per query and leaking it to GC. - utils: cache the tiktoken encoding and memoize count_messages_tokens by content, collapsing the several CPU-bound BPE passes per request into one. - coderai_broker: run touch()/publish_response Redis I/O off the loop via asyncio.to_thread (awaited to preserve chunk order), matching the existing consume_request pattern. - routes/coderai_broker: answer heartbeats immediately and process response frames on a dedicated FIFO worker so a slow publish can't delay the next heartbeat. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
- 21 Jul, 2026 7 commits
-
-
Stefy Lanza (nextime / spora ) authored
In OAuth mode _get_valid_api_key() notices the configured endpoint is not the ChatGPT backend, corrects base_url in memory and calls _update_provider_endpoint() to write the correction back. For a global provider that write called config.save_providers() -- a method that does not exist. The AttributeError was swallowed by the surrounding except and logged as a warning, so the correction only ever lived in memory: providers.json kept the wrong endpoint and every restart re-ran the same dance. codex_think sat like that, configured for api.openai.com/v1 while every request went to chatgpt.com. Rewrite providers.json in place instead, merging into the on-disk document rather than serialising the in-memory config, so a concurrent dashboard edit is not clobbered by stale state. The write goes through a temp file and os.replace() so a crash mid-write cannot truncate the config. Missing provider is now logged explicitly rather than silently creating an entry, and the except logs a traceback so the next failure of this kind is not invisible. Bump version to 0.99.92. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
openai_think is a codex provider in API-key mode, which posts to /v1/chat/completions. That endpoint rejects the request outright when a reasoning model is given function tools: 400 "Function tools with reasoning_effort are not supported for gpt-5.6-sol in /v1/chat/completions. To use function tools, use /v1/responses or set reasoning_effort to 'none'." aisbf never sends reasoning_effort -- the model applies it server-side -- so there is nothing to strip from the payload; the endpoint is the problem. Setting it to 'none' would work but silently disables reasoning on a provider chosen for it, so route these requests to /v1/responses instead, which supports tools and reasoning together. OAuth mode already spoke this protocol against the ChatGPT backend, so _handle_request_oauth2_mode is now a thin wrapper over a shared _handle_request_responses_api(); only the URL and auth headers differ between the two. The flattened tool format the codex converter already produces is what /v1/responses expects. Fix double-counted failures. codex's handle_request recorded a failure and re-raised, and the caller in handlers.py recorded it again, so one failed request counted twice and the three-strikes cooldown tripped after two requests instead of three. The handler layer records for every provider, so the provider-level call is the redundant one. Note the same pattern exists in google/qwen/kilo/openai/ollama/coderai and is left alone here -- fixed only where it was reproduced. Bump version to 0.99.91. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
A 429 from the ChatGPT backend is per-quota-bucket, not a provider fault: x-codex-active-limit names the bucket that refused, and the account keeps serving from another one meanwhile. In one hour of production traffic codex_think returned 1888 x 200 interleaved with 1282 x 429. Every 429 was recorded as a provider failure, so three in a row tripped the three-strikes cooldown and aisbf spent five minutes at a time rejecting requests itself -- including the majority the upstream would have answered. One client fired 32 requests during a cooldown and got 32 x 503 without a single one reaching OpenAI. - codex: raise RateLimitError on a 429 instead of raise_for_status(), without disabling the provider, on both the streaming and non-streaming OAuth paths. - handlers: _should_record_failure() now excludes upstream rate limits as well as CoderAI warm-up, and a quota refusal is forwarded to the client as 429 rather than a generic 500. Send ChatGPT-Account-ID again. The header was only set from tokens.account_id, which is null in every credentials file the login flow writes; the real value is in the id_token's chatgpt_account_id claim. Without it the backend picks a workspace itself, so an account belonging to several workspaces cannot be steered at the configured one. Make rate_limit actually do something. The spacing timestamp lived on the handler, but get_provider_handler() builds a fresh handler per request, so it was always 0 on arrival and no wait was ever applied -- a configured rate_limit was silently inert. Move the timestamps to a process-wide registry guarded by a per-slot lock, without which N concurrent requests all read the same stale timestamp and burst together. Verified: 4 concurrent requests at 0.5s spacing now take 1.50s, previously 0.00s. Bump version to 0.99.90. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Two bugs made codex_think advertise a context window it was not configured with, so clients that size their context from the model listing refused to run against it. 1. The endpoint-level model cache was keyed on type+endpoint alone. codex_think, openai_think and bigscreen are all codex:https://api.openai.com/v1 but authenticate as different accounts, so whichever prefetched first populated the shared entry and the others served its model list -- an OAuth ChatGPT provider inherited an API-key provider's generic OpenAI models, contexts and all. Same collision across the three kilo-* providers. Key the entry on a digest of the provider's credentials too. Also fix invalidate_provider_cache(), which tried to find a provider's endpoint entry with a substring match on a key that never contains the provider id. 2. Provider models were published exactly as the upstream API returned them, so default_context_size / default_max_request_tokens never reached the listing. Stamp them on, letting explicit configuration override the fetched value. _configured_context_size() deliberately does not call get_context_config_for_model(): that helper ends in _infer_context_size_from_model(), whose generic 8192 fallback is right for sizing a request but would overwrite a real fetched window (272000) with a guess when published. Bump version to 0.99.89. Co-Authored-By:
Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Every codex_think request was returning 503 within a millisecond with nothing in debug.log to explain it. The provider had simply been toggled off from the dashboard, but the six 503 sites in the request path all collapsed three distinct states -- the manual dashboard toggle, the failure cooldown and the usage-limit cooldown -- into a bare "Provider temporarily unavailable" with no log line, making a disabled provider indistinguishable from an upstream outage. Route all six through _raise_if_provider_unavailable(), which logs a WARNING and puts the actual reason (and remaining cooldown, where there is one) in the response detail. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
-
- 15 Jul, 2026 5 commits
-
-
Stefy Lanza (nextime / spora ) authored
A claude_think request with tools registered none: the CLI reported tools:[] at init with the shim still "pending", so the model had nothing to call and instead described the calls in prose ("[Tool: list_dir]") with stop_reason=end_turn. The turn looked successful, so this surfaced as a wrong answer rather than an error. The CLI connects --mcp-config servers "fully async (nonblocking)" and starts the turn without waiting, so whether any tools exist is a race between the shim's handshake and the first API request. Marking the server alwaysLoad puts it on the CLI's blocking path instead (its own two code paths: alwaysLoad servers are awaited, the rest are fire-and-forget). The race is host-dependent, which is why local testing never caught it — my box won it every time and registered the tools. Confirmed on the box that loses it, with identical CLI versions: without alwaysLoad -> tools=[], mcp=pending with alwaysLoad -> tools=['mcp__aisbf__get_weather'], mcp=connected Ruled out first: the shim is deployed there, both interpreters run it in ~20ms, and it answers initialize/tools-list correctly under production's python. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
The CLI credentials upload stored claude_config.cli_credentials_file server-side, but the path was then lost and CLI mode fell back to the OAuth2 credentials file: - uploadClaudeCliFile() only showed a toast; unlike its sibling uploadFileChunked(), it never mirrored the stored path into providersData. - saveProvider() posts providersData wholesale, and api_provider_save replaced the provider config outright, so a page loaded before the upload wiped the key on the next save. - There was no field for cli_credentials_file — it appeared only in help text — so the value was invisible and unrecoverable once lost. Preserve keys inside *_config blocks the client did not send, which fixes the whole class rather than this one key: any server-set key would have been dropped the same way. Only merges into a block the client actually sent, so a provider type change still drops the old block and nothing is resurrected. A key the client does send always wins, including falsy values like use_cli_mode=false. Add the missing text field (admin only — DB users keep CLI credentials in user_oauth2_credentials, not in claude_config), and mirror the upload result into providersData. The chunk endpoint now also returns config_path, the tilde form actually written to providers.json; file_path is absolute, so mirroring it would post back a value that disagrees with what the server stored. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
A CLI .credentials.json placed at a provider's credentials_file loaded fine and then reported "credentials are invalid or missing": _load_credentials() did a bare json.load, but every reader looks for `access_token`, while the CLI schema nests `claudeAiOauth.accessToken`. The tokens are equivalent — only the spelling differs — so normalize on load instead of rejecting them. ClaudeAuth.normalize_tokens() is the inverse of ClaudeProviderHandler._oauth_tokens_to_cli_credentials(); keep the two in step. It also carries subscriptionType/rateLimitTier across, so converting back to CLI shape reproduces the real values rather than falling back to that function's 'pro'/'default_claude_ai' defaults. Anything already in AISBF shape is untouched. Applied to the DB path too, which assigns auth.tokens directly and hits the identical mismatch if a CLI blob was stored verbatim. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
CLI mode never worked. Four independent defects, any one fatal: - The stdin frame used {"type":"user_message",...}; the CLI expects the Anthropic envelope {"type":"user","message":{...}} and silently discards anything else, so requests produced no output at all. With --input-format stream-json the CLI also ignores a prompt passed as an argv positional, so it must go over stdin. - _handle_cli_streaming_request() took no tools parameter but was called with tools=..., raising TypeError on every streaming request. - The event parser dispatched on a top-level content_block_delta/message_stop, but the CLI wraps Anthropic events as {"type":"stream_event","event":{...}}, so those branches were dead code. - Tool definitions were passed to --tools, which only selects built-in tools by name; a JSON blob there registers nothing. Tools now reach the model through an MCP stdio shim (claude_mcp_shim.py), which advertises the caller's definitions via tools/list. It never executes: in the OpenAI protocol the client runs tools, so the first tool_use ends the turn and is returned as tool_calls. Past calls/results are replayed as text since each request is a fresh session, with a system-prompt directive so the model trusts a replayed result instead of re-calling. The shim forces one deviation from the intended flag set: --disallowedTools 'mcp__*' is a blanket deny that also blocks the shim, and deny beats --allowedTools, so it cannot be kept alongside tool calling. It is retained when no tools are requested; with tools, --strict-mcp-config preserves the same isolation by loading only our config and ignoring the host's MCP servers. Consequences of the event model (one assistant event per content block, not per message): break on message_delta stop_reason == 'tool_use' rather than the first tool_use, or parallel calls are dropped; dedupe text/arguments across the delta and assistant paths; and emit sequential tool_call indices, since content-block indices count text/thinking blocks and leave holes that break client-side accumulation. System messages now go to --system-prompt instead of being inlined as user text the model could ignore. Non-streaming reports real token usage instead of zeros. Also fix the dashboard toggle that made this unreachable: providers.py imported _claude_cli_mode from startup, binding a copy of False at import time, while detection only ever wrote app_state['_claude_cli_mode']. The template therefore always received False and the use_cli_mode checkbox never rendered. Inject the value through init() like every other route global, and drop the orphaned startup global. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Add is_provider_disabled_cheap() so the rotation scan can skip providers that are disabled — manually via the dashboard toggle or by an auto-disable cooldown — without constructing the handler. Building a handler validates credentials, which for some provider types performs a network round-trip, so a disabled provider was still being contacted. The cheap pre-check reads the same cache/DB keys as is_rate_limited() and fails safe, leaving the authoritative check in place. Also stop recording a failure for non-retryable errors (400/401/403/404/422). Those are client/configuration problems, not provider-health problems: counting them tripped the consecutive-failure auto-disable and pulled healthy providers out of rotation for the whole cooldown. Co-Authored-By:Claude Opus 4.8 (1M context) <noreply@anthropic.com>
-
- 24 Jun, 2026 1 commit
-
-
Stefy Lanza (nextime / spora ) authored
When a coderai broker provider's worker is offline, requests are held and retried while it warms up. The hold window was hardcoded (10s x 3 = 30s). Expose it per provider via coderai_config: - broker_queue_timeout_seconds: total hold time (0 disables holding) - broker_warmup_wait_seconds: poll interval between retries - broker_max_warmup_waits: explicit retry count (when no queue timeout) Defaults preserve prior behavior. Applied to both warm-up retry loops. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
-
- 23 Jun, 2026 2 commits
-
-
Stefy Lanza (nextime / spora ) authored
Extend max output token resolution so per-model and default values are honored at every layer when the client omits max_tokens. Effective priority (client value always wins if present): autoselect per-model -> rotation per-model -> provider per-model -> provider default -> rotation default -> autoselect default - rotation path: consult the selected provider's config (new RotationHandler._get_provider_config) so a provider default_max_tokens applies even though rotations can't configure max_tokens themselves - AutoselectModelInfo.max_tokens: new per-model override field - AutoselectHandler._apply_autoselect_max_tokens applied in both the streaming and non-streaming dispatch: per-model override set directly (highest), autoselect default threaded via _autoselect_default_max_tokens as a lowest-priority fallback - rotation and both RequestHandler injections consume the threaded autoselect default, covering autoselect->rotation and autoselect->provider/model dispatch routes Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
-
Stefy Lanza (nextime / spora ) authored
Provider-level defaults were ignored when a provider had no per-model configs, so coderai/broker providers reported a fixed 4096 context window and clients fell back to their own defaults. - model list (handlers.py): fall back to provider_config.default_context_size for context_window/context_length before inferring - rotation/autoselect model list (routes/api.py): attach context_window/ context_length resolved from rotation/autoselect/provider defaults (was absent entirely, causing clients to default to 4096) - handle_rotation_model_list: honor provider default_context_size before heuristic auto-derivation - max output tokens: add ProviderModelConfig.max_tokens and default_max_tokens on Provider/Rotation/Autoselect configs, plus get_max_completion_tokens_for_model resolver, applied as a fallback in all request paths when the client omits max_tokens - get_max_request_tokens_for_model: fall back to provider_config.default_max_request_tokens; drop duplicated dead block Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
-