1. 26 Jul, 2026 13 commits
  2. 25 Jul, 2026 2 commits
    • Stefy Lanza (nextime / spora )'s avatar
    • Stefy Lanza (nextime / spora )'s avatar
      Don't let request-scoped errors trip the provider failure cooldown · fae38ea0
      Stefy Lanza (nextime / spora ) authored
      A model that intermittently lacks an image endpoint on kilo returns 404
      "No endpoints found that support image input". The kilo handler counted every
      exception (that 404, a 400 for an undecodable image, a 429) toward the
      three-strike cooldown, so a few request-scoped errors disabled kilo-spora for
      ALL traffic for 5 minutes — during which a high-rate client piled up thousands
      of 503 "in failure cooldown" rejections.
      
      Add a shared classifier in providers/base.py: is_request_scoped_error /
      is_request_scoped_status / extract_http_status treat 400/404/413/415/422/429
      (and the wrapped streaming-error message form) as request-scoped — about the
      specific request, not provider health. Genuine faults (5xx, timeouts, network)
      still count.
      
      Wire it into both places that record failures:
      - KiloProviderHandler.handle_request and _handle_streaming_request skip
        record_failure() for request-scoped errors (429 still routes through
        handle_429_error for adaptive backoff).
      - handlers._should_record_failure excludes them for every provider.
      
      The error still propagates to the caller; it just no longer disables the whole
      provider. Only a genuinely unhealthy provider now enters the cooldown.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      fae38ea0
  3. 24 Jul, 2026 4 commits
    • Stefy Lanza (nextime / spora )'s avatar
    • Stefy Lanza (nextime / spora )'s avatar
      Disable a provider per rotation, not globally · f8d7d79d
      Stefy Lanza (nextime / spora ) authored
      Previously the Disable button on a rotation flipped a single global
      provider_manual_disabled flag: disabling a provider in one rotation removed it
      from every rotation, from autoselect, and (before this series) from direct API
      calls too. That is not what "disable in this rotation" should mean.
      
      Disabling is now membership in a single rotation, expressed as `enabled: false`
      on that provider's entry in the rotation config.
      
      handlers.py: the rotation scan no longer consults the global manual flag. It
      skips an entry when `provider.get('enabled') is False` (cheap, before building
      the handler), and otherwise skips only on a genuine auto-disable cooldown
      (failure threshold / usage limit) via the new _provider_in_availability_cooldown
      helper. is_provider_disabled_cheap is no longer used here (import dropped).
      Autoselect inherits this since it dispatches through handle_rotation_request.
      
      rotations.html / user_rotations.html: the per-provider toggle flips
      `enabled` on that rotation entry and persists just that rotation (full-config
      POST for admin, single-rotation POST for DB users), updating only that entry's
      button instead of every rotation sharing the provider. The global
      manual-status fetch is gone.
      
      The legacy global manual-disable endpoints remain but are now inert for
      rotation and direct-call gating.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      f8d7d79d
    • Stefy Lanza (nextime / spora )'s avatar
    • Stefy Lanza (nextime / spora )'s avatar
      Manual disable is rotation-only; include media in response cache key · e6616376
      Stefy Lanza (nextime / spora ) authored
      Two independent fixes surfaced while diagnosing kilo-spora vision requests
      returning 503 and (once enabled) risking stale image answers.
      
      handlers.py: _raise_if_provider_unavailable no longer rejects a direct,
      provider-named request just because the provider is manually disabled. The
      dashboard disable toggle is meant to remove a provider from rotation/autoselect
      (those scans use is_provider_disabled_cheap) — it must not take the provider
      offline across the whole API. Direct calls still 503 on genuine unavailability
      (failure cooldown or usage-limit cooldown).
      
      cache.py: _generate_cache_key hashed only the text parts of multimodal
      messages, dropping image_url/video_url/audio_url. A caller that sends a fixed
      prompt with a varying image (e.g. the real-estate OUTDOOR/INDOOR classifier)
      collapsed every image to one cache key, so the first image's answer was served
      for all subsequent images. Fold each media reference into the message hash so
      different attachments produce different keys.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      e6616376
  4. 22 Jul, 2026 6 commits
    • Stefy Lanza (nextime / spora )'s avatar
      Merge branch 'fix/stream-wrapper-dict-response' · 9f9ec1a0
      Stefy Lanza (nextime / spora ) authored
      Synthesize a valid SSE stream when a dict response hits the streaming path.
      9f9ec1a0
    • Stefy Lanza (nextime / spora )'s avatar
      Synthesize a valid SSE stream when a dict response hits the streaming path · 5255a04e
      Stefy Lanza (nextime / spora ) authored
      The chunked-request handler always returns a complete chat.completion
      dict ("Streaming is not supported for chunked rotation requests"), but
      stream=true callers still wrap it in the SSE stream generator, which
      iterated the dict and emitted its KEYS as string chunks (data: "id",
      data: "choices", ...). Clients crashed with "'str' object has no
      attribute 'choices'". Previously masked: chunked requests died earlier
      on the null-content TypeError, so this path was unreachable.
      
      When the streaming wrapper receives a plain dict, emit one well-formed
      chat.completion.chunk instead - delta carrying content and/or tool_calls
      (with the per-entry index streaming requires), the real finish_reason,
      usage attached - followed by data: [DONE].
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      5255a04e
    • Stefy Lanza (nextime / spora )'s avatar
      Merge branch 'fix/chunked-rotation-toolcall-none-content' · 1e30f7f8
      Stefy Lanza (nextime / spora ) authored
      Stop chunked requests crashing on tool-call responses with null content.
      1e30f7f8
    • Stefy Lanza (nextime / spora )'s avatar
      Stop chunked requests crashing on tool-call responses with null content · 6a73a196
      Stefy Lanza (nextime / spora ) authored
      Tool-call completions carry content=None (the key exists, so a .get
      default never applies). Both chunked request paths concatenated it
      directly ("" += None -> TypeError), recording a provider failure on
      every attempt even though upstream returned 200 - rotation 'lisa'
      exhausted codex_think and returned 429 despite available usage.
      
      Coalesce null content to '' and pass tool_calls through to the combined
      response with finish_reason "tool_calls" (previously they were silently
      discarded and agents received an empty completion).
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      6a73a196
    • Stefy Lanza (nextime / spora )'s avatar
      Merge branch 'fix/broker-keepalive-eventloop-blocking' · 2ff363aa
      Stefy Lanza (nextime / spora ) authored
      Stop event-loop blocking that trips broker WebSocket keepalive timeouts.
      2ff363aa
    • Stefy Lanza (nextime / spora )'s avatar
      Stop event-loop blocking that trips broker WebSocket keepalive timeouts · fe47ead1
      Stefy Lanza (nextime / spora ) authored
      Under load the CoderAI broker WebSocket was being torn down with
      "1011 keepalive ping timeout" and reconnecting (surfaced as 502s by the
      fronting proxy). Root cause: synchronous work run directly on the asyncio
      event loop starves the WebSocket keepalive so PONGs miss the 20s window.
      
      Remove the blockers:
      
      - analytics: dispatch the blocking token_usage / per-user INSERTs to a
        worker thread (fire-and-forget when a loop is running, inline otherwise)
        instead of calling them inline from async request handlers.
      - database: add a MySQL connection pool (ping+reconnect, direct-connect
        fallback) and return connections to the pool on with-block exit instead
        of opening a fresh TCP+auth connection per query and leaking it to GC.
      - utils: cache the tiktoken encoding and memoize count_messages_tokens by
        content, collapsing the several CPU-bound BPE passes per request into one.
      - coderai_broker: run touch()/publish_response Redis I/O off the loop via
        asyncio.to_thread (awaited to preserve chunk order), matching the existing
        consume_request pattern.
      - routes/coderai_broker: answer heartbeats immediately and process response
        frames on a dedicated FIFO worker so a slow publish can't delay the next
        heartbeat.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      fe47ead1
  5. 21 Jul, 2026 7 commits
    • Stefy Lanza (nextime / spora )'s avatar
      Persist the codex endpoint correction instead of losing it · c72b8a07
      Stefy Lanza (nextime / spora ) authored
      In OAuth mode _get_valid_api_key() notices the configured endpoint is
      not the ChatGPT backend, corrects base_url in memory and calls
      _update_provider_endpoint() to write the correction back. For a global
      provider that write called config.save_providers() -- a method that does
      not exist. The AttributeError was swallowed by the surrounding
      except and logged as a warning, so the correction only ever lived in
      memory: providers.json kept the wrong endpoint and every restart re-ran
      the same dance. codex_think sat like that, configured for
      api.openai.com/v1 while every request went to chatgpt.com.
      
      Rewrite providers.json in place instead, merging into the on-disk
      document rather than serialising the in-memory config, so a concurrent
      dashboard edit is not clobbered by stale state. The write goes through a
      temp file and os.replace() so a crash mid-write cannot truncate the
      config. Missing provider is now logged explicitly rather than silently
      creating an entry, and the except logs a traceback so the next failure
      of this kind is not invisible.
      
      Bump version to 0.99.92.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      c72b8a07
    • Stefy Lanza (nextime / spora )'s avatar
      Route tool calls on reasoning models to the Responses API · f674ba8e
      Stefy Lanza (nextime / spora ) authored
      openai_think is a codex provider in API-key mode, which posts to
      /v1/chat/completions. That endpoint rejects the request outright when a
      reasoning model is given function tools:
      
        400 "Function tools with reasoning_effort are not supported for
        gpt-5.6-sol in /v1/chat/completions. To use function tools, use
        /v1/responses or set reasoning_effort to 'none'."
      
      aisbf never sends reasoning_effort -- the model applies it server-side --
      so there is nothing to strip from the payload; the endpoint is the
      problem. Setting it to 'none' would work but silently disables reasoning
      on a provider chosen for it, so route these requests to /v1/responses
      instead, which supports tools and reasoning together.
      
      OAuth mode already spoke this protocol against the ChatGPT backend, so
      _handle_request_oauth2_mode is now a thin wrapper over a shared
      _handle_request_responses_api(); only the URL and auth headers differ
      between the two. The flattened tool format the codex converter already
      produces is what /v1/responses expects.
      
      Fix double-counted failures. codex's handle_request recorded a failure
      and re-raised, and the caller in handlers.py recorded it again, so one
      failed request counted twice and the three-strikes cooldown tripped
      after two requests instead of three. The handler layer records for every
      provider, so the provider-level call is the redundant one. Note the same
      pattern exists in google/qwen/kilo/openai/ollama/coderai and is left
      alone here -- fixed only where it was reproduced.
      
      Bump version to 0.99.91.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      f674ba8e
    • Stefy Lanza (nextime / spora )'s avatar
      Stop upstream 429s from disabling a codex provider · f9d15eee
      Stefy Lanza (nextime / spora ) authored
      A 429 from the ChatGPT backend is per-quota-bucket, not a provider
      fault: x-codex-active-limit names the bucket that refused, and the
      account keeps serving from another one meanwhile. In one hour of
      production traffic codex_think returned 1888 x 200 interleaved with
      1282 x 429. Every 429 was recorded as a provider failure, so three in a
      row tripped the three-strikes cooldown and aisbf spent five minutes at a
      time rejecting requests itself -- including the majority the upstream
      would have answered. One client fired 32 requests during a cooldown and
      got 32 x 503 without a single one reaching OpenAI.
      
      - codex: raise RateLimitError on a 429 instead of raise_for_status(),
        without disabling the provider, on both the streaming and
        non-streaming OAuth paths.
      - handlers: _should_record_failure() now excludes upstream rate limits
        as well as CoderAI warm-up, and a quota refusal is forwarded to the
        client as 429 rather than a generic 500.
      
      Send ChatGPT-Account-ID again. The header was only set from
      tokens.account_id, which is null in every credentials file the login
      flow writes; the real value is in the id_token's chatgpt_account_id
      claim. Without it the backend picks a workspace itself, so an account
      belonging to several workspaces cannot be steered at the configured one.
      
      Make rate_limit actually do something. The spacing timestamp lived on
      the handler, but get_provider_handler() builds a fresh handler per
      request, so it was always 0 on arrival and no wait was ever applied --
      a configured rate_limit was silently inert. Move the timestamps to a
      process-wide registry guarded by a per-slot lock, without which N
      concurrent requests all read the same stale timestamp and burst
      together. Verified: 4 concurrent requests at 0.5s spacing now take
      1.50s, previously 0.00s.
      
      Bump version to 0.99.90.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      f9d15eee
    • Stefy Lanza (nextime / spora )'s avatar
      Advertise a provider's configured context window in /v1/models · b1320e45
      Stefy Lanza (nextime / spora ) authored
      Two bugs made codex_think advertise a context window it was not
      configured with, so clients that size their context from the model
      listing refused to run against it.
      
      1. The endpoint-level model cache was keyed on type+endpoint alone.
         codex_think, openai_think and bigscreen are all
         codex:https://api.openai.com/v1 but authenticate as different
         accounts, so whichever prefetched first populated the shared entry
         and the others served its model list -- an OAuth ChatGPT provider
         inherited an API-key provider's generic OpenAI models, contexts and
         all. Same collision across the three kilo-* providers. Key the entry
         on a digest of the provider's credentials too. Also fix
         invalidate_provider_cache(), which tried to find a provider's
         endpoint entry with a substring match on a key that never contains
         the provider id.
      
      2. Provider models were published exactly as the upstream API returned
         them, so default_context_size / default_max_request_tokens never
         reached the listing. Stamp them on, letting explicit configuration
         override the fetched value.
      
      _configured_context_size() deliberately does not call
      get_context_config_for_model(): that helper ends in
      _infer_context_size_from_model(), whose generic 8192 fallback is right
      for sizing a request but would overwrite a real fetched window (272000)
      with a guess when published.
      
      Bump version to 0.99.89.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      b1320e45
    • Stefy Lanza (nextime / spora )'s avatar
    • Stefy Lanza (nextime / spora )'s avatar
      Say why a provider request is rejected with 503 · a968e661
      Stefy Lanza (nextime / spora ) authored
      Every codex_think request was returning 503 within a millisecond with
      nothing in debug.log to explain it. The provider had simply been toggled
      off from the dashboard, but the six 503 sites in the request path all
      collapsed three distinct states -- the manual dashboard toggle, the
      failure cooldown and the usage-limit cooldown -- into a bare
      "Provider temporarily unavailable" with no log line, making a disabled
      provider indistinguishable from an upstream outage.
      
      Route all six through _raise_if_provider_unavailable(), which logs a
      WARNING and puts the actual reason (and remaining cooldown, where there
      is one) in the response detail.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      a968e661
    • Stefy Lanza (nextime / spora )'s avatar
      Update · e6065d17
      Stefy Lanza (nextime / spora ) authored
      e6065d17
  6. 15 Jul, 2026 5 commits
    • Stefy Lanza (nextime / spora )'s avatar
      Make the CLI wait for the MCP tool shim; bump to 0.99.87 · da972924
      Stefy Lanza (nextime / spora ) authored
      A claude_think request with tools registered none: the CLI reported tools:[] at
      init with the shim still "pending", so the model had nothing to call and instead
      described the calls in prose ("[Tool: list_dir]") with stop_reason=end_turn. The
      turn looked successful, so this surfaced as a wrong answer rather than an error.
      
      The CLI connects --mcp-config servers "fully async (nonblocking)" and starts the
      turn without waiting, so whether any tools exist is a race between the shim's
      handshake and the first API request. Marking the server alwaysLoad puts it on
      the CLI's blocking path instead (its own two code paths: alwaysLoad servers are
      awaited, the rest are fire-and-forget).
      
      The race is host-dependent, which is why local testing never caught it — my box
      won it every time and registered the tools. Confirmed on the box that loses it,
      with identical CLI versions:
      
        without alwaysLoad -> tools=[],                        mcp=pending
        with    alwaysLoad -> tools=['mcp__aisbf__get_weather'], mcp=connected
      
      Ruled out first: the shim is deployed there, both interpreters run it in ~20ms,
      and it answers initialize/tools-list correctly under production's python.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      da972924
    • Stefy Lanza (nextime / spora )'s avatar
      Stop the admin provider UI from silently dropping cli_credentials_file; bump to 0.99.86 · 294a7273
      Stefy Lanza (nextime / spora ) authored
      The CLI credentials upload stored claude_config.cli_credentials_file server-side,
      but the path was then lost and CLI mode fell back to the OAuth2 credentials file:
      
      - uploadClaudeCliFile() only showed a toast; unlike its sibling
        uploadFileChunked(), it never mirrored the stored path into providersData.
      - saveProvider() posts providersData wholesale, and api_provider_save replaced
        the provider config outright, so a page loaded before the upload wiped the key
        on the next save.
      - There was no field for cli_credentials_file — it appeared only in help text —
        so the value was invisible and unrecoverable once lost.
      
      Preserve keys inside *_config blocks the client did not send, which fixes the
      whole class rather than this one key: any server-set key would have been dropped
      the same way. Only merges into a block the client actually sent, so a provider
      type change still drops the old block and nothing is resurrected. A key the
      client does send always wins, including falsy values like use_cli_mode=false.
      
      Add the missing text field (admin only — DB users keep CLI credentials in
      user_oauth2_credentials, not in claude_config), and mirror the upload result
      into providersData. The chunk endpoint now also returns config_path, the tilde
      form actually written to providers.json; file_path is absolute, so mirroring it
      would post back a value that disagrees with what the server stored.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      294a7273
    • Stefy Lanza (nextime / spora )'s avatar
      Accept Claude CLI-format credentials files · 77ac9262
      Stefy Lanza (nextime / spora ) authored
      A CLI .credentials.json placed at a provider's credentials_file loaded fine and
      then reported "credentials are invalid or missing": _load_credentials() did a
      bare json.load, but every reader looks for `access_token`, while the CLI schema
      nests `claudeAiOauth.accessToken`. The tokens are equivalent — only the spelling
      differs — so normalize on load instead of rejecting them.
      
      ClaudeAuth.normalize_tokens() is the inverse of
      ClaudeProviderHandler._oauth_tokens_to_cli_credentials(); keep the two in step.
      It also carries subscriptionType/rateLimitTier across, so converting back to CLI
      shape reproduces the real values rather than falling back to that function's
      'pro'/'default_claude_ai' defaults. Anything already in AISBF shape is untouched.
      
      Applied to the DB path too, which assigns auth.tokens directly and hits the
      identical mismatch if a CLI blob was stored verbatim.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      77ac9262
    • Stefy Lanza (nextime / spora )'s avatar
      Fix Claude CLI mode: transport, real tool calling via MCP shim, dashboard toggle; bump to 0.99.85 · 2c3e2d8e
      Stefy Lanza (nextime / spora ) authored
      CLI mode never worked. Four independent defects, any one fatal:
      
      - The stdin frame used {"type":"user_message",...}; the CLI expects the
        Anthropic envelope {"type":"user","message":{...}} and silently discards
        anything else, so requests produced no output at all. With
        --input-format stream-json the CLI also ignores a prompt passed as an argv
        positional, so it must go over stdin.
      - _handle_cli_streaming_request() took no tools parameter but was called with
        tools=..., raising TypeError on every streaming request.
      - The event parser dispatched on a top-level content_block_delta/message_stop,
        but the CLI wraps Anthropic events as {"type":"stream_event","event":{...}},
        so those branches were dead code.
      - Tool definitions were passed to --tools, which only selects built-in tools by
        name; a JSON blob there registers nothing.
      
      Tools now reach the model through an MCP stdio shim (claude_mcp_shim.py), which
      advertises the caller's definitions via tools/list. It never executes: in the
      OpenAI protocol the client runs tools, so the first tool_use ends the turn and
      is returned as tool_calls. Past calls/results are replayed as text since each
      request is a fresh session, with a system-prompt directive so the model trusts
      a replayed result instead of re-calling.
      
      The shim forces one deviation from the intended flag set: --disallowedTools
      'mcp__*' is a blanket deny that also blocks the shim, and deny beats
      --allowedTools, so it cannot be kept alongside tool calling. It is retained
      when no tools are requested; with tools, --strict-mcp-config preserves the same
      isolation by loading only our config and ignoring the host's MCP servers.
      
      Consequences of the event model (one assistant event per content block, not per
      message): break on message_delta stop_reason == 'tool_use' rather than the first
      tool_use, or parallel calls are dropped; dedupe text/arguments across the delta
      and assistant paths; and emit sequential tool_call indices, since content-block
      indices count text/thinking blocks and leave holes that break client-side
      accumulation.
      
      System messages now go to --system-prompt instead of being inlined as user text
      the model could ignore. Non-streaming reports real token usage instead of zeros.
      
      Also fix the dashboard toggle that made this unreachable: providers.py imported
      _claude_cli_mode from startup, binding a copy of False at import time, while
      detection only ever wrote app_state['_claude_cli_mode']. The template therefore
      always received False and the use_cli_mode checkbox never rendered. Inject the
      value through init() like every other route global, and drop the orphaned
      startup global.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      2c3e2d8e
    • Stefy Lanza (nextime / spora )'s avatar
      Skip already-disabled providers before building handlers in rotation · 2f3cbf0e
      Stefy Lanza (nextime / spora ) authored
      Add is_provider_disabled_cheap() so the rotation scan can skip providers
      that are disabled — manually via the dashboard toggle or by an auto-disable
      cooldown — without constructing the handler. Building a handler validates
      credentials, which for some provider types performs a network round-trip, so
      a disabled provider was still being contacted. The cheap pre-check reads the
      same cache/DB keys as is_rate_limited() and fails safe, leaving the
      authoritative check in place.
      
      Also stop recording a failure for non-retryable errors (400/401/403/404/422).
      Those are client/configuration problems, not provider-health problems: counting
      them tripped the consecutive-failure auto-disable and pulled healthy providers
      out of rotation for the whole cooldown.
      Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
      2f3cbf0e
  7. 24 Jun, 2026 1 commit
    • Stefy Lanza (nextime / spora )'s avatar
      Make CoderAI broker queue hold time configurable; bump to 0.99.83 · 8efe5650
      Stefy Lanza (nextime / spora ) authored
      When a coderai broker provider's worker is offline, requests are held and
      retried while it warms up. The hold window was hardcoded (10s x 3 = 30s).
      Expose it per provider via coderai_config:
      - broker_queue_timeout_seconds: total hold time (0 disables holding)
      - broker_warmup_wait_seconds: poll interval between retries
      - broker_max_warmup_waits: explicit retry count (when no queue timeout)
      
      Defaults preserve prior behavior. Applied to both warm-up retry loops.
      Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
      8efe5650
  8. 23 Jun, 2026 2 commits
    • Stefy Lanza (nextime / spora )'s avatar
      Resolve max_tokens per-model + defaults across rotations and autoselect; bump to 0.99.82 · 5dff11f4
      Stefy Lanza (nextime / spora ) authored
      Extend max output token resolution so per-model and default values are
      honored at every layer when the client omits max_tokens.
      
      Effective priority (client value always wins if present):
        autoselect per-model -> rotation per-model -> provider per-model
        -> provider default -> rotation default -> autoselect default
      
      - rotation path: consult the selected provider's config (new
        RotationHandler._get_provider_config) so a provider default_max_tokens
        applies even though rotations can't configure max_tokens themselves
      - AutoselectModelInfo.max_tokens: new per-model override field
      - AutoselectHandler._apply_autoselect_max_tokens applied in both the
        streaming and non-streaming dispatch: per-model override set directly
        (highest), autoselect default threaded via _autoselect_default_max_tokens
        as a lowest-priority fallback
      - rotation and both RequestHandler injections consume the threaded
        autoselect default, covering autoselect->rotation and
        autoselect->provider/model dispatch routes
      Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
      5dff11f4
    • Stefy Lanza (nextime / spora )'s avatar
      Honor provider/rotation/autoselect context & token defaults; bump to 0.99.81 · d626dda9
      Stefy Lanza (nextime / spora ) authored
      Provider-level defaults were ignored when a provider had no per-model
      configs, so coderai/broker providers reported a fixed 4096 context window
      and clients fell back to their own defaults.
      
      - model list (handlers.py): fall back to provider_config.default_context_size
        for context_window/context_length before inferring
      - rotation/autoselect model list (routes/api.py): attach context_window/
        context_length resolved from rotation/autoselect/provider defaults (was
        absent entirely, causing clients to default to 4096)
      - handle_rotation_model_list: honor provider default_context_size before
        heuristic auto-derivation
      - max output tokens: add ProviderModelConfig.max_tokens and
        default_max_tokens on Provider/Rotation/Autoselect configs, plus
        get_max_completion_tokens_for_model resolver, applied as a fallback in
        all request paths when the client omits max_tokens
      - get_max_request_tokens_for_model: fall back to
        provider_config.default_max_request_tokens; drop duplicated dead block
      Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
      d626dda9