- Contains shared tri-state feature resolution for request-time controls such as context condensation, response cache, prompt batching, prompt security, Context Lens analytics, and NSFW/privacy classification
### aisbf/models.py
Pydantic models for data validation:
...
...
@@ -143,6 +144,13 @@ Request handling logic:
- `AutoselectHandler` - Handles AI-powered model selection
- Reuses shared Studio capability inference for model metadata normalization
- Supports both streaming and non-streaming responses
- Performs native local prompt analysis before upstream execution when enabled, including prompt-security scanning, Context Lens-style prompt composition capture, and optional local blocking for high-risk prompts
### aisbf/prompt_analysis.py
Native prompt security and prompt composition analytics:
- Performs local regex/heuristic scanning inspired by Prompt Sail-style prompt governance
- Builds Context Lens-style composition summaries from messages, tools, prompt/input payloads, and token-role distribution
- Produces redacted evidence previews and hashed request fingerprints for persistence without storing raw sensitive content by default
AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Claude Code, Ollama, Kiro, Kilocode, Codex, Qwen) with support for provider rotation, AI-assisted model selection, unified wallet system, and error tracking.
AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Claude Code, Ollama, Kiro, Kilocode, Codex, Qwen, CoderAI, RunPod) with support for provider rotation, AI-assisted model selection, marketplace exports, multimodal Studio workflows, unified wallet operations, and operational analytics.
### Key Features
-**Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Claude Code (OAuth2 or CLI), Ollama, Kiro (Amazon Q Developer), Kilocode (OAuth2), Codex (OAuth2), and Qwen (API Key/OAuth2)
-**Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Claude Code (OAuth2 or CLI), Ollama, Kiro (Amazon Q Developer), Kilocode (OAuth2), Codex (OAuth2), Qwen (API Key/OAuth2), CoderAI, and RunPod
-**Unified Wallet System**: Fiat wallet with crypto/PayPal/Stripe top-ups and auto top-up for subscription renewals
-**Claude OAuth2 Authentication**: Full OAuth2 PKCE flow for Claude Code with automatic token refresh, Chrome extension for remote servers, and curl_cffi TLS fingerprinting support
-**Claude CLI Mode**: When the `claude` binary is present in PATH at startup, AISBF automatically enables CLI proxy mode — requests are piped through the official Anthropic CLI (`claude -p`) instead of the HTTP API, using per-user isolated config directories with a 10-minute idle cleanup
-**Claude CLI Mode**: When the `claude` binary is present in PATH at startup, AISBF automatically enables CLI proxy mode and uses per-user isolated config directories with automatic idle cleanup
-**Kiro-cli Support**: Full support for Amazon Q Developer CLI authentication with Device Authorization Grant
-**Kilocode OAuth2 Authentication**: OAuth2 Device Authorization Grant for Kilo Code with automatic token refresh
-**Codex OAuth2 Authentication**: OAuth2 Device Authorization Grant for OpenAI Codex with automatic token refresh and API key exchange
-**Qwen Authentication**: API key authentication (recommended) or OAuth2 (discontinued as of April 2026)
-**CoderAI Broker & Bridge**: Broker-backed and direct CoderAI integrations with WebSocket transport, NAT-friendly outbound registration, session persistence, runtime telemetry, and Studio-native proxy forwarding
-**RunPod Runtime Integration**: Pod-backed, serverless-template, and public-catalog RunPod providers with lifecycle management, persistent runtime state, startup polling, idle shutdown, and protocol-aware delegation to OpenAI, Ollama, or CoderAI handlers
-**Rotation Models**: Intelligent load balancing across multiple providers with weighted model selection and automatic failover
-**Autoselect Models**: AI-powered model selection that analyzes request content to route to the most appropriate specialized model
-**Semantic Classification**: Fast hybrid BM25 + semantic model selection using sentence transformers (optional)
...
...
@@ -24,16 +26,62 @@ AISBF is a modular proxy server for managing multiple AI provider integrations.
-**Provider-Native Caching**: 50-70% cost reduction using Anthropic `cache_control`, Google Context Caching, and OpenAI-compatible APIs
-**Response Caching**: 20-30% cache hit rate with intelligent request deduplication using SHA256-based cache keys
-**Smart Request Batching**: 15-25% latency reduction by batching similar requests within 100ms window
-**Token Usage Analytics**: Comprehensive analytics dashboard with token usage tracking, cost estimation, performance metrics, and export functionality
-**Token Usage Analytics**: Comprehensive analytics dashboard with token usage tracking, cost estimation, performance metrics, broker telemetry, and export functionality
-**Marketplace & References**: Publish providers, models, rotations, and autoselects to a shared market, import them as locked references, settle usage revenue, and administer listings from dedicated dashboard pages
-**AISBF Studio**: Multimodal dashboard workspace with function bindings for chat, image, video, audio, embeddings, and 3D workflows, plus reusable characters, environments, voices, archive assets, and custom pipelines
-**Context Management**: Automatic context condensation with 8+ methods when approaching model limits
-**SSL/TLS Support**: Built-in HTTPS support with Let's Encrypt integration and automatic certificate renewal
-**TOR Hidden Service**: Full support for exposing AISBF over TOR network as a hidden service
-**MCP Server**: Model Context Protocol server for remote agent configuration and model access
-**Persistent Database**: SQLite/MySQL-based tracking of token usage, context dimensions, model embeddings, and user configurations
-**Persistent Database**: SQLite/MySQL-based tracking of token usage, context dimensions, market data, RunPod runtime state, broker sessions, and user configurations
-**Multi-User Support**: User management with isolated configurations, role-based access control, and API token management
-**User-Specific API Endpoints**: Dedicated API endpoints for authenticated users to access their own configurations with Bearer token authentication
-**Proxy-Awareness**: Full support for reverse proxy deployments with automatic URL generation and subpath support
## New Since 0.99.65
### CoderAI broker and telemetry
- Added broker-side WebSocket endpoints for global and user-scoped CoderAI sessions
- Added provider-scoped registration tokens with inline rotation in the provider editor
- Persisted broker session metadata to `~/.aisbf/coderai_broker_sessions.json` so dashboards keep the last known status after restart
- Added dashboard visibility into session owner, client ID, connection state, transport, advertised Studio endpoints, and estimated performance metrics such as latency and tokens-per-second
- Expanded bridge proxy support so Studio-native endpoints, long-running jobs, multipart payloads, progress polling, and non-chat streaming envelopes can transit the broker connection
### RunPod provider runtime management
- Added `runpod` provider type with validation and dashboard configuration support
- Added `pod`, `serverless_template`, and `public` RunPod operating modes
- Added wrapper delegation so non-public RunPod resources can expose OpenAI-compatible, Ollama, or CoderAI-backed runtimes through one AISBF provider
- Added persistent RunPod runtime state in the database, including endpoint URL, resource kind, public catalog metadata, and last-used timestamps
- Added startup polling, timeout controls, idle shutdown, and runtime refresh actions in the provider dashboard
- Added public catalog support with per-model protocol override for `runpod_public`, `openai`, `ollama`, or `coderai`
### Marketplace and reference imports
- Added a dedicated market administration page for filtering, reviewing, and paging through listings
- Added marketplace publishing and settlement logic so providers, models, rotations, and autoselects can be shared and monetized
- Added listing analytics snapshots, revenue tracking, usage counters, and market fee support
- Added market import references so users can import shared resources as locked read-only references inside their provider, rotation, and autoselect selectors
- Added runtime resolution of market references plus improved dashboard rendering for locked resources and import variants
- Added user filtering and improved relevance ordering for market discovery and admin search
### AISBF Studio expansion
- Added Studio dashboard shell with multimodal tabs for chat, image, video, audio, embeddings, profiles, pipelines, archive, and 3D workflows
- Added Studio function-binding APIs for admin and user scopes so each workflow can be bound to the right provider/model role
- Added file-backed and database-backed persistence for Studio characters, environments, voices, archive items, custom pipelines, and binding definitions
- Added custom pipeline creation, update, delete, and run endpoints in both admin and user scopes
- Added capability-aware Studio adapter inference and profile shaping so different providers receive the payload format they expect for multimodal endpoints
- Added support for progress endpoints, reusable thumbnails, and user-owned Studio assets
### Operational hardening and dashboard fixes
- Added automatic cleanup of stale self-registered accounts that never log in within 14 days
- Reused cached provider model data before forced auto-detect refreshes to reduce unnecessary upstream calls
- Normalized remaining dashboard proxy-aware paths and bootstrap URLs to improve reverse-proxy compatibility
- Fixed analytics, dashboard state restoration, helper regressions, and market rendering issues introduced during the 0.99.65 to 0.99.66 cycle
## Project Structure
```
...
...
@@ -42,45 +90,36 @@ aisbf/
│ ├── __init__.py # Module initialization with exports
│ ├── config.py # Configuration management
│ ├── models.py # Pydantic models
│ ├── providers.py # Provider handlers
│ ├── handlers.py # Request handlers
│ ├── payments/ # Payment system
│ ├── analytics.py # Analytics and telemetry
│ ├── coderai_broker.py # CoderAI broker session registry and transport
│ ├── studio.py # Studio catalog helpers
│ ├── studio_adapters.py # Studio adapter inference and payload shaping
│ ├── studio_services.py # Studio persistence and runtime services
│ ├── providers/ # Provider handlers
│ │ ├── __init__.py
│ │ ├── models.py # Payment models
│ │ ├── service.py # Payment service
│ │ ├── fiat/ # Fiat payment handlers
│ │ │ ├── __init__.py
│ │ │ ├── stripe_handler.py # Stripe integration
│ │ │ └── paypal_handler.py # PayPal integration
│ │ ├── crypto/ # Crypto payment handlers
│ │ │ ├── __init__.py
│ │ │ ├── wallet.py # HD wallet manager
│ │ │ └── monitor.py # Crypto payment monitor
│ │ ├── subscription/ # Subscription management
│ │ │ ├── __init__.py
│ │ │ ├── manager.py # Subscription lifecycle
│ │ │ ├── renewal.py # Renewal processing
│ │ │ └── quota.py # Quota management
│ │ └── wallet/ # Wallet system (NEW)
│ │ ├── __init__.py
│ │ └── manager.py # Wallet operations
│ ├── notifications/ # Notification system
│ ├── __init__.py
│ │ ├── base.py # Shared provider utilities
│ │ ├── claude.py
│ │ ├── coderai.py # CoderAI direct and bridge integration
│ │ ├── codex.py
│ │ ├── google.py
│ │ ├── ollama.py
│ │ ├── openai.py
│ │ ├── qwen.py
│ │ ├── runpod.py # RunPod runtime lifecycle and delegation
│ │ └── kiro/
│ ├── auth/ # Authentication modules
│ ├── routes/ # FastAPI routes
│ │ ├── coderai_broker.py # Broker transport endpoints
Authenticated users can access their own configurations via user-specific API endpoints. These endpoints require either a valid API token (generated in the user dashboard) or session authentication.
## Marketplace and References
#### Authentication
AISBF includes a built-in marketplace for sharing configured resources between users.
**Option 1: Bearer Token (Recommended for API access)**
```bash
Authorization: Bearer YOUR_API_TOKEN
```
### What can be published
**Option 2: Query Parameter**
```bash
?token=YOUR_API_TOKEN
```
- Providers
- Individual provider models
- Rotations
- Autoselect configurations
#### User API Endpoints
### Marketplace features
| Endpoint | Description |
|----------|-------------|
| `GET /api/u/{username}/models` | List available models from user's own configurations |
| `GET /api/u/{username}/providers` | List user's provider configurations |
| `GET /api/u/{username}/rotations` | List user's rotation configurations |
| `GET /api/u/{username}/autoselects` | List user's autoselect configurations |
| `POST /api/u/{username}/chat/completions` | Chat completions using user's own models |
| `GET /api/u/{username}/{config_type}/models` | List models for specific config type |
- Dedicated admin market page at `/dashboard/admin/market`
- Listing filters by search term, owner, source type, active status, and availability
- Pricing per million tokens and per 1,000 requests
- Revenue, request-count, vote, and analytics snapshots for each listing
- Settlement support for usage-based sharing
- User export controls and market visibility filtering
**Access Control:**
-**Global Tokens** (from `aisbf.json`): Access to global endpoints only (`/api/...`), no access to user-specific endpoints
-**User Tokens** (from dashboard): Access to their user-specific endpoints (`/api/u/username/...`) and global endpoints
### Imported references
#### Example: Using User API with cURL
Users can import market listings as references instead of duplicating the underlying configuration.
- Idle session cleanup: temp dirs are removed after 10 minutes of inactivity; active requests always get a fresh subprocess
- Multiple parallel requests are handled concurrently via `asyncio.create_subprocess_exec`
- Credentials can be supplied in two ways:
-**"Use Claude CLI mode" checkbox**: derives credentials automatically from existing OAuth2 tokens already stored in AISBF, converting them to the `claudeAiOauth` schema expected by the CLI
-**Explicit upload**: users can upload their own `~/.claude/.credentials.json` file to use a specific account (takes priority over OAuth2-derived credentials)
- Admin (config-file user): CLI credentials file path stored in `providers.json` under `claude_config.cli_credentials_file`
- Database users (all roles): CLI credentials stored in `user_oauth2_credentials` with `auth_type='claude_cli_credentials'`
##### Credentials file schema
The CLI expects a `.credentials.json` with this structure:
When "Use Claude CLI mode" is checked without an explicit file upload, AISBF converts the stored OAuth2 tokens to this format automatically (`expires_at` seconds → `expiresAt` milliseconds, `scope` string → `scopes` list).
### Ollama
- Uses direct HTTP API
- No API key required
- Local model hosting support
### Kiro (Amazon Q Developer)
- Native integration with Kiro authentication
- Supports IDE credentials and CLI authentication
- No separate API key required
### Kilocode
- OAuth2 Device Authorization Grant flow
- Supports both API key and OAuth2 authentication
- Dashboard OAuth2 authentication UI
### Codex
- OAuth2 Device Authorization Grant for OpenAI protocol
- API key exchange from ID token
- Dashboard authentication UI
### Qwen
- API key authentication (recommended)
- OpenAI-compatible DashScope API endpoint
- Supports all Qwen models
## Advanced Features
### Provider-Native Caching
AISBF supports provider-native caching APIs for cost reduction:
#### Supported Providers
-**Anthropic**: `cache_control` with ephemeral caching
-**Google**: Context Caching API with TTL support
-**OpenAI**: Automatic prefix caching
#### Configuration
```json
{
"providers":{
"anthropic":{
"enable_native_caching":true,
"min_cacheable_tokens":1024
}
}
}
```
### Response Caching
Intelligent response caching with semantic deduplication:
# AISBF - AI Service Broker Framework || AI Should Be Free
A modular proxy server for managing multiple AI provider integrations with unified API interface. AISBF provides intelligent routing, load balancing, and AI-assisted model selection to optimize AI service usage across multiple providers.
A modular proxy server for managing multiple AI provider integrations with unified API interface. AISBF provides intelligent routing, load balancing, AI-assisted model selection, hosted model marketplaces, and multimodal Studio workflows across local and remote providers.
---
...
...
@@ -19,20 +19,32 @@ Also available via TOR for privacy-first access:
## Key Features
-**Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Claude Code (OAuth2 or CLI), Ollama, Kiro, Kilocode, Codex, and Qwen
-**Claude CLI Mode**: When the `claude` binary is in PATH, requests are proxied through the official Anthropic CLI (`claude -p`) — uses each user's own account, fully permitted by Claude's terms of service
-**Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Claude Code (OAuth2 or CLI), Ollama, Kiro, Kilocode, Codex, Qwen, CoderAI, and RunPod
-**CoderAI Broker Mode**: NAT-friendly outbound WebSocket broker with provider-scoped registration tokens, persisted session metadata, direct dashboard status, and Studio endpoint forwarding
-**RunPod Runtime Management**: Pod-backed, serverless-template, and public-catalog RunPod providers with runtime state persistence, startup polling, idle shutdown, and wrapper-mode delegation to OpenAI, Ollama, or CoderAI
-**AISBF Studio**: Multimodal dashboard workspace for chat, image, video, audio, embedding, and 3D workflows with reusable characters, environments, voices, archives, and custom pipelines
-**Marketplace & References**: Publish providers, models, rotations, and autoselects to a shared market, import them as locked references, and track listing analytics and revenue
-**Claude CLI Mode**: When the `claude` binary is in PATH, requests are proxied through the official Anthropic CLI (`claude -p`) using each user's own account
-**Unified Wallet System**: Fiat wallet with crypto/PayPal/Stripe top-ups and auto top-up for subscription renewals
-**Intelligent Routing**: Weighted load balancing and AI-assisted model selection
-**Streaming Support**: Full support for streaming responses from all providers
-**Web Dashboard**: Complete configuration and management interface
-**Multi-User Support**: Isolated configurations with role-based access control
-**Token Usage Analytics**: Comprehensive analytics with cost estimation and export
-**Token Usage Analytics**: Comprehensive analytics with cost estimation, broker telemetry, and export
-**Adaptive Rate Limiting**: Learns from 429 responses for optimal request rates
-**Provider-Native Caching**: 50-70% cost reduction with Anthropic, Google, and OpenAI caching
-**Context Management**: Automatic condensation with 8+ methods when approaching limits
-**SSL/TLS & TOR**: Built-in HTTPS with Let's Encrypt and TOR hidden service support
-**MCP Server**: Model Context Protocol for remote agent integration
## What's New Since 0.99.65
-**CoderAI broker telemetry and NAT traversal**: Broker sessions now persist state across restarts, expose connection and performance telemetry in the dashboard, and support Studio-native proxying over WebSocket for remote or firewalled workers
-**RunPod provider support**: AISBF can now manage RunPod pods, serverless templates, and public endpoints from the provider editor, including runtime refresh and protocol-aware delegation
-**Marketplace administration**: Added dedicated market admin pages, publishing and settlement flows, user export filters, locked imported references, and improved search and ordering
-**Studio expansion**: Added dashboard Studio bindings, multimodal function routing, reusable profile assets, custom pipelines, and admin/user scoped Studio persistence APIs
-**Operational hardening**: Signup cleanup removes stale self-registered accounts after 14 days of inactivity, dashboard proxy path handling was normalized, and provider model caches are reused before refresh
## Quick Start
### Installation
...
...
@@ -64,10 +76,12 @@ Access the dashboard at `http://localhost:17765/dashboard` (default credentials:
The dashboard provides:
- Provider configuration and API key management
- RunPod runtime controls and CoderAI broker session monitoring
- Rotation and autoselect model setup
- AISBF Studio multimodal workflows and pipeline bindings
- User wallet management and top-up options
- Token usage analytics and cost tracking
-Real-time monitoring and rate limit management
- Token usage analytics, broker telemetry, and cost tracking
-Marketplace publishing, importing, and administration
- SSL/TLS and TOR configuration
- Multi-user administration
...
...
@@ -94,12 +108,20 @@ curl -X POST http://localhost:17765/api/wallet/topup \
@@ -89,6 +89,22 @@ The outbound WebSocket connection must include:
-`username`: either `global` or the AISBF username for user-owned providers
-`registration_token`: provider-scoped secret from AISBF provider configuration
### Current server-side resolution order
AISBF resolves broker identity in this exact order when the WebSocket handshake arrives:
-`provider_id`: query param `provider_id`, then header `x-coderai-provider-id`, then default `coderai`
-`client_id`: query param `client_id`, then header `x-coderai-client-id`, then generated fallback `anon-<unix_timestamp>`
-`username`: query param `username`, then header `x-coderai-username`, then the path scope name (`global` or the `/api/u/{username}` path segment)
-`registration_token`: query param `registration_token`, then header `x-coderai-registration-token`
Important constraints:
- the `registration_token` is required for admission
-`Authorization: Bearer ...` is currently not used by the broker WebSocket admission check
- if you omit `client_id`, AISBF generates an `anon-*` client id and future broker routing will only work if AISBF also targets that exact generated value
- the `client_id` used by the CoderAI client must match the `coderai_config.client_id` used by the AISBF provider, or the broker can show the session as connected while requests still fail to route
## Optional Headers
AISBF also accepts or may expect these headers:
...
...
@@ -109,6 +125,35 @@ Recommended behavior:
Open the outbound WebSocket to the correct scoped AISBF endpoint.
The handshake is a normal WebSocket upgrade request, which starts as an HTTP `GET` carrying query parameters. This is expected.
- send the same identity in both query parameters and headers
- keep `client_id` stable across reconnects
- always reconnect with the same provider scope and owner scope
### 2. Wait for `registered` event
AISBF immediately sends a registration acknowledgment event on successful admission.
...
...
@@ -135,11 +180,21 @@ Store:
-`client_id`
-`username`
-`scope_name`
-`owner_user_id`
-`expires_at`
Notes:
- this event means the socket is admitted and the session row exists
- it does not yet mean hardware/capabilities metadata has been uploaded
- the client should send the explicit `register` operation immediately after this event
### 3. Send explicit `register` operation
After the `registered` event, CoderAI must send a `register` message describing its capabilities, hardware inventory, and advertised endpoints.
AISBF currently processes `register` as a normal inbound WebSocket message and responds with `status=ok` using the same `request_id`.
### 4. Enter long-lived receive loop
Then keep listening for incoming broker requests from AISBF.
...
...
@@ -233,6 +288,60 @@ CoderAI should send this after receiving the initial AISBF `registered` event.
AISBF replies with a success envelope.
### Fields AISBF currently reads from the `register` message
Top-level:
-`v`
-`op` with value `register`
-`request_id`
- optional top-level `registration_token`
- optional top-level `capabilities`
From `payload`:
-`endpoint`
-`transport`
-`registration_token`
-`studio_endpoints`
-`hardware`
-`gpus`
-`gpu_count`
-`total_vram_mb`
-`available_vram_mb`
-`capabilities`
AISBF behavior:
- if `payload.registration_token` or top-level `registration_token` is present and does not match the handshake token, AISBF replies with an error envelope
- if token matches, AISBF persists the metadata onto the broker session
-`payload.capabilities` takes precedence over missing top-level capability data
- if `gpus`, `gpu_count`, `total_vram_mb`, or `available_vram_mb` are omitted at the top level, AISBF falls back to the values inside `payload.hardware`
Minimal acceptable `register` message:
```json
{
"v":1,
"op":"register",
"request_id":"reg-1",
"payload":{
"transport":"websocket",
"registration_token":"<same_registration_token>",
"capabilities":{}
}
}
```
Recommended full `register` message:
- include `endpoint`
- include `transport`
- include `registration_token`
- include `hardware.gpus`, `hardware.gpu_count`, `hardware.total_vram_mb`, `hardware.available_vram_mb`
- include `studio_endpoints`
- include `capabilities`
### Hardware Reporting Requirements
The `register` payload should include the best hardware view available to the running CoderAI process.
...
...
@@ -326,6 +435,37 @@ Heartbeat payloads may also refresh dynamic hardware state such as changing free
}
```
Current AISBF note:
- AISBF acknowledges heartbeat messages and merges the heartbeat `payload` into session metadata
- keep heartbeat payloads small and non-blocking
- use heartbeats for lightweight dynamic updates only; do not block the main receive loop on expensive hardware rescans
## Async Client Requirements
The broker WebSocket integration must be fully asynchronous.
CoderAI client requirements:
- the main receive loop must never block on model loading, inference, GPU inspection, or disk/network I/O
- expensive work should run in background tasks or worker executors while the socket remains responsive to incoming frames and ping/pong traffic
- the client should be able to receive broker requests while also sending progress or result frames for earlier requests
- the client must not serialize all work behind registration or heartbeat handling
AISBF broker behavior:
- AISBF now drains queued outbound broker requests in a background async task while independently reading inbound websocket messages
- this means the CoderAI client should expect inbound requests to arrive even while it is still sending heartbeat or response messages for unrelated work
- operations are correlated strictly by `request_id`; client implementations must not rely on message ordering alone
Recommended client architecture:
1. one async reader task for inbound WebSocket frames
2. one async writer path or send queue for outbound replies/events
3. per-request async tasks for local execution
4. a lightweight periodic heartbeat task
5. explicit request correlation by `request_id`
AISBF merges those updates into the broker session metadata.
@@ -200,6 +200,20 @@ When CoderAI dials AISBF broker directly, it should connect using:
-`provider_id=<provider_id>`
-`client_id=<client_id>`
-`username=<username-or-global>`
-`registration_token=<owner-configured-token>`
AISBF broker admission currently resolves connection data in this order:
-`provider_id`: query param, then `x-coderai-provider-id`, then default `coderai`
-`client_id`: query param, then `x-coderai-client-id`, then generated fallback `anon-<timestamp>`
-`username`: query param, then `x-coderai-username`, then the scoped path value
-`registration_token`: query param, then `x-coderai-registration-token`
Important:
- the WebSocket broker flow starts as an HTTP `GET` upgrade request; this is expected
- the broker currently validates `registration_token`, not `Authorization: Bearer ...`, during WebSocket admission
-`client_id` must stay stable and match the `coderai_config.client_id` AISBF uses for that provider, or requests like `models.list` may fail even if the dashboard shows the session as connected
If `payload.registration_token` or top-level `registration_token` is present and does not match the handshake token, AISBF replies with an error envelope.
### Envelope format
AISBF sends one JSON request envelope per operation:
...
...
@@ -314,6 +392,17 @@ Important:
- This keeps AISBF transport-simple and lets CoderAI own protocol correctness.
- Include `data: [DONE]\n\n` as one of the streamed chunks when the upstream semantics require it.
## Async broker behavior requirements
The broker client must be fully asynchronous.
- do not block the WebSocket receive loop on hardware probing, model loading, inference, file I/O, or reconnect bookkeeping
- handle inbound requests concurrently and correlate replies by `request_id`
- keep heartbeat handling lightweight
- preserve responsiveness to ping/pong traffic while local work is running
AISBF now independently drains queued outbound broker requests while also reading inbound WebSocket messages, so client implementations should not assume a strict request/response lockstep over the socket.
## Broker session visibility, persistence, and multi-node routing
<strong>Scope note:</strong> this page controls the legacy user-specific native prompt-caching overrides used by provider adapters such as OpenAI-compatible, Claude, and Kilo flows.
The newer request-time response-cache policy now lives in the dashboard feature overrides under <strong>Response Cache</strong>.
<pstyle="color: var(--color-muted); margin: 0; font-size: 14px;">Enable or disable provider-native prompt caching overrides for all your providers and models.</p>
<small style="color: var(--color-muted); display: block; margin-bottom: 15px;">Per-provider tri-state overrides. Inherit follows the global default.</small>
${triStateSelect('Context Condensation',provider.enable_context_condensation,`updateProviderTriState('${key}', 'enable_context_condensation', this.value)`,'Controls whether condensation is active for this provider.')}
${triStateSelect('Response Cache',provider.enable_response_cache,`updateProviderTriState('${key}', 'enable_response_cache', this.value)`,'Overrides the global response-cache default for this provider.')}
${triStateSelect('Prompt Batching',provider.enable_prompt_batching,`updateProviderTriState('${key}', 'enable_prompt_batching', this.value)`,'Overrides batching behavior for this provider.')}
${triStateSelect('Prompt Security',provider.enable_prompt_security,`updateProviderTriState('${key}', 'enable_prompt_security', this.value)`,'Runs local prompt scanning before provider execution.')}
${triStateSelect('Context Lens Analytics',provider.enable_context_lens,`updateProviderTriState('${key}', 'enable_context_lens', this.value)`,'Captures prompt composition analytics for this provider.')}
${triStateSelect('NSFW Classification',provider.enable_nsfw_classification,`updateProviderTriState('${key}', 'enable_nsfw_classification', this.value)`,'Controls NSFW classification checks for this provider.')}
${triStateSelect('Privacy Classification',provider.enable_privacy_classification,`updateProviderTriState('${key}', 'enable_privacy_classification', this.value)`,'Controls privacy classification checks for this provider.')}
<divclass="section-title"><iclass="fas fa-filter"></i><spandata-i18n="settings_page.section_classification">Features & Prompt Analysis</span></div>
<pstyle="color: var(--color-muted); margin-bottom: 18px;">Global defaults for legacy classifiers, execution features, and prompt analysis / security controls.</p>
<divclass="form-group">
<label>
...
...
@@ -521,6 +535,39 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
</label>
<smallstyle="color: var(--color-subtle); display: block; margin-top: 5px;">Enable semantic content classification using the configured model</small>
</div>
<divclass="section-title"style="margin-top:28px;"><iclass="fas fa-sliders-h"></i> Global Feature Defaults</div>
<pstyle="color:var(--color-muted);margin-bottom:16px;font-size:.9em;">These tri-state defaults drive inheritance for provider, model, rotation, and autoselect overrides.</p>
{% set feature_controls = config.feature_controls if config.feature_controls else none %}
{% set prompt_security = feature_controls.prompt_security if feature_controls and feature_controls.prompt_security else none %}
<divclass="tri-grid">
{{ tri_state_select('feature_nsfw_classification_mode', feature_controls.nsfw_classification.mode if feature_controls and feature_controls.nsfw_classification else 'inherit', 'NSFW Classification', 'Default NSFW classification behavior.') }}
{{ tri_state_select('feature_privacy_classification_mode', feature_controls.privacy_classification.mode if feature_controls and feature_controls.privacy_classification else 'inherit', 'Privacy Classification', 'Default privacy classification behavior.') }}
{{ tri_state_select('feature_context_condensation_mode', feature_controls.context_condensation.mode if feature_controls and feature_controls.context_condensation else 'inherit', 'Context Condensation', 'Default context condensation behavior.') }}
{{ tri_state_select('feature_response_cache_mode', feature_controls.response_cache.mode if feature_controls and feature_controls.response_cache else 'inherit', 'Response Cache', 'Default response-cache behavior.') }}
{{ tri_state_select('feature_prompt_batching_mode', feature_controls.prompt_batching.mode if feature_controls and feature_controls.prompt_batching else 'inherit', 'Prompt Batching', 'Default batching behavior.') }}
{{ tri_state_select('feature_prompt_security_mode', prompt_security.security_scan.mode if prompt_security and prompt_security.security_scan else 'inherit', 'Prompt Security', 'Default local prompt scanning behavior.') }}
{{ tri_state_select('feature_context_lens_mode', prompt_security.context_lens.mode if prompt_security and prompt_security.context_lens else 'inherit', 'Context Lens Analytics', 'Default prompt composition analytics behavior.') }}
{{ tri_state_select('feature_block_high_risk_prompts_mode', prompt_security.block_high_risk_prompts.mode if prompt_security and prompt_security.block_high_risk_prompts else 'inherit', 'Block High-Risk Prompts', 'Default local high-risk blocking behavior.') }}
<pstyle="color:var(--color-muted);margin-bottom:16px;font-size:.9em;">Controls how prompt analysis findings are stored when prompt-security features are active.</p>
<divclass="tri-grid">
{{ tri_state_select('feature_persist_prompt_text_mode', prompt_security.persist_prompt_text.mode if prompt_security and prompt_security.persist_prompt_text else 'inherit', 'Persist Prompt Text', 'Store prompt text with prompt-analysis records when allowed.') }}
{{ tri_state_select('feature_redact_before_persist_mode', prompt_security.redact_before_persist.mode if prompt_security and prompt_security.redact_before_persist else 'inherit', 'Redact Before Persist', 'Redact sensitive fragments before storing prompt-analysis records.') }}
<smallstyle="color: var(--color-subtle); display: block; margin-top: 5px;">Minimum risk level that should be treated as high-risk for blocking and alerting.</small>
