# AISBF - AI Service Broker Framework || AI Should Be Free

## AI.PROMPT Management

This AI.PROMPT file is automatically updated when significant changes are made to the project structure, installation process, or key functionality. The file should be updated when:

1. **Module Structure Changes**: When files are moved between directories or new modules are added
2. **Installation Process Changes**: When setup.py, installation scripts, or installation documentation is modified
3. **Configuration Management Changes**: When config.py, configuration file locations, or configuration behavior is modified
4. **Key Functionality Changes**: When major features are added or existing functionality is significantly modified
5. **Command Line Interface Changes**: When the aisbf script commands or behavior are modified
6. **Development vs Production Workflow Changes**: When the development/production distinction or workflows are modified
7. **PyPI Packaging Changes**: When packaging configuration, build scripts, or PyPI-related files are modified

### How to Update AI.PROMPT

When making changes that affect the project structure or functionality:

1. **Identify the scope of changes** - Determine which sections of AI.PROMPT need updating
2. **Update relevant sections** - Modify the appropriate parts of the documentation
3. **Add new sections if needed** - Create new documentation for new features
4. **Remove outdated information** - Delete sections that are no longer relevant
5. **Test the changes** - Verify that the documentation accurately reflects the current state
6. **Update the AI.PROMPT Management section** - Add notes about what was changed and why

### Common Update Scenarios

- **Moving files**: Update the Directory Structure and Module Structure sections
- **Changing installation**: Update the Installation Process and Key Functions sections
- **Modifying configuration**: Update the Configuration Management section
- **Adding new commands**: Update the AISBF Script Commands section
- **Changing workflows**: Update the Development vs Production section

### Version Control

AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.

## Project Overview

## Project Overview

AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Ollama) with support for provider rotation and error tracking.

## Directory Structure

```
aisbf/
├── aisbf/                    # Main Python module
│   ├── __init__.py          # Module initialization with exports
│   ├── config.py            # Configuration management
│   ├── models.py            # Pydantic models
│   ├── handlers.py          # Request handlers
│   ├── auth/                # Authentication modules
│   │   ├── __init__.py      # Auth module exports
│   │   ├── kiro.py          # Kiro auth manager
│   │   ├── claude.py        # Claude OAuth2 auth (was claude_auth.py)
│   │   └── kilo.py          # Kilo OAuth2 auth (was kilo_oauth2.py)
│   └── providers/           # Provider handlers (split into individual modules)
│       ├── __init__.py      # Re-exports, PROVIDER_HANDLERS, get_provider_handler()
│       ├── base.py          # BaseProviderHandler, AnthropicFormatConverter, AdaptiveRateLimiter
│       ├── google.py        # GoogleProviderHandler
│       ├── openai.py        # OpenAIProviderHandler
│       ├── anthropic.py     # AnthropicProviderHandler
│       ├── claude.py        # ClaudeProviderHandler (OAuth2-based)
│       ├── kilo.py          # KiloProviderHandler (OAuth2-based, supports Kilocode)
│       ├── ollama.py        # OllamaProviderHandler
│       └── kiro/            # Kiro provider (subpackage, supports kiro-cli)
│           ├── __init__.py
│           ├── handler.py
│           ├── converters.py
│           ├── converters_openai.py
│           ├── models.py
│           ├── parsers.py
│           └── utils.py
├── config/                   # Configuration files directory
│   ├── providers.json       # Default provider configurations
│   ├── rotations.json       # Default rotation configurations
│   └── autoselect.json      # Default autoselect configurations
├── main.py                   # FastAPI application entry point
├── setup.py                  # Installation script with custom install logic
├── pyproject.toml            # Modern packaging configuration for PyPI
├── MANIFEST.in               # Package manifest for distribution
├── build.sh                  # Build script for PyPI packages
├── clean.sh                  # Clean script for build artifacts
├── start_proxy.sh           # Development start script
├── aisbf.sh                 # Production start script with venv support
├── requirements.txt         # Python dependencies
├── PYPI.md                 # PyPI publishing guide
├── DOCUMENTATION.md          # Complete API documentation
├── README.md                # Project documentation
├── CHANGELOG.md             # Version history
└── AI.PROMPT                # AI-assisted development guidelines
```

## Module Structure

### aisbf/__init__.py
Exports all main components:
- `config`, `Config`, `ProviderConfig`, `RotationConfig`, `AppConfig`
- `Message`, `ChatCompletionRequest`, `ChatCompletionResponse`, `Model`, `Provider`, `ErrorTracking`
- `BaseProviderHandler`, `GoogleProviderHandler`, `OpenAIProviderHandler`, `AnthropicProviderHandler`, `OllamaProviderHandler`, `get_provider_handler`, `PROVIDER_HANDLERS`
- `RequestHandler`, `RotationHandler`, `AutoselectHandler`

### aisbf/config.py
Configuration management with smart file location detection:
- Checks installed location first (`/usr/share/aisbf/` or `~/.local/share/aisbf/`)
- Falls back to source tree `config/` directory for development
- Creates `~/.aisbf/` directory on first run
- Copies default configs to user directory
- Reads from `~/.aisbf/` on subsequent runs
- Contains shared tri-state feature resolution for request-time controls such as context condensation, response cache, prompt batching, prompt security, Context Lens analytics, and NSFW/privacy classification

### aisbf/models.py
Pydantic models for data validation:
- `Message` - Chat message structure
- `ChatCompletionRequest` - Request model
- `ChatCompletionResponse` - Response model
- `Model` - Model information
- `Provider` - Provider information
- `ErrorTracking` - Error tracking data

### aisbf/providers/ (package)
Provider handler implementations, split into individual modules:
- `base.py` - `BaseProviderHandler`, `AnthropicFormatConverter`, `AdaptiveRateLimiter`, shared utilities
- `google.py` - `GoogleProviderHandler` - Google GenAI integration
- `openai.py` - `OpenAIProviderHandler` - OpenAI API integration
- `anthropic.py` - `AnthropicProviderHandler` - Anthropic API key integration
- `claude.py` - `ClaudeProviderHandler` - Claude OAuth2 integration (fully working)
- `kilo.py` - `KiloProviderHandler` - Kilo OAuth2/Device Authorization Grant integration
- `ollama.py` - `OllamaProviderHandler` - Ollama local integration
- `kiro/` - `KiroProviderHandler` - Kiro/Amazon Q Developer integration (subpackage, supports kiro-cli)
- `__init__.py` - Re-exports all handlers, `PROVIDER_HANDLERS` dict, `get_provider_handler()` factory

### aisbf/auth/ (package)
Authentication modules:
- `kiro.py` - `KiroAuthManager` for Kiro/Amazon Q Developer auth
- `claude.py` - `ClaudeAuth` for Claude OAuth2 PKCE flow (fully working with token refresh)
- `kilo.py` - `KiloOAuth2` for Kilo Device Authorization Grant (Kilocode support)

### aisbf/handlers.py
Request handling logic:
- `RequestHandler` - Handles chat completions and model lists
- `RotationHandler` - Handles provider rotation
- `AutoselectHandler` - Handles AI-powered model selection
- Reuses shared Studio capability inference for model metadata normalization
- Supports both streaming and non-streaming responses
- Performs native local prompt analysis before upstream execution when enabled, including prompt-security scanning, Context Lens-style prompt composition capture, and optional local blocking for high-risk prompts

### aisbf/prompt_analysis.py
Native prompt security and prompt composition analytics:
- Performs local regex/heuristic scanning inspired by Prompt Sail-style prompt governance
- Builds Context Lens-style composition summaries from messages, tools, prompt/input payloads, and token-role distribution
- Produces redacted evidence previews and hashed request fingerprints for persistence without storing raw sensitive content by default

### aisbf/studio.py
Shared Studio integration helpers:
- Normalizes dashboard-visible provider, rotation, and autoselect resources into a unified Studio catalog
- Infers and merges Studio capability metadata for autodetected and manually configured models
- Supports admin/global and user-owned dashboard catalog scopes
- Carries effective Studio adapter metadata for provider models so Studio workflow proxying can choose provider-aware payload shaping

### aisbf/studio_adapters.py
Studio adapter inference helpers:
- Defines manual/automatic Studio adapter choices for provider models
- Defines provider-specific Studio adapter profile choices layered above adapter family inference
- Infers the best adapter from provider type and model capabilities when no manual override is set
- Infers the best adapter profile from the configured provider identity, endpoint, and model metadata when no manual override is set
- Supplies the effective adapter used by Studio workflow payload normalization
- Contains provider-family-aware Studio payload transformation helpers used by the generic proxy path before requests are forwarded upstream
- Contains explicit profile-specific request shaping for OpenAI-style, OpenRouter-style, Gemini-style, Anthropic-style, and Ollama-style Studio media workflows
- Includes provider-id/endpoint specific profiles such as Kilo/OpenRouter-style, GitHub Models-style, and Azure OpenAI-style request shaping

### aisbf/studio_services.py
Studio persistence and runtime helpers for dashboard Studio:
- Stores characters, environments, voices, archive items, and custom pipelines per Studio scope
- Uses file-backed storage for the config admin scope under `~/.aisbf/studio/`
- Uses database-backed storage for user-owned Studio assets and pipelines
- Stores admin custom pipelines in `~/.aisbf/pipelines.json` as a plain JSON array
- Stores persistent Studio functionality-to-model bindings in `~/.aisbf/studio_bindings.json` for config admin scope and in the user prompt override key `studio_function_bindings` for user scope
- Loads Studio chat system prompt from `STUDIO_SYSTEM.md`, using file-backed admin defaults and database-backed user overrides
- Normalizes Studio profile payloads and pipeline definitions for both admin and user scopes

### main.py
FastAPI application:
- Imports from `aisbf` module
- Defines API endpoints:
  - `GET /` - Root endpoint with provider list
  - `POST /api/{provider_id}/chat/completions` - Chat completions
  - `GET /api/{provider_id}/models` - List models
- Supports both providers and rotations
- Multi-user support with user-specific API endpoints
- CORS enabled for all origins

## Installation Process

### Installation from PyPI (Recommended)
```bash
pip install aisbf
```

### User Installation from Source (no root required)
```bash
python setup.py install
```
Automatically adds `--user` flag for non-root users.

**Installation Structure:**
- Package files installed to: `~/.local/share/aisbf/aisbf/` (preserves subdirectory structure)
- Config files and main.py: `~/.local/share/aisbf/`
- Virtual environment: `~/.local/share/aisbf/venv/`
- Requirements installed only on first venv creation

**Note:** Dependencies from requirements.txt are only installed when creating the virtual environment for the first time. Subsequent starts do not re-run pip install to avoid unnecessary upgrades. The venv is only recreated if it doesn't exist.

### System-wide Installation from Source (requires root)
```bash
sudo python setup.py install
```

Installs to:
- `/usr/local/lib/python*/dist-packages/aisbf/` - Package
- `/usr/share/aisbf/venv/` - Virtual environment
- `/usr/share/aisbf/` - Config files and main.py
- `/usr/local/bin/aisbf` - Executable script

## Configuration Management

### Configuration File Locations

**Installed Configuration Files (read-only defaults):**
- User: `~/.local/share/aisbf/providers.json`, `~/.local/share/aisbf/rotations.json`, `~/.local/share/aisbf/main.py`
- System: `/usr/share/aisbf/providers.json`, `/usr/share/aisbf/rotations.json`, `/usr/share/aisbf/main.py`

**User Configuration Files (writable):**
- `~/.aisbf/providers.json` - Provider configurations
- `~/.aisbf/rotations.json` - Rotation configurations
- `~/.aisbf/autoselect.json` - Autoselect configurations
- `~/.aisbf/aisbf.json` - Global AISBF settings, including `feature_controls`

**Development Mode:**
- `config/providers.json` and `config/rotations.json` in source tree

### First Run Behavior
1. Checks for config files in installed location
2. Creates `~/.aisbf/` directory if needed
3. Copies default configs from installed location to `~/.aisbf/`
4. Loads configuration from `~/.aisbf/` on subsequent runs

### Feature Control Hierarchy
- Shared request-time features use tri-state inheritance: `inherit`, `enabled`, `disabled`
- Global defaults live in `aisbf.json` under `feature_controls`
- Provider/model/rotation/autoselect configs may override global defaults with `enable_*` or related boolean override fields
- Current tri-state-controlled feature families include:
  - context condensation
  - response cache
  - prompt batching
  - prompt security scanning
  - Context Lens prompt analytics
  - NSFW classification
  - privacy classification
  - high-risk prompt blocking

### Prompt Analysis Persistence
- Native prompt analysis results are stored in the configuration database using:
  - `prompt_analysis_runs`
  - `prompt_analysis_findings`
- `/dashboard/analytics` now includes Prompt Analysis sections for both admin and user scopes
- Visitor/activity analytics remain config-admin-only, while prompt-analysis summaries can be shown in filtered user scope

## AISBF Script Commands

The installed `aisbf` script supports:

### Starting in Foreground (Default)
```bash
aisbf
```
Starts server in foreground with visible output.

### Starting as Daemon
```bash
aisbf daemon
```
- Starts in background
- Saves PID to `/tmp/aisbf.pid`
- Redirects output to `/dev/null`
- Prints PID of started process

### Checking Status
```bash
aisbf status
```
Checks if AISBF is running and reports status/PID.

### Stopping the Daemon
```bash
aisbf stop
```
Stops running daemon and removes PID file.

## Development vs Production

### Development
Use `start_proxy.sh`:
- Creates local venv in `./venv/`
- Installs dependencies from `requirements.txt`
- Starts server with auto-reload enabled
- Uses `config/` directory for configuration

### Production
Install with `python setup.py install` or `pip install aisbf`:
- Creates isolated venv
- Installs all dependencies
- Provides `aisbf` command with daemon support
- Uses installed config files and main.py from share directory

## PyPI Packaging

### Build Scripts

#### build.sh
Automated build script for creating PyPI distribution packages:
- Checks for build tools (build, twine)
- Installs them if not present
- Cleans previous build artifacts
- Builds package using `python -m build`
- Verifies package with `twine check`
- Displays created files and upload instructions

Usage:
```bash
./build.sh
```

#### clean.sh
Comprehensive cleanup script for removing build artifacts:
- Removes `dist/`, `build/`, `*.egg-info` directories
- Removes Python cache directories and compiled files
- Removes test and coverage artifacts

Usage:
```bash
./clean.sh
```

### Package Files

- **pyproject.toml**: Modern packaging configuration with project metadata
- **MANIFEST.in**: Package manifest specifying files to include in distribution
- **setup.py**: Installation script with custom install logic (venv creation, file copying)
- **PYPI.md**: Comprehensive PyPI publishing guide

### Build Artifacts (Ignored by Git)

The following files and directories are created during build and should not be committed:
- `dist/` - Distribution packages (*.tar.gz, *.whl)
- `build/` - Build artifacts
- `*.egg-info/` - Package metadata directories
- `__pycache__/` - Python bytecode cache
- `*.pyc`, `*.pyo`, `*.pyd` - Compiled Python files

### Publishing Workflow

1. Build the package: `./build.sh`
2. Test on TestPyPI: `python -m twine upload --repository testpypi dist/*`
3. Install from TestPyPI: `pip install --index-url https://test.pypi.org/simple/ aisbf`
4. Publish to PyPI: `python -m twine upload dist/*`

See [`PYPI.md`](PYPI.md) for detailed instructions.

## Important Conventions

1. **Module Imports**: Always use relative imports within the `aisbf` module (e.g., `from .config import config`)
2. **Configuration**: Never hardcode paths; use the config system to locate files
3. **Error Handling**: Provider handlers track failures and implement rate limiting
4. **Installation**: Always use `python setup.py install` for proper setup
5. **Virtual Environment**: The installed `aisbf` script uses the venv created during installation
6. **PID File**: Daemon mode uses `/tmp/aisbf.pid` for process tracking

## Key Functions

### aisbf/config.py
- `Config._get_config_source_dir()` - Returns appropriate config directory
- `Config._ensure_config_directory()` - Creates user config directory
- `Config._load_providers()` - Loads provider configurations
- `Config._load_rotations()` - Loads rotation configurations

### aisbf/providers.py
- `get_provider_handler(provider_id, api_key)` - Factory function for provider handlers
- Provider handlers implement `handle_request()` and `get_models()` methods

### aisbf/handlers.py
- `RequestHandler.handle_chat_completion()` - Non-streaming chat completion
- `RequestHandler.handle_streaming_chat_completion()` - Streaming chat completion
- `RequestHandler.handle_model_list()` - List available models
- `RotationHandler.handle_rotation_request()` - Handle rotation requests
- `RotationHandler.handle_rotation_model_list()` - List rotation models

### main.py
- `main()` - Entry point for the application
- FastAPI endpoints defined as async functions

## Dependencies

Key dependencies from requirements.txt:
- fastapi - Web framework
- uvicorn - ASGI server
- pydantic - Data validation
- httpx - HTTP client
- google-genai - Google AI SDK
- openai - OpenAI SDK
- anthropic - Anthropic SDK

## Testing

When making changes:
1. Test with `start_proxy.sh` for development
2. Install with `python setup.py install` for production testing
3. Test all `aisbf` script commands (default, daemon, status, stop)
4. Verify configuration file locations and behavior
5. Test both user and system installations

## Common Tasks

### Adding a New Provider
1. Create a new file `aisbf/providers/<provider_name>.py` with a handler class inheriting from `BaseProviderHandler`
2. Import and add the handler to `PROVIDER_HANDLERS` in `aisbf/providers/__init__.py`
3. Add provider configuration to `config/providers.json`

### Configuration Architecture

**API Key Management:**
- API keys are stored centrally in `config/providers.json`
- Each provider definition includes an `api_key` field
- Rotation and autoselect configurations reference providers by name only
- The system automatically resolves API keys by matching provider names

**Priority Order for API Keys:**
1. API key from provider config (providers.json) - highest priority
2. API key from rotation config (rotations.json) - fallback for backward compatibility

**Model Selection in Rotations:**
- If models are specified in rotation config: uses those models
- If no models specified: randomly selects from provider's available models
- Models from provider config are used with default weight of 1

**Default Settings Hierarchy:**
Settings can be specified at three levels with the following priority:
1. Model-specific settings (highest priority)
2. Rotation default settings
3. Provider default settings (lowest priority)

**Supported Default Fields:**
- `default_rate_limit`: Rate limiting between requests
- `default_max_request_tokens`: Maximum tokens per request
- `default_rate_limit_TPM`: Tokens per minute limit
- `default_rate_limit_TPH`: Tokens per hour limit
- `default_rate_limit_TPD`: Tokens per day limit
- `default_context_size`: Context window size
- `default_condense_context`: Context condensation threshold
- `default_condense_method`: Context condensation method
- `default_error_cooldown`: Cooldown period in seconds after 3 consecutive failures (default: 300)

### Kiro Gateway Integration

**Overview:**
Kiro Gateway is a third-party proxy gateway that provides OpenAI and Anthropic-compatible APIs for Kiro (Amazon Q Developer / AWS CodeWhisperer). It's available as a separate GitHub project and has been integrated as a provider type in AISBF.

**What is Kiro Gateway:**
- Proxy gateway for accessing Claude models through Kiro IDE/CLI credentials
- Provides OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/models`)
- Supports both Kiro IDE and kiro-cli authentication methods
- Offers access to Claude models (Sonnet 4.5, Haiku 4.5, Opus 4.5, etc.)
- Includes features like extended thinking, tool calling, and streaming

**Integration Architecture:**
- `KiroProviderHandler` class in `aisbf/providers.py` - treats kiro-gateway as an OpenAI-compatible endpoint
- Uses the OpenAI SDK to communicate with kiro-gateway
- Supports all standard AISBF features: streaming, tools, rate limiting, error tracking

**Configuration:**

**IMPORTANT:** Kiro providers use a different configuration structure than other providers. Instead of an `api_key` field, they use a `kiro_config` object with authentication settings specific to Kiro/Amazon Q Developer.

**Standard Provider Configuration (Google, OpenAI, Anthropic, etc.):**
```json
{
  "gemini": {
    "id": "gemini",
    "name": "Google AI Studio",
    "endpoint": "https://generativelanguage.googleapis.com/v1beta",
    "type": "google",
    "api_key_required": true,
    "api_key": "YOUR_GEMINI_API_KEY",
    "rate_limit": 0,
    "default_rate_limit": 0,
    "default_max_request_tokens": 1000000,
    "default_context_size": 1000000,
    "models": [
      {
        "name": "gemini-2.0-flash",
        "rate_limit": 0,
        "max_request_tokens": 1000000,
        "context_size": 1000000
      }
    ]
  }
}
```

**Kiro Provider Configuration (uses kiro_config instead of api_key):**
```json
{
  "kiro": {
    "id": "kiro",
    "name": "Kiro (Amazon Q Developer)",
    "endpoint": "https://q.us-east-1.amazonaws.com",
    "type": "kiro",
    "api_key_required": false,
    "rate_limit": 0,
    "kiro_config": {
      "region": "us-east-1",
      "creds_file": "~/.config/Code/User/globalStorage/amazon.q/credentials.json",
      "sqlite_db": "",
      "refresh_token": "",
      "profile_arn": "",
      "client_id": "",
      "client_secret": ""
    },
    "models": [
      {
        "name": "anthropic.claude-3-5-sonnet-20241022-v2:0",
        "rate_limit": 0,
        "max_request_tokens": 200000,
        "context_size": 200000
      }
    ]
  }
}
```

**Kiro Configuration Fields (kiro_config):**
- `region`: AWS region (default: "us-east-1")
- `creds_file`: Path to Kiro IDE credentials JSON file (Option 1)
- `sqlite_db`: Path to kiro-cli SQLite database (Option 2)
- `refresh_token`: Direct refresh token for authentication (Option 3)
- `profile_arn`: AWS CodeWhisperer profile ARN (optional)
- `client_id`: OAuth client ID for AWS SSO OIDC (used with refresh_token)
- `client_secret`: OAuth client secret for AWS SSO OIDC (used with refresh_token)

Choose one authentication method:
1. **Kiro IDE credentials**: Set `creds_file` to the path of your Kiro IDE credentials
2. **kiro-cli database**: Set `sqlite_db` to the path of your kiro-cli SQLite database
3. **Direct credentials**: Set `refresh_token` and optionally `client_id`/`client_secret` for AWS SSO OIDC

In `config/rotations.json` (API keys are now referenced from providers.json):
```json
{
  "coding": {
    "model_name": "coding",
    "notifyerrors": false,
    "default_rate_limit": 0,
    "default_context_size": 100000,
    "providers": [
      {
        "provider_id": "gemini",
        "models": [
          {
            "name": "gemini-2.0-flash",
            "weight": 3,
            "rate_limit": 0
          }
        ]
      },
      {
        "provider_id": "openai"
      }
    ]
  },
  "kiro-claude": {
    "model_name": "kiro-claude",
    "providers": [
      {
        "provider_id": "kiro",
        "models": [
          {
            "name": "claude-sonnet-4-5",
            "weight": 3
          }
        ]
      }
    ]
  }
}
```

**Note:** In the example above, the "openai" provider entry has no models specified, so the system will randomly select from all models defined in the provider's configuration.

**Setup Requirements:**
1. Kiro Gateway must be running (typically on `http://localhost:8000`)
2. Kiro Gateway must be configured with valid Kiro credentials (IDE or CLI)
3. Set `PROXY_API_KEY` in kiro-gateway's `.env` file
4. Use that same API key in AISBF's provider configuration

**Available Models:**
- `claude-sonnet-4-5` - Enhanced model, balanced performance
- `claude-haiku-4-5` - Fast model for quick responses
- `claude-opus-4-5` - Top-tier model (may require paid tier)
- `claude-sonnet-4` - Previous generation model
- `auto` - Automatic model selection

**Usage:**
Once configured, kiro-gateway can be used like any other provider in AISBF:
- Direct provider access: `/api/kiro/chat/completions`
- Rotation access: `/api/kiro-claude/chat/completions`
- Model listing: `/api/kiro/models`

**Benefits:**
- Access Claude models through Kiro credentials without direct Anthropic API access
- Leverage Kiro's free tier or paid plans
- Use Claude models in AISBF rotations alongside other providers
- Automatic failover and load balancing with other providers

### Claude Code Provider Integration

**Overview:**
Claude Code is an OAuth2-based authentication method for accessing Claude models through the official Anthropic API with subscription-based access. It uses the same authentication flow as the official claude-cli tool.

**What is Claude Code:**
- OAuth2 authentication for Claude API access
- Uses subscription-based access (claude-code)
- Compatible with claude-cli credentials
- Provides access to latest Claude models (3.7 Sonnet, 3.5 Sonnet, 3.5 Haiku, etc.)
- Supports all Claude features: streaming, tools, vision, extended thinking

**Integration Architecture:**
- [`ClaudeAuth`](aisbf/auth/claude.py) class handles OAuth2 PKCE flow
- [`ClaudeProviderHandler`](aisbf/providers/claude.py) in [`aisbf/providers/claude.py`](aisbf/providers/claude.py) manages API requests
- Supports all standard AISBF features: streaming, tools, rate limiting, error tracking

**Configuration:**

**IMPORTANT:** Claude providers use OAuth2 authentication instead of API keys. The `claude_config` object contains authentication settings.

**Claude Provider Configuration:**
```json
{
  "claude": {
    "id": "claude",
    "name": "Claude Code (OAuth2)",
    "endpoint": "https://api.anthropic.com/v1",
    "type": "claude",
    "api_key_required": false,
    "rate_limit": 0,
    "claude_config": {
      "credentials_file": "~/.claude_credentials.json"
    },
    "models": [
      {
        "name": "claude-3-7-sonnet-20250219",
        "rate_limit": 0,
        "max_request_tokens": 200000,
        "context_size": 200000
      }
    ]
  }
}
```

**Claude Configuration Fields (claude_config):**
- `credentials_file`: Path to OAuth2 credentials file (default: `~/.claude_credentials.json`)

**Authentication Flow:**
1. First request triggers OAuth2 flow if no credentials exist
2. Browser opens to https://claude.ai for authentication
3. User logs in with Claude account
4. Authorization code exchanged for access token
5. Credentials saved to file for future use
6. Automatic token refresh when expired

**Setup Requirements:**
1. Claude subscription (claude-code access)
2. Web browser for initial authentication
3. Port 54545 available for OAuth callback

**Available Models:**
- `claude-3-7-sonnet-20250219` - Latest Claude 3.7 Sonnet
- `claude-3-5-sonnet-20241022` - Claude 3.5 Sonnet
- `claude-3-5-haiku-20241022` - Claude 3.5 Haiku (fast)
- `claude-3-opus-20240229` - Claude 3 Opus (top-tier)

**Usage:**
Once configured, claude provider can be used like any other provider in AISBF:
- Direct provider access: `/api/claude/chat/completions`
- Rotation access: `/api/claude-code/chat/completions`
- Model listing: `/api/claude/models`

**Benefits:**
- Access Claude models through OAuth2 subscription
- No need to manage API keys
- Automatic token refresh
- Use Claude models in AISBF rotations alongside other providers
- Automatic failover and load balancing with other providers

### Qwen Provider Integration

**Overview:**
Qwen is Alibaba Cloud's large language model service that provides OAuth2-based authentication for accessing Qwen models through the DashScope OpenAI-compatible API. It's integrated as a provider type in AISBF.

**What is Qwen:**
- OAuth2 Device Authorization Grant with PKCE for secure authentication
- OpenAI-compatible API endpoint through DashScope
- Provides access to Qwen models (Qwen Plus, Qwen Turbo, Qwen Max, Coder Model)
- Supports streaming, tool calling, and standard chat completions
- Free tier available with quota limits (1,000 requests/day, 60 requests/minute)

**Integration Architecture:**
- [`QwenOAuth2`](aisbf/auth/qwen.py) class handles OAuth2 Device Authorization Grant with PKCE
- [`QwenProviderHandler`](aisbf/providers/qwen.py) manages API requests using OpenAI SDK
- Supports all standard AISBF features: streaming, tools, rate limiting, error tracking
- Automatic token refresh with cross-process synchronization

**Configuration:**

**IMPORTANT:** Qwen providers use OAuth2 authentication instead of API keys. The `qwen_config` object contains authentication settings. Optionally, an API key can be provided to bypass OAuth2.

**Qwen Provider Configuration:**
```json
{
  "qwen": {
    "id": "qwen",
    "name": "Qwen (OAuth2)",
    "endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1",
    "type": "qwen",
    "api_key_required": false,
    "rate_limit": 0,
    "qwen_config": {
      "credentials_file": "~/.aisbf/qwen_credentials.json",
      "api_key": ""
    },
    "models": [
      {
        "name": "qwen-plus",
        "rate_limit": 0,
        "max_request_tokens": 32000,
        "context_size": 32000
      }
    ]
  }
}
```

**Qwen Configuration Fields (qwen_config):**
- `credentials_file`: Path to OAuth2 credentials file (default: `~/.aisbf/qwen_credentials.json`)
- `api_key`: Optional API key to bypass OAuth2 (leave empty to use OAuth2)
- `region`: Region for API key authentication (china-beijing, singapore, us-virginia, china-hongkong, germany-frankfurt)
- `workspace_id`: Workspace ID for Germany region (default: "Default Workspace")

**Authentication Flow:**
1. First request triggers OAuth2 Device Authorization flow if no credentials exist
2. User visits verification URL and enters user code
3. Authorization code exchanged for access token with PKCE
4. Credentials saved to file for future use
5. Automatic token refresh when expired (30-second buffer)
6. Cross-process token synchronization with file locking

**Setup Requirements:**
1. Qwen account (free tier available)
2. Web browser for initial authentication
3. Internet connection for OAuth2 flow

**Available Models:**

*OAuth2 Authentication (Fixed Model List):*
- `coder-model` - Specialized coding model with 1M context (fixed for OAuth2 auth)

*API Key Authentication (Dynamic Model List):*
- `qwen-plus` - Enhanced model with 32K context
- `qwen-turbo` - Fast model for quick responses
- `qwen-max` - Top-tier model with advanced capabilities
- `coder-model` - Specialized coding model (maps to qwen3.6-plus)
- Additional models fetched from DashScope API endpoint

**Usage:**
Once configured, qwen provider can be used like any other provider in AISBF:
- Direct provider access: `/api/qwen/chat/completions`
- Rotation access: `/api/qwen-rotation/chat/completions`
- Model listing: `/api/qwen/models`

**Benefits:**
- Access Qwen models through OAuth2 subscription or API key
- No need to manage API keys manually (OAuth2 mode)
- Automatic token refresh
- Use Qwen models in AISBF rotations alongside other providers
- Automatic failover and load balancing with other providers

**Authentication Behavior:**

**OAuth2 Mode (Default - No API Key):**
- Uses OAuth2 Device Authorization Grant with PKCE
- Model list is fixed to only `coder-model` with 1M context tokens
- Endpoint: `https://dashscope.aliyuncs.com/compatible-mode/v1`
- Requires browser-based authentication flow
- Automatic token refresh

**API Key Mode (Optional):**
- Set `api_key` in `qwen_config` to use API key authentication
- Model list is fetched dynamically from DashScope `/models` endpoint
- If models are defined in provider configuration, uses those instead
- No OAuth2 authentication flow required
- Obtain API key from Alibaba Cloud DashScope console

**Region-Based Endpoints (API Key Mode):**
- `china-beijing`: `https://dashscope.aliyuncs.com/compatible-mode/v1` (default)
- `singapore`: `https://dashscope-intl.aliyuncs.com/compatible-mode/v1`
- `us-virginia`: `https://dashscope-us.aliyuncs.com/compatible-mode/v1`
- `china-hongkong`: `https://cn-hongkong.dashscope.aliyuncs.com/compatible-mode/v1`
- `germany-frankfurt`: `https://{workspace_id}.eu-central-1.maas.aliyuncs.com/compatible-mode/v1`

### Modifying Configuration
1. Edit files in `~/.aisbf/` for user-specific changes
2. Edit files in installed location for system-wide defaults
3. Restart AISBF for changes to take effect

### Updating Installation
1. Modify `setup.py` for installation changes
2. Reinstall with `python setup.py install`
3. Test all functionality

## File Naming Conventions

### Adding License Headers

When creating new scripts or Python source files, always include the following license header at the top of the file:

```python
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>

AISBF - AI Service Broker Framework || AI Should Be Free

[File description]

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

Why did the programmer quit his job? Because he didn't get arrays!
"""
```

### Adding Script Headers

For shell scripts, include the following header:

```bash
#!/bin/bash
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
```

### File Naming Conventions

- Python modules: lowercase with underscores (e.g., `config.py`, `handlers.py`)
- Configuration files: lowercase with underscores (e.g., `providers.json`)
- Scripts: lowercase with underscores (e.g., `start_proxy.sh`)
- Classes: PascalCase (e.g., `RequestHandler`, `ProviderConfig`)
- Functions: lowercase with underscores (e.g., `get_provider_handler`)

## Important Notes

- The `aisbf` module is designed to be installed as a package
- Configuration files are separated from code for easy customization
- Virtual environment isolation ensures dependency management
- Daemon mode allows for production deployment
- Development mode supports rapid iteration with auto-reload

## AI.PROMPT Management

This AI.PROMPT file is automatically updated when significant changes are made to the project structure, installation process, or key functionality. The file should be updated when:

1. **Module Structure Changes**: When files are moved between directories or new modules are added
2. **Installation Process Changes**: When setup.py, installation scripts, or installation documentation is modified
3. **Configuration Management Changes**: When config.py, configuration file locations, or configuration behavior is modified
4. **Key Functionality Changes**: When major features are added or existing functionality is significantly modified
5. **Command Line Interface Changes**: When the aisbf script commands or behavior are modified
6. **Development vs Production Workflow Changes**: When the development/production distinction or workflows are modified
7. **PyPI Packaging Changes**: When packaging configuration, build scripts, or PyPI-related files are modified
8. **Documentation Updates**: When README.md, DOCUMENTATION.md, or other documentation files are significantly updated

### Recent Updates

**2026-04-11 - Qwen Provider Authentication-Based Model Handling & Region Support**
- Modified QwenProviderHandler.get_models() to handle OAuth2 vs API key authentication differently
- OAuth2 authentication: Returns fixed model list with only "coder-model" (1M context)
- API key authentication: Fetches full model list from DashScope /models endpoint (if no models configured)
- Added region-based endpoint support for API key authentication:
  - Singapore: dashscope-intl.aliyuncs.com
  - US (Virginia): dashscope-us.aliyuncs.com
  - China (Beijing): dashscope.aliyuncs.com
  - China (Hong Kong): cn-hongkong.dashscope.aliyuncs.com
  - Germany (Frankfurt): {workspace_id}.eu-central-1.maas.aliyuncs.com
- Added region and workspace_id configuration fields to qwen_config
- Updated dashboard templates to show region selector and workspace ID field
- Updated _get_sdk_client() and _get_auth_headers() to support both OAuth2 and API key authentication
- Updated AI.PROMPT documentation to reflect authentication-specific model behavior and region configuration
- Fixed aisbf.json not being copied to ~/.aisbf/ directory on first run
- Updated _ensure_config_directory() in aisbf/config.py to include aisbf.json in the list of files to copy
- Previously only providers.json, rotations.json, and autoselect.json were copied
- This caused aisbf.json to remain in ~/.local/share/aisbf/ instead of being accessible in ~/.aisbf/
- Now all configuration files are properly copied to the user's home directory for easy editing
- Version bumped to 0.9.6 in setup.py and pyproject.toml

**2026-04-03 - Version 0.9.4 - Setup.py Fixes and Documentation Updates**
- Fixed setup.py data_files structure to preserve subdirectory structure during installation
- Split data_files into separate entries for each subpackage (aisbf/, aisbf/providers/, aisbf/providers/kiro/, aisbf/auth/)
- Added 6 missing dashboard templates to setup.py (user_index.html, user_providers.html, etc.)
- Removed install_requires from setup.py since pypy creates a virtualenv with its own dependencies
- Updated aisbf.sh to only install requirements on first venv creation
- Provider module refactoring: Split monolithic providers into individual files
- Added Kiro-cli provider support, enhanced Claude Code OAuth2, and Kilocode OAuth2 support
- Updated README.md with Kiro-cli and Kilocode configuration examples
- Removed duplicate entries in documentation (Token Usage Analytics, Claude OAuth2)
- Updated CHANGELOG.md with all new features and fixes
- Version bumped to 0.9.4

**2026-04-03 - Provider Module Refactoring (Phase 2)**
- Split monolithic `aisbf/providers/__init__.py` (301K chars) into individual module files
- Created `aisbf/providers/base.py` with shared utilities: `BaseProviderHandler`, `AnthropicFormatConverter`, `AdaptiveRateLimiter`, `AISBF_DEBUG`
- Created individual handler files: `google.py`, `openai.py`, `anthropic.py`, `claude.py`, `kilo.py`, `ollama.py`
- Moved auth modules: `aisbf/claude_auth.py` → `aisbf/auth/claude.py`, `aisbf/kilo_oauth2.py` → `aisbf/auth/kilo.py`
- Updated `aisbf/auth/__init__.py` to export `ClaudeAuth` and `KiloOAuth2`
- Rewrote `aisbf/providers/__init__.py` as slim re-export module with `PROVIDER_HANDLERS` dict and `get_provider_handler()` factory
- Updated `aisbf/__init__.py` to import new handler classes (`ClaudeProviderHandler`, `KiloProviderHandler`)
- Updated `pyproject.toml` and `setup.py` with new file paths
- All existing `from aisbf.providers import X` imports continue to work (backward compatible)
- Deleted old `aisbf/claude_auth.py` and `aisbf/kilo_oauth2.py` (now in `aisbf/auth/`)

**2026-03-23 - TOR Hidden Service Support**
- Added full TOR hidden service support for exposing AISBF over TOR network
- Created aisbf/tor.py module with TorHiddenService class for managing TOR connections
- Added TorConfig model to config.py for TOR configuration management
- Updated config/aisbf.json with TOR configuration section (enabled, control_port, control_host, control_password, hidden_service_dir, hidden_service_port, socks_port, socks_host)
- Integrated TOR service into main.py with startup/shutdown event handlers
- Added TOR configuration UI to dashboard settings page with real-time status display
- Added get_tor_status MCP tool for monitoring TOR hidden service status (fullconfig access required)
- Updated README.md with comprehensive TOR configuration guide
- Updated DOCUMENTATION.md with detailed TOR setup, troubleshooting, and security considerations
- Added stem library dependency to requirements.txt
- Bumped version to 0.5.0 in pyproject.toml
- Supports both ephemeral (temporary) and persistent (fixed onion address) hidden services
- Dashboard displays TOR status, onion address, and provides configuration interface
- All AISBF endpoints (API, dashboard, MCP) accessible over TOR network
- Automatic TOR service initialization on startup when enabled
- Proper cleanup on shutdown to remove ephemeral services

**2026-03-23 - Configurable Error Cooldown Implementation**
- Added `error_cooldown` field to Model class in models.py for model-specific cooldown configuration
- Added `default_error_cooldown` field to ProviderConfig in config.py for provider-level defaults
- Added `default_error_cooldown` field to RotationConfig in config.py for rotation-level defaults
- Updated BaseProviderHandler.record_failure() to use cascading cooldown configuration (model > provider > system default of 300 seconds)
- Updated DOCUMENTATION.md with comprehensive error_cooldown configuration examples
- Updated AI.PROMPT with error_cooldown in supported default fields
- Providers can now have customizable cooldown periods after 3 consecutive failures instead of hardcoded 5 minutes
- Configuration follows the same cascading pattern as other settings (context_size, rate_limit, etc.)

**2026-03-23 - Proxy-Awareness Implementation**
- Added ProxyHeadersMiddleware to handle reverse proxy deployments
- Implemented automatic detection of proxy headers (X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port, X-Forwarded-Prefix, X-Forwarded-For)
- Updated all RedirectResponse calls to use proxy-aware url_for() function
- Updated templates to use proxy-aware URL generation
- Added comprehensive nginx proxy configuration examples to DOCUMENTATION.md
- Application now fully supports deployment behind reverse proxies with subpaths
- Dashboard URLs, redirects, and API responses automatically adjust based on proxy configuration

**2026-03-23 - Kiro Provider Dashboard Support**
- Updated providers.html dashboard to show kiro-specific configuration fields
- Kiro providers now display `kiro_config` options (region, creds_file, sqlite_db, refresh_token, profile_arn, client_id, client_secret) instead of standard api_key fields
- Added `updateProviderType()` function to handle switching between kiro and other provider types
- Added `updateKiroConfig()` function to manage kiro-specific configuration fields
- Updated `addProvider()` function to ask for provider type and set appropriate defaults for kiro providers
- Dashboard automatically initializes kiro_config when provider type is changed to "kiro"
- Dashboard automatically removes kiro_config when provider type is changed from "kiro" to another type
- Updated AI.PROMPT documentation to clarify kiro provider configuration differences

**2026-05-10 - Dashboard Studio Integration**
- Added `aisbf/studio.py` to build the dashboard Studio catalog for provider models, rotations, and autoselect entries
- Added `/dashboard/studio` and `/dashboard/studio/catalog` for role-aware Studio access inside the dashboard shell
- Provider model save and autodetect flows now persist normalized Studio capability metadata for manual and inferred models
- Studio catalog entries can expose partial aggregate capabilities so incomplete metadata stays visible instead of disappearing

**2026-05-11 - Dashboard Studio Runtime and Persistence Expansion**
- Replaced the partial dashboard Studio shell with the original full Studio UI adapted to dashboard templates, CSS, and proxy-aware routing
- Added scope-aware Studio API bases so the config admin uses `/api/v1` while user-scoped sessions use `/api/u/{username}`
- Restricted global Studio scope to the config admin defined by `aisbf.json`; database-backed admins remain user-scoped
- Added `aisbf/studio_services.py` for Studio asset and pipeline persistence across admin/file and user/database scopes
- Added `studio_assets` and `studio_pipelines` database tables for user-owned Studio storage, created through normal startup migrations
- Kept global config admin characters, environments, voices, archive items, and pipelines file-backed under `~/.aisbf/studio/`
- Saved admin custom pipelines in `~/.aisbf/pipelines.json` and user custom pipelines in the database
- Added `/api/v1/...` and `/api/u/{username}/...` Studio aliases for profile, archive, pipeline, and proxied advanced media routes
- Dashboard Studio advanced media tools must proxy to provider-facing `v1/...` endpoints; no local ffmpeg/OpenCV/InsightFace/Real-ESRGAN processing is assumed

**2026-03-22 - Configuration Refactoring**
- Centralized API key storage in providers.json
- API keys are now stored only in provider definitions, not in rotation/autoselect configs
- Rotation and autoselect configurations reference providers by name only
- Added support for provider-only entries in rotations (no models specified)
- When no models specified, system randomly selects from provider's available models
- Added default settings support at provider level (default_rate_limit, default_max_request_tokens, etc.)
- Added default settings support at rotation level
- Settings priority: model-specific > rotation defaults > provider defaults
- Updated ProviderConfig with default setting fields
- Updated RotationConfig with default setting fields
- Added _apply_defaults_to_model() method in RotationHandler
- Updated configuration examples in AI.PROMPT

**2026-02-07 - Version 0.2.7**
- Added max_request_tokens support for automatic request splitting
- Updated ProviderConfig to include optional models field with max_request_tokens
- Updated Model class to include max_request_tokens field
- Added token counting utilities using tiktoken and langchain-text-splitters in aisbf/utils.py
- Updated RequestHandler and RotationHandler to automatically split requests exceeding max_request_tokens
- Updated config/providers.json with example models including max_request_tokens
- Updated config/rotations.json with example models including max_request_tokens
- Added tiktoken and langchain-text-splitters dependencies to requirements.txt

**2026-02-06 - Version 0.2.6**
- Bumped version to 0.2.6 in setup.py, pyproject.toml, and aisbf/__init__.py
- Updated README.md and DOCUMENTATION.md with comprehensive API endpoint documentation
- Added detailed sections for General, Provider, Rotation, and Autoselect endpoints
- Documented rotation load balancing and AI-assisted autoselect based on content analysis

## max_request_tokens Feature

### Overview

The `max_request_tokens` feature allows automatic splitting of large requests into multiple smaller requests when the token count exceeds a specified limit. This is useful for models with strict token limits or for handling very large inputs.

### Configuration

#### providers.json

In [`providers.json`](config/providers.json), each provider can optionally include a `models` field containing model definitions:

```json
{
  "providers": {
    "gemini": {
      "id": "gemini",
      "name": "Google AI Studio",
      "endpoint": "https://generativelanguage.googleapis.com/v1beta",
      "type": "google",
      "api_key_required": true,
      "rate_limit": 0,
      "models": [
        {
          "name": "gemini-2.0-flash",
          "rate_limit": 0,
          "max_request_tokens": 1000000
        }
      ]
    }
  }
}
```

#### rotations.json

In [`rotations.json`](config/rotations.json), each model definition can optionally include `max_request_tokens`:

```json
{
  "rotations": {
    "coding": {
      "model_name": "coding",
      "providers": [
        {
          "provider_id": "openai",
          "api_key": "YOUR_OPENAI_API_KEY",
          "models": [
            {
              "name": "gpt-4",
              "weight": 2,
              "rate_limit": 0,
              "max_request_tokens": 128000
            }
          ]
        }
      ]
    }
  }
}
```

### Behavior

 
## Token Rate Limiting Feature

### Overview

The token rate limiting feature allows you to set limits on token usage per model to control costs and prevent abuse. When a token rate limit is exceeded, the provider is automatically disabled for a specified duration.

### Configuration

#### providers.json

In [`providers.json`](config/providers.json), model definitions can include optional token rate limit fields:

```json
{
  "providers": {
    "gemini": {
      "id": "gemini",
      "name": "Google AI Studio",
      "endpoint": "https://generativelanguage.googleapis.com/v1beta",
      "type": "google",
      "api_key_required": true,
      "rate_limit": 0,
      "models": [
        {
          "name": "gemini-2.0-flash",
          "rate_limit": 0,
          "max_request_tokens": 100000,
          "rate_limit_TPM": 15000,
          "rate_limit_TPH": 100000,
          "rate_limit_TPD": 1000000
        }
      ]
    }
  }
}
```

#### rotations.json

In [`rotations.json`](config/rotations.json), model definitions can include optional token rate limit fields:

```json
{
  "rotations": {
    "coding": {
      "model_name": "coding",
      "providers": [
        {
          "provider_id": "gemini",
          "api_key": "YOUR_GEMINI_API_KEY",
          "models": [
            {
              "name": "gemini-2.0-flash",
              "weight": 3,
              "rate_limit": 0,
              "max_request_tokens": 100000,
              "rate_limit_TPM": 15000,
              "rate_limit_TPH": 100000,
              "rate_limit_TPD": 1000000
            }
          ]
        }
      ]
    }
  }
}
```

### Rate Limit Fields

- **`rate_limit_TPM`**: Maximum tokens allowed per minute (Tokens Per Minute)
- **`rate_limit_TPH`**: Maximum tokens allowed per hour (Tokens Per Hour)
- **`rate_limit_TPD`**: Maximum tokens allowed per day (Tokens Per Day)

### Behavior

When a token rate limit is exceeded:

1. The provider is automatically disabled for a duration based on which limit was exceeded:
   - TPM limit exceeded: Provider disabled for 1 minute
   - TPH limit exceeded: Provider disabled for 1 hour
   - TPD limit exceeded: Provider disabled for 1 day

2. The rotation handler will skip the disabled provider and try the next available model

3. Token usage is tracked across all requests and accumulated for each time window

4. Old usage data outside the time window is automatically cleaned up

### Implementation

- Token counting uses tiktoken for accurate token estimation
- Usage is tracked per model in the provider handler
- Rate limit checks are performed before making requests
- Providers are automatically re-enabled after the cooldown period expires

When a request is made to a provider or rotation:

1. The system checks if the model has a `max_request_tokens` configured
2. If configured, the system counts the tokens in the request using tiktoken
3. If the request token count exceeds `max_request_tokens`, the request is automatically split into multiple chunks
4. Each chunk is sent sequentially to the provider
5. Responses from all chunks are combined into a single response
6. The combined response includes metadata indicating it was chunked:
   - `aisbf_chunked: true`
   - `aisbf_total_chunks: <number of chunks>`

### Limitations

- Streaming is not supported for chunked requests (falls back to non-streaming)
- Tools and tool_choice are only sent with the first chunk
- If a chunk fails, the system returns partial results from successful chunks

### Token Counting

The system uses tiktoken for accurate token counting. The encoding is automatically selected based on the model name:
- OpenAI models (gpt-*, text-embedding-*) use cl100k_base encoding
- Other models use a default encoding

#### Text Splitting

The system uses `langchain-text-splitters`'s `TokenTextSplitter` for intelligent text splitting:
- **chunk_size**: Set slightly below max_request_tokens to allow room for model output (e.g., max_request_tokens - 200)
- **chunk_overlap**: 100 tokens shared between chunks for context continuity
- This helps the model understand the transition between chunks


### Example

A request with 7,999 tokens and `max_request_tokens: 4000` would be split into 2 chunks:
- Chunk 1: ~3,800 tokens (with 100 token overlap from previous)
- Chunk 2: ~3,999 tokens

The responses are combined and returned as a single response with metadata indicating the chunking occurred.

### How to Update AI.PROMPT

When making changes that affect the project structure or functionality:

1. **Identify the scope of changes** - Determine which sections of AI.PROMPT need updating
2. **Update relevant sections** - Modify the appropriate parts of the documentation
3. **Add new sections if needed** - Create new documentation for new features
4. **Remove outdated information** - Delete sections that are no longer relevant
5. **Test the changes** - Verify that the documentation accurately reflects the current state
6. **Update the AI.PROMPT Management section** - Add notes about what was changed and why

### Common Update Scenarios

- **Moving files**: Update the Directory Structure and Module Structure sections
- **Changing installation**: Update the Installation Process and Key Functions sections
- **Modifying configuration**: Update the Configuration Management section
- **Adding new commands**: Update the AISBF Script Commands section
- **Changing workflows**: Update the Development vs Production section

### Version Control

AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.

## Proxy-Awareness and Reverse Proxy Support

### Overview

AISBF is fully proxy-aware and can be deployed behind reverse proxies like nginx, Apache, or Caddy. The application automatically detects and respects standard proxy headers to ensure correct URL generation and routing.

### Implementation

**Proxy Headers Middleware** ([`main.py`](main.py:349-408)):
- `ProxyHeadersMiddleware` class handles proxy header detection
- Automatically updates request scope with proxy information
- Processes standard proxy headers before other middleware
- Must be added before other middleware to ensure proper request scope modification

**Supported Proxy Headers**:
- `X-Forwarded-Proto`: Original protocol (http/https)
- `X-Forwarded-Host`: Original hostname
- `X-Forwarded-Port`: Original port number
- `X-Forwarded-Prefix` or `X-Script-Name`: URL subpath/prefix
- `X-Forwarded-For`: Original client IP address

**Helper Functions** ([`main.py`](main.py:410-445)):
- `get_base_url(request)`: Returns full base URL respecting proxy configuration
- `url_for(request, path)`: Generates proxy-aware URLs for any path
- `setup_template_globals()`: Configures Jinja2 templates with proxy-aware URL functions

### URL Generation

**In Python Code**:
```python
# Use url_for() for all redirects
return RedirectResponse(url=url_for(request, "/dashboard"), status_code=303)

# Get base URL
base_url = get_base_url(request)  # e.g., "https://example.com:8443/aisbf"
```

**In Jinja2 Templates**:
```html
<!-- Use url_for() function in templates -->
<a href="{{ url_for(request, '/dashboard/providers') }}">Providers</a>

<!-- For JavaScript fetch calls -->
fetch('{{ url_for(request, "/dashboard/restart") }}', {
    method: 'POST',
    headers: {'Content-Type': 'application/json'}
})
```

### Deployment Scenarios

**Scenario 1: Root Path Deployment**
- URL: `https://aisbf.example.com/`
- No special configuration needed
- Proxy sets: `X-Forwarded-Proto: https`, `X-Forwarded-Host: aisbf.example.com`

**Scenario 2: Subpath Deployment**
- URL: `https://example.com/aisbf/`
- Proxy must set: `X-Forwarded-Prefix: /aisbf`
- AISBF automatically adjusts all URLs to include `/aisbf` prefix

**Scenario 3: Custom Port**
- URL: `https://example.com:8443/`
- Proxy sets: `X-Forwarded-Port: 8443`
- URLs include port number when non-standard

### Nginx Configuration

See [`DOCUMENTATION.md`](DOCUMENTATION.md) "Proxy Deployment" section for complete nginx configuration examples including:
- Simple reverse proxy (root path)
- Reverse proxy with subpath
- Load balancing multiple instances
- API-only deployment with rate limiting

### Testing Proxy Configuration

After deploying behind a proxy:
1. Access the dashboard and verify all links work correctly
2. Test redirects (login, logout) work properly
3. Verify the restart button uses correct URL
4. Test API endpoints through the proxy
5. Check that streaming responses work correctly

### Common Issues and Solutions

**Issue**: Dashboard links return 404 errors
**Solution**: Ensure `X-Forwarded-Prefix` header is set correctly in nginx

**Issue**: Redirect loops after login
**Solution**: Verify `X-Forwarded-Proto` is set to `https` for HTTPS deployments

**Issue**: URLs missing subpath prefix
**Solution**: Check that `X-Forwarded-Prefix` or `X-Script-Name` header is configured

**Issue**: Streaming responses not working
**Solution**: Ensure `proxy_buffering off` is set in nginx configuration

### Files Modified for Proxy-Awareness

- [`main.py`](main.py): Added ProxyHeadersMiddleware, url_for(), get_base_url() functions
- [`templates/base.html`](templates/base.html): Updated all URLs to use url_for() function
- All dashboard route handlers in [`main.py`](main.py): Updated RedirectResponse calls to use url_for()
- [`DOCUMENTATION.md`](DOCUMENTATION.md): Added comprehensive nginx proxy configuration examples

## CoderAI Broker Mode

AISBF now includes an AISBF-side broker for NAT-friendly CoderAI integration.

### Overview

- New provider type: `coderai`
- Supports direct HTTP, direct WebSocket bridge, and AISBF-hosted outbound broker sessions
- Broker endpoint lives inside AISBF and lets remote CoderAI clients maintain an outbound persistent WebSocket when AISBF cannot reach them directly

### Key Files

- `aisbf/providers/coderai.py` - CoderAI provider handler with broker-aware routing
- `aisbf/coderai_broker.py` - in-memory broker/session registry and request correlation
- `aisbf/routes/coderai_broker.py` - WebSocket/session status routes for the broker
- `docs/coderai-integration.md` - external integration contract for CoderAI-side implementation

### Broker Routes

- `GET /api/coderai/broker/sessions` - list active broker sessions
- `GET /api/coderai/broker/providers/{provider_id}/status` - inspect active session status for a provider/client
- `WS /api/coderai/wss` - persistent outbound WebSocket endpoint for global-scope CoderAI clients
- `WS /api/u/{username}/coderai/wss` - persistent outbound WebSocket endpoint for user-owned CoderAI clients

### Broker Request Flow

1. CoderAI opens outbound WebSocket to AISBF broker
2. AISBF registers a scoped live session keyed by `provider_id` and `client_id`, plus `username` / `global` scope
3. CoderAI sends `register` metadata/capabilities over that socket
4. `CoderAIProviderHandler` prefers the live broker session when `coderai_config.broker_preferred=true`
5. In multi-node deployments, AISBF forwards `models.list`, `chat.completions`, `capabilities`, `register`, and `proxy` operations through cache-backed broker queues keyed by the active session id
6. The AISBF node holding the actual WebSocket consumes queued requests, sends them to CoderAI, and publishes replies back through cache-backed reply keys
7. Response envelopes are correlated by `request_id` and returned to the original AISBF request from any AISBF node

### Configuration Notes

`coderai_config` now supports:

- `broker_enabled`
- `broker_mode`
- `broker_preferred`
- `client_id`
- `bridge_path`
- `registration_path`
- `registration_token`

The dashboard provider editors now expose `coderai_config.registration_token` directly for both:

- global admin-managed CoderAI providers
- user-owned CoderAI providers

Token rotation is performed inline from the provider editor and updates the owning provider configuration immediately.

### Dashboard Visibility

- Admin and user provider pages display live broker session status directly inside each `coderai` provider card
- Visible metadata includes owner, client id, transport, endpoint, last seen, connected-at time, and advertised Studio endpoints
- `GET /dashboard/api/coderai/broker/sessions` returns broker sessions filtered to the current dashboard principal

### Important Limitation

Active broker routing still requires a live WebSocket, but AISBF now persists broker session metadata snapshots in `~/.aisbf/coderai_broker_sessions.json` and mirrors live broker state into the configured cache backend so clustered AISBF nodes can route through one connected broker client. Remote CoderAI must still reconnect before requests can be routed again after a total disconnect.

### Security Considerations

When deploying behind a proxy:
- AISBF should bind to localhost only (default: `127.0.0.1:17765`)
- Proxy should handle SSL/TLS termination
- Enable authentication in AISBF configuration
- Configure rate limiting in the proxy
- Set security headers in proxy configuration
- Monitor access logs for suspicious activity
