Skip to content

A
aisbf
Commit a3506741 authored by Your Name's avatar Your Name
Browse files
Options

All update up to proxy

parent 36fc3e9f
Hide whitespace changes
Inline Side-by-side
Showing with 4618 additions and 451 deletions
AI.PROMPT
View file @ a3506741
......@@ -383,7 +383,9 @@ Kiro Gateway is a third-party proxy gateway that provides OpenAI and Anthropic-c
**Configuration:**
In `config/providers.json`:
**IMPORTANT:** Kiro providers use a different configuration structure than other providers. Instead of an `api_key` field, they use a `kiro_config` object with authentication settings specific to Kiro/Amazon Q Developer.
**Standard Provider Configuration (Google, OpenAI, Anthropic, etc.):**
```json
{
"gemini": {
......@@ -405,18 +407,32 @@ In `config/providers.json`:
"context_size": 1000000
}
]
},
}
}
```
**Kiro Provider Configuration (uses kiro_config instead of api_key):**
```json
{
"kiro": {
"id": "kiro",
"name": "Kiro Gateway (Amazon Q Developer)",
"endpoint": "http://localhost:8000/v1",
"name": "Kiro (Amazon Q Developer)",
"endpoint": "https://q.us-east-1.amazonaws.com",
"type": "kiro",
"api_key_required": true,
"api_key": "YOUR_KIRO_API_KEY",
"api_key_required": false,
"rate_limit": 0,
"kiro_config": {
"region": "us-east-1",
"creds_file": "~/.config/Code/User/globalStorage/amazon.q/credentials.json",
"sqlite_db": "",
"refresh_token": "",
"profile_arn": "",
"client_id": "",
"client_secret": ""
},
"models": [
{
"name": "claude-sonnet-4-5",
"name": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"rate_limit": 0,
"max_request_tokens": 200000,
"context_size": 200000
......@@ -426,6 +442,20 @@ In `config/providers.json`:
}
```
**Kiro Configuration Fields (kiro_config):**
- `region`: AWS region (default: "us-east-1")
- `creds_file`: Path to Kiro IDE credentials JSON file (Option 1)
- `sqlite_db`: Path to kiro-cli SQLite database (Option 2)
- `refresh_token`: Direct refresh token for authentication (Option 3)
- `profile_arn`: AWS CodeWhisperer profile ARN (optional)
- `client_id`: OAuth client ID for AWS SSO OIDC (used with refresh_token)
- `client_secret`: OAuth client secret for AWS SSO OIDC (used with refresh_token)
Choose one authentication method:
1. **Kiro IDE credentials**: Set `creds_file` to the path of your Kiro IDE credentials
2. **kiro-cli database**: Set `sqlite_db` to the path of your kiro-cli SQLite database
3. **Direct credentials**: Set `refresh_token` and optionally `client_id`/`client_secret` for AWS SSO OIDC
In `config/rotations.json` (API keys are now referenced from providers.json):
```json
{
......@@ -590,6 +620,25 @@ This AI.PROMPT file is automatically updated when significant changes are made t
### Recent Updates
**2026-03-23 - Proxy-Awareness Implementation**
- Added ProxyHeadersMiddleware to handle reverse proxy deployments
- Implemented automatic detection of proxy headers (X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port, X-Forwarded-Prefix, X-Forwarded-For)
- Updated all RedirectResponse calls to use proxy-aware url_for() function
- Updated templates to use proxy-aware URL generation
- Added comprehensive nginx proxy configuration examples to DOCUMENTATION.md
- Application now fully supports deployment behind reverse proxies with subpaths
- Dashboard URLs, redirects, and API responses automatically adjust based on proxy configuration
**2026-03-23 - Kiro Provider Dashboard Support**
- Updated providers.html dashboard to show kiro-specific configuration fields
- Kiro providers now display `kiro_config` options (region, creds_file, sqlite_db, refresh_token, profile_arn, client_id, client_secret) instead of standard api_key fields
- Added `updateProviderType()` function to handle switching between kiro and other provider types
- Added `updateKiroConfig()` function to manage kiro-specific configuration fields
- Updated `addProvider()` function to ask for provider type and set appropriate defaults for kiro providers
- Dashboard automatically initializes kiro_config when provider type is changed to "kiro"
- Dashboard automatically removes kiro_config when provider type is changed from "kiro" to another type
- Updated AI.PROMPT documentation to clarify kiro provider configuration differences
**2026-03-22 - Configuration Refactoring**
- Centralized API key storage in providers.json
- API keys are now stored only in provider definitions, not in rotation/autoselect configs
......@@ -841,4 +890,118 @@ When making changes that affect the project structure or functionality:
### Version Control
AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.
\ No newline at end of file
AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.
## Proxy-Awareness and Reverse Proxy Support
### Overview
AISBF is fully proxy-aware and can be deployed behind reverse proxies like nginx, Apache, or Caddy. The application automatically detects and respects standard proxy headers to ensure correct URL generation and routing.
### Implementation
**Proxy Headers Middleware** ([`main.py`](main.py:349-408)):
- `ProxyHeadersMiddleware` class handles proxy header detection
- Automatically updates request scope with proxy information
- Processes standard proxy headers before other middleware
- Must be added before other middleware to ensure proper request scope modification
**Supported Proxy Headers**:
- `X-Forwarded-Proto`: Original protocol (http/https)
- `X-Forwarded-Host`: Original hostname
- `X-Forwarded-Port`: Original port number
- `X-Forwarded-Prefix` or `X-Script-Name`: URL subpath/prefix
- `X-Forwarded-For`: Original client IP address
**Helper Functions** ([`main.py`](main.py:410-445)):
- `get_base_url(request)`: Returns full base URL respecting proxy configuration
- `url_for(request, path)`: Generates proxy-aware URLs for any path
- `setup_template_globals()`: Configures Jinja2 templates with proxy-aware URL functions
### URL Generation
**In Python Code**:
```python
# Use url_for() for all redirects
return RedirectResponse(url=url_for(request, "/dashboard"), status_code=303)
# Get base URL
base_url = get_base_url(request) # e.g., "https://example.com:8443/aisbf"
```
**In Jinja2 Templates**:
```html
<!-- Use url_for() function in templates -->
<a href="{{ url_for(request, '/dashboard/providers') }}">Providers</a>
<!-- For JavaScript fetch calls -->
fetch('{{ url_for(request, "/dashboard/restart") }}', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
})
```
### Deployment Scenarios
**Scenario 1: Root Path Deployment**
- URL: `https://aisbf.example.com/`
- No special configuration needed
- Proxy sets: `X-Forwarded-Proto: https`, `X-Forwarded-Host: aisbf.example.com`
**Scenario 2: Subpath Deployment**
- URL: `https://example.com/aisbf/`
- Proxy must set: `X-Forwarded-Prefix: /aisbf`
- AISBF automatically adjusts all URLs to include `/aisbf` prefix
**Scenario 3: Custom Port**
- URL: `https://example.com:8443/`
- Proxy sets: `X-Forwarded-Port: 8443`
- URLs include port number when non-standard
### Nginx Configuration
See [`DOCUMENTATION.md`](DOCUMENTATION.md) "Proxy Deployment" section for complete nginx configuration examples including:
- Simple reverse proxy (root path)
- Reverse proxy with subpath
- Load balancing multiple instances
- API-only deployment with rate limiting
### Testing Proxy Configuration
After deploying behind a proxy:
1. Access the dashboard and verify all links work correctly
2. Test redirects (login, logout) work properly
3. Verify the restart button uses correct URL
4. Test API endpoints through the proxy
5. Check that streaming responses work correctly
### Common Issues and Solutions
**Issue**: Dashboard links return 404 errors
**Solution**: Ensure `X-Forwarded-Prefix` header is set correctly in nginx
**Issue**: Redirect loops after login
**Solution**: Verify `X-Forwarded-Proto` is set to `https` for HTTPS deployments
**Issue**: URLs missing subpath prefix
**Solution**: Check that `X-Forwarded-Prefix` or `X-Script-Name` header is configured
**Issue**: Streaming responses not working
**Solution**: Ensure `proxy_buffering off` is set in nginx configuration
### Files Modified for Proxy-Awareness
- [`main.py`](main.py): Added ProxyHeadersMiddleware, url_for(), get_base_url() functions
- [`templates/base.html`](templates/base.html): Updated all URLs to use url_for() function
- All dashboard route handlers in [`main.py`](main.py): Updated RedirectResponse calls to use url_for()
- [`DOCUMENTATION.md`](DOCUMENTATION.md): Added comprehensive nginx proxy configuration examples
### Security Considerations
When deploying behind a proxy:
- AISBF should bind to localhost only (default: `127.0.0.1:17765`)
- Proxy should handle SSL/TLS termination
- Enable authentication in AISBF configuration
- Configure rate limiting in the proxy
- Set security headers in proxy configuration
- Monitor access logs for suspicious activity
\ No newline at end of file
API_EXAMPLES.md
View file @ a3506741
......@@ -3,23 +3,52 @@
This document provides practical examples for using the AISBF API endpoints.
## Table of Contents
- [OpenAI-Compatible v1 Endpoints](#openai-compatible-v1-endpoints)
- [Three Proxy Paths](#three-proxy-paths)
- [Chat Completions](#chat-completions)
- [Audio Endpoints](#audio-endpoints)
- [Image Generation](#image-generation)
- [Embeddings](#embeddings)
- [Model Listing](#model-listing)
- [Rotations](#rotations)
- [Autoselect](#autoselect)
- [Advanced Features](#advanced-features)
## Three Proxy Paths
AISBF provides three ways to proxy AI models:
### PATH 1: Direct Provider Models
Format: `{provider_id}/{model_name}`
```bash
# Examples:
"openai/gpt-4"
"gemini/gemini-2.0-flash"
"anthropic/claude-3-5-sonnet-20241022"
"kilotest/kilo/free"
```
### PATH 2: Rotations
Format: `rotation/{rotation_name}`
```bash
# Examples:
"rotation/coding"
"rotation/general"
```
### PATH 3: Autoselect
Format: `autoselect/{autoselect_name}`
```bash
# Examples:
"autoselect/autoselect"
```
## OpenAI-Compatible v1 Endpoints
The v1 endpoints follow the standard OpenAI API format with `provider/model` notation.
The v1 endpoints follow the standard OpenAI API format and support all three proxy paths.
### Chat Completions
#### Using cURL
#### PATH 1: Direct Provider Models
Using cURL:
```bash
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
......@@ -31,26 +60,7 @@ curl -X POST http://localhost:17765/api/v1/chat/completions \
}'
```
#### Using Python
```python
import requests
response = requests.post(
"http://localhost:17765/api/v1/chat/completions",
json={
"model": "openai/gpt-4",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
]
}
)
print(response.json())
```
#### Using Python with OpenAI SDK
Using Python with OpenAI SDK:
```python
from openai import OpenAI
......@@ -69,31 +79,7 @@ response = client.chat.completions.create(
print(response.choices[0].message.content)
```
#### Streaming Response
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:17765/api/v1",
api_key="dummy"
)
stream = client.chat.completions.create(
model="gemini/gemini-2.0-flash",
messages=[
{"role": "user", "content": "Write a short poem"}
],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
```
#### Using Different Providers
Different providers:
```bash
# Google Gemini
curl -X POST http://localhost:17765/api/v1/chat/completions \
......@@ -111,6 +97,14 @@ curl -X POST http://localhost:17765/api/v1/chat/completions \
"messages": [{"role": "user", "content": "Hello"}]
}'
# Custom provider with nested model path
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "kilotest/kilo/free",
"messages": [{"role": "user", "content": "Hello"}]
}'
# Ollama (local)
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
......@@ -120,16 +114,130 @@ curl -X POST http://localhost:17765/api/v1/chat/completions \
}'
```
#### PATH 2: Rotations (Load Balancing)
Using cURL:
```bash
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "rotation/coding",
"messages": [
{"role": "user", "content": "Write a Python function to sort a list"}
]
}'
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:17765/api/v1",
api_key="dummy"
)
response = client.chat.completions.create(
model="rotation/coding",
messages=[
{"role": "user", "content": "Write a Python function to sort a list"}
]
)
print(response.choices[0].message.content)
```
#### PATH 3: Autoselect (AI-Powered Selection)
Using cURL:
```bash
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "autoselect/autoselect",
"messages": [
{"role": "user", "content": "Debug this Python code: def add(a,b): return a-b"}
]
}'
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:17765/api/v1",
api_key="dummy"
)
response = client.chat.completions.create(
model="autoselect/autoselect",
messages=[
{"role": "user", "content": "Debug this Python code: def add(a,b): return a-b"}
]
)
print(response.choices[0].message.content)
```
#### Streaming Response
Works with all three proxy paths:
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:17765/api/v1",
api_key="dummy"
)
# PATH 1: Direct provider
stream = client.chat.completions.create(
model="gemini/gemini-2.0-flash",
messages=[{"role": "user", "content": "Write a short poem"}],
stream=True
)
# PATH 2: Rotation
stream = client.chat.completions.create(
model="rotation/coding",
messages=[{"role": "user", "content": "Write a short poem"}],
stream=True
)
# PATH 3: Autoselect
stream = client.chat.completions.create(
model="autoselect/autoselect",
messages=[{"role": "user", "content": "Write a short poem"}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
```
## Audio Endpoints
**Note:** Audio endpoints support all three proxy paths (direct providers, rotations, and autoselect).
### Audio Transcription
Using `/api/audio/transcriptions`:
```bash
curl -X POST http://localhost:17765/api/audio/transcriptions \
-F "file=@audio.mp3" \
-F "model=openai/whisper-1"
```
Using `/api/v1/audio/transcriptions` (OpenAI-compatible):
```bash
curl -X POST http://localhost:17765/api/v1/audio/transcriptions \
-F "file=@audio.mp3" \
-F "model=openai/whisper-1"
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
......@@ -149,8 +257,9 @@ print(transcript.text)
### Text-to-Speech
Using `/api/audio/speech`:
```bash
curl -X POST http://localhost:17765/api/v1/audio/speech \
curl -X POST http://localhost:17765/api/audio/speech \
-H "Content-Type: application/json" \
-d '{
"model": "openai/tts-1",
......@@ -160,6 +269,7 @@ curl -X POST http://localhost:17765/api/v1/audio/speech \
--output speech.mp3
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
......@@ -179,8 +289,11 @@ response.stream_to_file("speech.mp3")
## Image Generation
**Note:** Image generation supports all three proxy paths (direct providers, rotations, and autoselect).
Using `/api/images/generations`:
```bash
curl -X POST http://localhost:17765/api/v1/images/generations \
curl -X POST http://localhost:17765/api/images/generations \
-H "Content-Type: application/json" \
-d '{
"model": "openai/dall-e-3",
......@@ -190,6 +303,7 @@ curl -X POST http://localhost:17765/api/v1/images/generations \
}'
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
......@@ -210,8 +324,11 @@ print(response.data[0].url)
## Embeddings
**Note:** Embeddings support all three proxy paths (direct providers, rotations, and autoselect).
Using `/api/embeddings`:
```bash
curl -X POST http://localhost:17765/api/v1/embeddings \
curl -X POST http://localhost:17765/api/embeddings \
-H "Content-Type: application/json" \
-d '{
"model": "openai/text-embedding-ada-002",
......@@ -219,6 +336,7 @@ curl -X POST http://localhost:17765/api/v1/embeddings \
}'
```
Using Python with OpenAI SDK:
```python
from openai import OpenAI
......@@ -237,12 +355,27 @@ print(response.data[0].embedding)
## Model Listing
### List All Models
### List All Models (All Three Proxy Paths)
The `/api/models` endpoint lists models from all three proxy paths:
Using cURL:
```bash
curl http://localhost:17765/api/v1/models
curl http://localhost:17765/api/models
```
Using Python:
```python
import requests
response = requests.get("http://localhost:17765/api/models")
models = response.json()["data"]
for model in models:
print(f"{model['id']} - Type: {model.get('type', 'unknown')}")
```
Using OpenAI SDK:
```python
from openai import OpenAI
......@@ -256,99 +389,59 @@ for model in models.data:
print(f"{model.id} - {model.owned_by}")
```
## Rotations
Example output:
```
openai/gpt-4 - Type: provider
gemini/gemini-2.0-flash - Type: provider
rotation/coding - Type: rotation
rotation/general - Type: rotation
autoselect/autoselect - Type: autoselect
```
Rotations provide weighted load balancing across multiple providers.
## Legacy Endpoints
### Using Rotation with v1 Endpoint
For backward compatibility, these endpoints are still available:
### Legacy Provider Endpoints
```bash
curl -X POST http://localhost:17765/api/v1/chat/completions \
# Direct provider access (model without provider prefix)
curl -X POST http://localhost:17765/api/openai/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "coding",
"messages": [
{"role": "user", "content": "Write a Python function to sort a list"}
]
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello"}]
}'
```
### Using Legacy Rotation Endpoint
```bash
curl -X POST http://localhost:17765/api/rotations/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "coding",
"messages": [
{"role": "user", "content": "Write a Python function to sort a list"}
]
}'
# List provider models
curl http://localhost:17765/api/openai/models
```
### List Available Rotations
### Legacy Rotation Endpoints
```bash
# List rotations
curl http://localhost:17765/api/rotations
```
### List Rotation Models
```bash
curl http://localhost:17765/api/rotations/models
```
## Autoselect
Autoselect uses AI to automatically select the best model based on your request.
### Using Autoselect with v1 Endpoint
```bash
curl -X POST http://localhost:17765/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "autoselect",
"messages": [
{"role": "user", "content": "Debug this Python code: def add(a,b): return a-b"}
]
}'
```
### Using Legacy Autoselect Endpoint
```bash
curl -X POST http://localhost:17765/api/autoselect/chat/completions \
# Use rotation (model name = rotation name)
curl -X POST http://localhost:17765/api/rotations/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "autoselect",
"messages": [
{"role": "user", "content": "Debug this Python code: def add(a,b): return a-b"}
]
"model": "coding",
"messages": [{"role": "user", "content": "Write code"}]
}'
```
### List Available Autoselect Configurations
### Legacy Autoselect Endpoints
```bash
# List autoselect configurations
curl http://localhost:17765/api/autoselect
```
## Legacy Provider Endpoints
You can also use provider-specific endpoints:
```bash
# Direct provider access
curl -X POST http://localhost:17765/api/openai/chat/completions \
# Use autoselect (model name = autoselect name)
curl -X POST http://localhost:17765/api/autoselect/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello"}]
"model": "autoselect",
"messages": [{"role": "user", "content": "Help me"}]
}'
# List provider models
curl http://localhost:17765/api/openai/models
```
## Authentication
......@@ -383,6 +476,7 @@ response = client.chat.completions.create(
### Using fetch API
PATH 1: Direct Provider
```javascript
const response = await fetch('http://localhost:17765/api/v1/chat/completions', {
method: 'POST',
......@@ -401,8 +495,47 @@ const data = await response.json();
console.log(data.choices[0].message.content);
```
PATH 2: Rotation
```javascript
const response = await fetch('http://localhost:17765/api/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'rotation/coding',
messages: [
{ role: 'user', content: 'Write a sorting function' }
]
})
});
const data = await response.json();
console.log(data.choices[0].message.content);
```
PATH 3: Autoselect
```javascript
const response = await fetch('http://localhost:17765/api/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'autoselect/autoselect',
messages: [
{ role: 'user', content: 'Help me with this task' }
]
})
});
const data = await response.json();
console.log(data.choices[0].message.content);
```
### Using OpenAI SDK
Works with all three proxy paths:
```javascript
import OpenAI from 'openai';
......@@ -411,18 +544,30 @@ const client = new OpenAI({
apiKey: 'dummy'
});
const response = await client.chat.completions.create({
// PATH 1: Direct provider
const response1 = await client.chat.completions.create({
model: 'openai/gpt-4',
messages: [
{ role: 'user', content: 'Hello, how are you?' }
]
messages: [{ role: 'user', content: 'Hello' }]
});
// PATH 2: Rotation
const response2 = await client.chat.completions.create({
model: 'rotation/coding',
messages: [{ role: 'user', content: 'Write code' }]
});
// PATH 3: Autoselect
const response3 = await client.chat.completions.create({
model: 'autoselect/autoselect',
messages: [{ role: 'user', content: 'Help me' }]
});
console.log(response.choices[0].message.content);
console.log(response1.choices[0].message.content);
```
### Streaming in JavaScript
Works with all three proxy paths:
```javascript
import OpenAI from 'openai';
......@@ -431,8 +576,9 @@ const client = new OpenAI({
apiKey: 'dummy'
});
// Use any of the three proxy paths
const stream = await client.chat.completions.create({
model: 'gemini/gemini-2.0-flash',
model: 'gemini/gemini-2.0-flash', // or 'rotation/coding' or 'autoselect/autoselect'
messages: [
{ role: 'user', content: 'Write a short poem' }
],
......
DOCUMENTATION.md
View file @ a3506741
......@@ -485,6 +485,377 @@ aisbf stop
```
Stops running daemon and removes PID file.
## Proxy Deployment
AISBF is fully proxy-aware and can be deployed behind reverse proxies like nginx, Apache, or Caddy. The application automatically detects and respects standard proxy headers to ensure correct URL generation and routing.
### Proxy-Awareness Features
AISBF automatically handles:
- **X-Forwarded-Proto**: Detects original protocol (http/https)
- **X-Forwarded-Host**: Detects original hostname
- **X-Forwarded-Port**: Detects original port
- **X-Forwarded-Prefix** or **X-Script-Name**: Detects URL subpath/prefix
- **X-Forwarded-For**: Detects original client IP address
All URLs in the dashboard, redirects, and API responses are automatically adjusted based on these headers.
### Nginx Proxy Configuration Examples
#### Example 1: Simple Reverse Proxy (Root Path)
Deploy AISBF at the root of a domain (e.g., `https://aisbf.example.com/`):
```nginx
server {
listen 80;
server_name aisbf.example.com;
# Redirect HTTP to HTTPS
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name aisbf.example.com;
# SSL configuration
ssl_certificate /etc/ssl/certs/aisbf.example.com.crt;
ssl_certificate_key /etc/ssl/private/aisbf.example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
# Proxy settings
location / {
proxy_pass http://127.0.0.1:17765;
proxy_http_version 1.1;
# Proxy headers for AISBF proxy-awareness
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# WebSocket support for streaming
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running requests
proxy_connect_timeout 60s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffering settings for streaming
proxy_buffering off;
proxy_cache off;
}
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# Logging
access_log /var/log/nginx/aisbf_access.log;
error_log /var/log/nginx/aisbf_error.log;
}
```
#### Example 2: Reverse Proxy with Subpath
Deploy AISBF at a subpath (e.g., `https://example.com/aisbf/`):
```nginx
server {
listen 443 ssl http2;
server_name example.com;
# SSL configuration
ssl_certificate /etc/ssl/certs/example.com.crt;
ssl_certificate_key /etc/ssl/private/example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
# AISBF at /aisbf subpath
location /aisbf/ {
# Remove /aisbf prefix before proxying
rewrite ^/aisbf/(.*) /$1 break;
proxy_pass http://127.0.0.1:17765;
proxy_http_version 1.1;
# Proxy headers for AISBF proxy-awareness
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# IMPORTANT: Set the subpath prefix
proxy_set_header X-Forwarded-Prefix /aisbf;
# WebSocket support for streaming
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running requests
proxy_connect_timeout 60s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffering settings for streaming
proxy_buffering off;
proxy_cache off;
}
# Other locations for your main application
location / {
# Your main application configuration
root /var/www/html;
index index.html;
}
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# Logging
access_log /var/log/nginx/example_access.log;
error_log /var/log/nginx/example_error.log;
}
```
#### Example 3: Load Balancing Multiple AISBF Instances
Deploy multiple AISBF instances for high availability:
```nginx
upstream aisbf_backend {
# Multiple AISBF instances
server 127.0.0.1:17765 max_fails=3 fail_timeout=30s;
server 127.0.0.1:17766 max_fails=3 fail_timeout=30s;
server 127.0.0.1:17767 max_fails=3 fail_timeout=30s;
# Load balancing method
least_conn; # Route to instance with fewest connections
# Session persistence (optional, for dashboard sessions)
ip_hash;
}
server {
listen 443 ssl http2;
server_name aisbf.example.com;
# SSL configuration
ssl_certificate /etc/ssl/certs/aisbf.example.com.crt;
ssl_certificate_key /etc/ssl/private/aisbf.example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_pass http://aisbf_backend;
proxy_http_version 1.1;
# Proxy headers for AISBF proxy-awareness
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# WebSocket support for streaming
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running requests
proxy_connect_timeout 60s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffering settings for streaming
proxy_buffering off;
proxy_cache off;
# Health check
proxy_next_upstream error timeout http_502 http_503 http_504;
}
# Health check endpoint
location /health {
proxy_pass http://aisbf_backend/health;
access_log off;
}
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# Logging
access_log /var/log/nginx/aisbf_access.log;
error_log /var/log/nginx/aisbf_error.log;
}
```
#### Example 4: API-Only Deployment with Rate Limiting
Deploy AISBF as an API-only service with rate limiting:
```nginx
# Define rate limiting zones
limit_req_zone $binary_remote_addr zone=aisbf_api:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=aisbf_dashboard:10m rate=5r/s;
server {
listen 443 ssl http2;
server_name api.example.com;
# SSL configuration
ssl_certificate /etc/ssl/certs/api.example.com.crt;
ssl_certificate_key /etc/ssl/private/api.example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
# API endpoints with rate limiting
location /api/ {
# Apply rate limiting
limit_req zone=aisbf_api burst=20 nodelay;
proxy_pass http://127.0.0.1:17765;
proxy_http_version 1.1;
# Proxy headers for AISBF proxy-awareness
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# WebSocket support for streaming
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running AI requests
proxy_connect_timeout 60s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
# Buffering settings for streaming
proxy_buffering off;
proxy_cache off;
# CORS headers (if needed)
add_header Access-Control-Allow-Origin "*" always;
add_header Access-Control-Allow-Methods "GET, POST, OPTIONS" always;
add_header Access-Control-Allow-Headers "Authorization, Content-Type" always;
}
# Dashboard with stricter rate limiting
location /dashboard/ {
limit_req zone=aisbf_dashboard burst=10 nodelay;
proxy_pass http://127.0.0.1:17765;
proxy_http_version 1.1;
# Proxy headers for AISBF proxy-awareness
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# Standard proxy settings
proxy_buffering off;
}
# Block all other paths
location / {
return 404;
}
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# Logging
access_log /var/log/nginx/aisbf_api_access.log;
error_log /var/log/nginx/aisbf_api_error.log;
}
```
### Testing Proxy Configuration
After configuring nginx, test the setup:
1. **Test nginx configuration:**
```bash
sudo nginx -t
```
2. **Reload nginx:**
```bash
sudo systemctl reload nginx
```
3. **Test AISBF through proxy:**
```bash
# Test root endpoint
curl https://aisbf.example.com/
# Test with subpath
curl https://example.com/aisbf/
# Test API endpoint
curl -X POST https://aisbf.example.com/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "provider/model-name", "messages": [{"role": "user", "content": "Hello"}]}'
```
4. **Check dashboard URLs:**
- Open the dashboard in a browser
- Verify all links and redirects work correctly
- Check that the restart button uses the correct URL
### Troubleshooting Proxy Issues
If you encounter issues:
1. **Check nginx error logs:**
```bash
sudo tail -f /var/log/nginx/aisbf_error.log
```
2. **Verify proxy headers are being sent:**
- Check AISBF logs for received headers
- Enable debug logging in AISBF if needed
3. **Common issues:**
- **404 errors with subpath**: Ensure `X-Forwarded-Prefix` header is set correctly
- **Redirect loops**: Check that `X-Forwarded-Proto` is set to `https`
- **Broken dashboard links**: Verify all proxy headers are configured
- **Streaming not working**: Ensure `proxy_buffering off` is set
### Security Considerations
When deploying behind a proxy:
1. **Bind AISBF to localhost only** (default: `127.0.0.1:17765`)
2. **Use HTTPS** for all external connections
3. **Enable authentication** in AISBF configuration
4. **Configure rate limiting** in nginx
5. **Set security headers** in nginx configuration
6. **Keep SSL certificates up to date**
7. **Monitor access logs** for suspicious activity
## Key Classes and Functions
### aisbf/config.py
......
README.md
View file @ a3506741
......@@ -14,7 +14,10 @@ A modular proxy server for managing multiple AI provider integrations with unifi
- **Token Rate Limiting**: Per-model token usage tracking with TPM (tokens per minute), TPH (tokens per hour), and TPD (tokens per day) limits
- **Automatic Provider Disabling**: Providers automatically disabled when token rate limits are exceeded
- **Context Management**: Automatic context condensation when approaching model limits with multiple condensation methods
- **Provider-Level Defaults**: Set default condensation settings at provider level with cascading fallback logic
- **Effective Context Tracking**: Reports total tokens used (effective_context) for every request
- **SSL/TLS Support**: Built-in HTTPS support with Let's Encrypt integration and automatic certificate renewal
- **Self-Signed Certificates**: Automatic generation of self-signed certificates for development/testing
## Author
......@@ -76,6 +79,64 @@ See [`PYPI.md`](PYPI.md) for detailed instructions on publishing to PyPI.
- Ollama (direct HTTP)
## Configuration
### SSL/TLS Configuration
AISBF supports HTTPS with automatic certificate management:
#### Self-Signed Certificates (Default)
- Automatically generated on first run when HTTPS is enabled
- Stored in `~/.aisbf/cert.pem` and `~/.aisbf/key.pem`
- Valid for 365 days
- Suitable for development and internal use
#### Let's Encrypt Integration
Configure a public domain in the dashboard settings to enable Let's Encrypt:
- Automatic certificate generation using certbot
- Automatic renewal when certificates expire within 30 days
- Valid certificates trusted by all browsers
- Requires certbot to be installed on the system
**Installation:**
```bash
# Ubuntu/Debian
sudo apt-get install certbot
# CentOS/RHEL
sudo yum install certbot
# macOS
brew install certbot
```
**Configuration:**
1. Navigate to Dashboard → Settings
2. Set Protocol to "HTTPS"
3. Enter your public domain (e.g., `api.example.com`)
4. Optionally specify custom certificate/key paths
5. Save settings
The system will automatically:
- Generate certificates using Let's Encrypt if a public domain is configured
- Fall back to self-signed certificates if Let's Encrypt fails
- Check certificate expiry on startup
- Renew certificates when they expire within 30 days
### Provider-Level Defaults
Providers can now define default settings that cascade to all models:
- **`default_context_size`**: Default maximum context size for all models in this provider
- **`default_condense_context`**: Default condensation threshold percentage (0-100)
- **`default_condense_method`**: Default condensation method(s) for all models
**Cascading Priority:**
1. Rotation model config (highest priority, only if explicitly set)
2. Provider model-specific config
3. Provider default config (NEW)
4. System defaults
This allows minimal model definitions in rotations - unspecified fields automatically inherit from provider defaults.
### Model Configuration
Models can be configured with the following optional fields:
......@@ -86,7 +147,7 @@ Models can be configured with the following optional fields:
- **`rate_limit_TPD`**: Maximum tokens allowed per day (Tokens Per Day)
- **`context_size`**: Maximum context size in tokens for the model. Used to determine when to trigger context condensation.
- **`condense_context`**: Percentage (0-100) at which to trigger context condensation. 0 means disabled, any other value triggers condensation when context reaches this percentage of context_size.
- **`condense_method`**: String or list of strings specifying condensation method(s). Supported values: "hierarchical", "conversational", "semantic", "algoritmic". Multiple methods can be chained together.
- **`condense_method`**: String or list of strings specifying condensation method(s). Supported values: "hierarchical", "conversational", "semantic", "algorithmic". Multiple methods can be chained together.
When token rate limits are exceeded, providers are automatically disabled:
- TPM limit exceeded: Provider disabled for 1 minute
......@@ -97,68 +158,133 @@ When token rate limits are exceeded, providers are automatically disabled:
When context exceeds the configured percentage of `context_size`, the system automatically condenses the prompt using one or more methods:
1. **Hierarchical**: Separates context into persistent (long-term facts) and transient (immediate task) layers
2. **Conversational**: Summarizes old messages using a smaller model to maintain conversation continuity
3. **Semantic**: Prunes irrelevant context based on current query using a smaller "janitor" model
4. **Algoritmic**: Uses mathematical compression for technical data and logs (similar to LLMLingua)
1. **Hierarchical**: Separates context into persistent (long-term facts) and transient (immediate task) layers - Pure algorithmic, no LLM needed
2. **Conversational**: Summarizes old messages using a smaller model to maintain conversation continuity - LLM-based
3. **Semantic**: Prunes irrelevant context based on current query using a smaller "janitor" model - LLM-based
4. **Algorithmic**: Uses mathematical compression for technical data and logs (similar to LLMLingua) - Pure algorithmic, no LLM needed
**Note:** Only `conversational` and `semantic` methods require LLM calls and use prompt files from `config/`. The `hierarchical` and `algorithmic` methods are pure algorithmic transformations.
See `config/providers.json` and `config/rotations.json` for configuration examples.
## API Endpoints
### Three Proxy Paths
AISBF provides three ways to proxy AI models:
#### PATH 1: Direct Provider Models
Format: `{provider_id}/{model_name}`
- Access specific models from configured providers
- Example: `openai/gpt-4`, `gemini/gemini-2.0-flash`, `anthropic/claude-3-5-sonnet-20241022`, `kilotest/kilo/free`
#### PATH 2: Rotations
Format: `rotation/{rotation_name}`
- Weighted load balancing across multiple providers
- Automatic failover on errors
- Example: `rotation/coding`, `rotation/general`
#### PATH 3: Autoselect
Format: `autoselect/{autoselect_name}`
- AI-powered model selection based on content analysis
- Automatic routing to specialized models
- Example: `autoselect/autoselect`
### General Endpoints
- `GET /` - Server status and provider list (includes providers, rotations, and autoselect)
- `GET /api/models` - List all available models from all three proxy paths
- `GET /api/v1/models` - OpenAI-compatible model listing (same as `/api/models`)
### OpenAI-Compatible v1 Endpoints (Recommended)
These endpoints follow the standard OpenAI API format with `provider/model` notation:
- `POST /api/v1/chat/completions` - Chat completions (model format: `provider/model-name`)
- Example: `{"model": "openai/gpt-4", "messages": [...]}`
- Supports providers, rotations, and autoselect
- `GET /api/v1/models` - List all available models from all providers
- `POST /api/v1/audio/transcriptions` - Audio transcription (model format: `provider/model-name`)
- `POST /api/v1/audio/speech` - Text-to-speech (model format: `provider/model-name`)
- `POST /api/v1/images/generations` - Image generation (model format: `provider/model-name`)
- `POST /api/v1/embeddings` - Text embeddings (model format: `provider/model-name`)
- `GET /api/proxy/{content_id}` - Proxy generated content (images, audio, etc.)
### Chat Completions
#### Unified Endpoints (Recommended)
These endpoints accept all three proxy path formats in the model field:
**OpenAI-Compatible Format:**
```bash
POST /api/v1/chat/completions
{
"model": "openai/gpt-4", # PATH 1: Direct provider
"messages": [...]
}
POST /api/v1/chat/completions
{
"model": "rotation/coding", # PATH 2: Rotation
"messages": [...]
}
POST /api/v1/chat/completions
{
"model": "autoselect/autoselect", # PATH 3: Autoselect
"messages": [...]
}
```
**Alternative Format:**
```bash
POST /api/{provider_id}/chat/completions
# Where provider_id can be:
# - A provider name (e.g., "openai", "gemini")
# - A rotation name (e.g., "coding", "general")
# - An autoselect name (e.g., "autoselect")
```
### Provider Endpoints (Legacy)
- `POST /api/{provider_id}/chat/completions` - Chat completions for a specific provider
- `GET /api/{provider_id}/models` - List available models for a specific provider
- `POST /api/{provider_id}/audio/transcriptions` - Audio transcription
- `POST /api/{provider_id}/audio/speech` - Text-to-speech
- `POST /api/{provider_id}/images/generations` - Image generation
- `POST /api/{provider_id}/embeddings` - Text embeddings
### Audio Endpoints
Model specified in request (supports all three proxy paths):
- `POST /api/audio/transcriptions` - Audio transcription
- Example: `{"model": "openai/whisper-1", "file": ...}` or `{"model": "rotation/coding", "file": ...}`
- `POST /api/audio/speech` - Text-to-speech
- Example: `{"model": "openai/tts-1", "input": "Hello"}` or `{"model": "rotation/coding", "input": "Hello"}`
- `POST /api/v1/audio/transcriptions` - OpenAI-compatible audio transcription
- `POST /api/v1/audio/speech` - OpenAI-compatible text-to-speech
### Image Endpoints
Model specified in request (supports all three proxy paths):
- `POST /api/images/generations` - Image generation
- Example: `{"model": "openai/dall-e-3", "prompt": "A cat"}` or `{"model": "rotation/coding", "prompt": "A cat"}`
- `POST /api/v1/images/generations` - OpenAI-compatible image generation
### Embeddings Endpoints
Model specified in request (supports all three proxy paths):
- `POST /api/embeddings` - Text embeddings
- Example: `{"model": "openai/text-embedding-ada-002", "input": "Hello"}` or `{"model": "rotation/coding", "input": "Hello"}`
- `POST /api/v1/embeddings` - OpenAI-compatible embeddings
### Legacy Endpoints
These endpoints are maintained for backward compatibility:
### Rotation Endpoints
- `GET /api/rotations` - List all available rotation configurations
- `POST /api/rotations/chat/completions` - Chat completions using rotation (load balancing across providers)
- **Rotation Models**: Weighted random selection of models across multiple providers
- Automatic failover between providers on errors
- Configurable weights for each model to prioritize preferred options
- Supports both streaming and non-streaming responses
- `POST /api/rotations/chat/completions` - Chat completions using rotation (model name = rotation name)
- `GET /api/rotations/models` - List all models across all rotation configurations
### Autoselect Endpoints
- `GET /api/autoselect` - List all available autoselect configurations
- `POST /api/autoselect/chat/completions` - Chat completions using AI-assisted selection based on content analysis
- **Autoselect Models**: AI analyzes request content to select the most appropriate model
- Automatic routing to specialized models based on task type (coding, analysis, creative writing, etc.)
- Fallback to default model if selection fails
- Supports both streaming and non-streaming responses
- `POST /api/autoselect/chat/completions` - Chat completions using autoselect (model name = autoselect name)
- `GET /api/autoselect/models` - List all models across all autoselect configurations
- `GET /api/{provider_id}/models` - List available models for a specific provider, rotation, or autoselect
### Content Proxy
- `GET /api/proxy/{content_id}` - Proxy generated content (images, audio, etc.)
### Dashboard Endpoints
- `GET /dashboard` - Web-based configuration dashboard
- `GET /dashboard/login` - Dashboard login page
- `POST /dashboard/login` - Handle dashboard authentication
- `GET /dashboard/logout` - Logout from dashboard
- `GET /dashboard/providers` - Edit providers configuration
- `GET /dashboard/providers` - Edit providers configuration (includes provider-level defaults for condensation)
- `GET /dashboard/rotations` - Edit rotations configuration
- `GET /dashboard/autoselect` - Edit autoselect configuration
- `GET /dashboard/settings` - Edit server settings
- `GET /dashboard/autoselect` - Edit autoselect configuration (supports multiple autoselect configs with "internal" model option)
- `GET /dashboard/settings` - Edit server settings (includes SSL/TLS configuration with Let's Encrypt support)
- `POST /dashboard/restart` - Restart the server
**Dashboard Features:**
- **Provider Configuration**: Set default condensation settings at provider level that cascade to all models
- **Autoselect Configuration**: Configure multiple autoselect models with "internal" option for using local HuggingFace models
- **SSL/TLS Management**: Configure HTTPS with automatic Let's Encrypt certificate generation and renewal
- **Collapsible UI**: All configuration sections use collapsible panels for better organization
## Error Handling
- Rate limiting for failed requests
- Automatic retry with provider rotation
......
aisbf.sh
View file @ a3506741
#!/bin/bash
#!/bin/bash
########################################################
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
......@@ -91,15 +91,22 @@ ensure_venv() {
fi
}
# Function to update venv packages silently
# Function to update venv packages (only install missing ones, no forced upgrades)
update_venv() {
# Update requirements if requirements.txt exists
# Only update if requirements file exists
if [ -f "$SHARE_DIR/requirements.txt" ]; then
"$VENV_DIR/bin/pip" install --upgrade -r "$SHARE_DIR/requirements.txt" -q 2>/dev/null
# Check if there are any new packages to install (not already satisfied)
"$VENV_DIR/bin/pip" install -r "$SHARE_DIR/requirements.txt" 2>&1 | grep -q "Requirement already satisfied"
ALREADY_SATISFIED=$?
if [ $ALREADY_SATISFIED -ne 0 ]; then
echo "Installing new requirements (this will take a while!) ... "
"$VENV_DIR/bin/pip" install -r "$SHARE_DIR/requirements.txt"
echo "[OK]"
else
echo "Virtual env already up to date"
fi
fi
# Update aisbf package silently
"$VENV_DIR/bin/pip" install --upgrade aisbf -q 2>/dev/null
}
# Function to start the server
......@@ -121,14 +128,19 @@ start_server() {
echo "Starting AISBF on port $PORT..."
# Start the proxy server with logging
# Redirect stderr to suppress BrokenPipeError during shutdown
uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | while IFS= read -r line; do
# Filter out BrokenPipeError logging errors
if [[ "$line" != *"--- Logging error ---"* ]] && [[ "$line" != *"BrokenPipeError"* ]] && [[ "$line" != *"Call stack:"* ]] && [[ "$line" != *"File "*"/python"* ]] && [[ "$line" != *"Message:"* ]] && [[ "$line" != *"Arguments:"* ]]; then
echo "$line" | tee -a "$LOG_DIR/aisbf_stdout.log"
fi
done
# Check if debug mode is enabled
if [ "$DEBUG" = "true" ]; then
echo "Debug mode enabled - showing all debug messages"
export AISBF_DEBUG=true
fi
# Start the proxy server - runs in foreground
# Use exec to replace the shell process so signals are properly handled
if [ "$DEBUG" = "true" ]; then
exec uvicorn main:app --host 127.0.0.1 --port $PORT --log-level debug 2>&1 | tee -a "$LOG_DIR/aisbf_stdout.log"
else
exec uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v -E "(--- Logging error ---|BrokenPipeError|Call stack:|Message:|Arguments:)" | tee -a "$LOG_DIR/aisbf_stdout.log"
fi
}
# Function to start as daemon
......@@ -156,9 +168,19 @@ start_daemon() {
echo "Starting AISBF on port $PORT in background..."
# Check if debug mode is enabled
if [ "$DEBUG" = "true" ]; then
echo "Debug mode enabled - showing all debug messages"
export AISBF_DEBUG=true
fi
# Start in background with nohup and logging
# Filter out BrokenPipeError logging errors
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v '--- Logging error ---' | grep -v 'BrokenPipeError' | grep -v 'Call stack:' | grep -v 'File .*python' | grep -v 'Message:' | grep -v 'Arguments:'" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
if [ "$DEBUG" = "true" ]; then
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT --log-level debug 2>&1" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
else
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v '--- Logging error ---' | grep -v 'BrokenPipeError' | grep -v 'Call stack:' | grep -v 'File .*python' | grep -v 'Message:' | grep -v 'Arguments:'" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
fi
PID=$!
echo $PID > "$PIDFILE"
echo "AISBF started in background (PID: $PID)"
......@@ -200,8 +222,59 @@ stop_daemon() {
fi
}
# Function to show help
show_help() {
echo "AISBF - AI Service Broker Framework"
echo ""
echo "Usage: aisbf.sh [OPTIONS] [COMMAND]"
echo ""
echo "Options:"
echo " --debug Enable debug mode with verbose logging"
echo " -h, --help Show this help message"
echo ""
echo "Commands:"
echo " daemon Start AISBF in background (daemon mode)"
echo " status Check if AISBF is running"
echo " stop Stop the AISBF daemon"
echo ""
echo "Examples:"
echo " aisbf.sh # Start in foreground"
echo " aisbf.sh --debug # Start with debug logging"
echo " aisbf.sh daemon # Start in background"
echo " aisbf.sh --debug daemon # Start in background with debug"
echo " aisbf.sh status # Check status"
echo " aisbf.sh stop # Stop the server"
}
# Parse command line arguments
DEBUG="false"
COMMAND=""
while [[ $# -gt 0 ]]; do
case $1 in
--debug|-d)
DEBUG="true"
shift
;;
-h|--help)
show_help
exit 0
;;
daemon|status|stop)
COMMAND="$1"
shift
;;
*)
echo "Unknown option: $1"
echo ""
show_help
exit 1
;;
esac
done
# Main command handling
case "$1" in
case "$COMMAND" in
daemon)
start_daemon
;;
......
aisbf/config.py
View file @ a3506741
......@@ -66,6 +66,7 @@ class ProviderConfig(BaseModel):
default_condense_method: Optional[Union[str, List[str]]] = None
class RotationConfig(BaseModel):
model_name: str
providers: List[Dict]
notifyerrors: bool = False
# Default settings for models in this rotation
......@@ -104,12 +105,16 @@ class Config:
if custom_dir:
self._custom_config_dir = Path(custom_dir)
# Track loaded file paths for summary
self._loaded_files = {}
self._ensure_config_directory()
self._load_providers()
self._load_rotations()
self._load_autoselect()
self._load_condensation()
self._initialize_error_tracking()
self._log_configuration_summary()
def _get_config_source_dir(self):
"""Get the directory containing default config files"""
......@@ -187,6 +192,7 @@ class Config:
with open(providers_path) as f:
data = json.load(f)
self.providers = {k: ProviderConfig(**v) for k, v in data['providers'].items()}
self._loaded_files['providers'] = str(providers_path.absolute())
logger.info(f"Loaded {len(self.providers)} providers: {list(self.providers.keys())}")
for provider_id, provider_config in self.providers.items():
logger.info(f" - {provider_id}: type={provider_config.type}, endpoint={provider_config.endpoint}")
......@@ -214,6 +220,7 @@ class Config:
logger.info(f"Loading rotations from: {rotations_path}")
with open(rotations_path) as f:
data = json.load(f)
self._loaded_files['rotations'] = str(rotations_path.absolute())
# Extract global notifyerrors setting (top-level, outside rotations)
self.global_notifyerrors = data.get('notifyerrors', False)
......@@ -254,18 +261,31 @@ class Config:
logger.info(f"=== Config._load_rotations END ===")
def _load_autoselect(self):
import logging
logger = logging.getLogger(__name__)
logger.info(f"=== Config._load_autoselect START ===")
autoselect_path = Path.home() / '.aisbf' / 'autoselect.json'
logger.info(f"Looking for autoselect at: {autoselect_path}")
if not autoselect_path.exists():
logger.info(f"User config not found, falling back to source config")
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
autoselect_path = source_dir / 'autoselect.json'
logger.info(f"Using source config at: {autoselect_path}")
except FileNotFoundError:
logger.error("Could not find autoselect.json configuration file")
raise FileNotFoundError("Could not find autoselect.json configuration file")
logger.info(f"Loading autoselect from: {autoselect_path}")
with open(autoselect_path) as f:
data = json.load(f)
self.autoselect = {k: AutoselectConfig(**v) for k, v in data.items()}
self._loaded_files['autoselect'] = str(autoselect_path.absolute())
logger.info(f"Loaded {len(self.autoselect)} autoselect configurations: {list(self.autoselect.keys())}")
logger.info(f"=== Config._load_autoselect END ===")
def _load_condensation(self):
"""Load condensation configuration from providers.json"""
......@@ -274,20 +294,26 @@ class Config:
logger.info(f"=== Config._load_condensation START ===")
providers_path = Path.home() / '.aisbf' / 'providers.json'
logger.info(f"Looking for condensation config in: {providers_path}")
if not providers_path.exists():
logger.info(f"User config not found, falling back to source config")
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
providers_path = source_dir / 'providers.json'
logger.info(f"Using source config at: {providers_path}")
except FileNotFoundError:
logger.warning("Could not find providers.json for condensation config")
self.condensation = CondensationConfig()
return
logger.info(f"Loading condensation config from: {providers_path}")
with open(providers_path) as f:
data = json.load(f)
condensation_data = data.get('condensation', {})
self.condensation = CondensationConfig(**condensation_data)
self._loaded_files['condensation'] = str(providers_path.absolute())
logger.info(f"Loaded condensation config: provider_id={self.condensation.provider_id}, model={self.condensation.model}, enabled={self.condensation.enabled}")
logger.info(f"=== Config._load_condensation END ===")
......@@ -299,6 +325,31 @@ class Config:
'last_failure': None,
'disabled_until': None
}
def _log_configuration_summary(self):
"""Log a summary of all loaded configuration files"""
import logging
logger = logging.getLogger(__name__)
logger.info("")
logger.info("=" * 80)
logger.info("=== CONFIGURATION FILES LOADED ===")
logger.info("=" * 80)
if 'providers' in self._loaded_files:
logger.info(f"Providers: {self._loaded_files['providers']}")
if 'rotations' in self._loaded_files:
logger.info(f"Rotations: {self._loaded_files['rotations']}")
if 'autoselect' in self._loaded_files:
logger.info(f"Autoselect: {self._loaded_files['autoselect']}")
if 'condensation' in self._loaded_files:
logger.info(f"Condensation: {self._loaded_files['condensation']}")
logger.info("=" * 80)
logger.info("")
def get_provider(self, provider_id: str) -> ProviderConfig:
import logging
......
aisbf/context.py
View file @ a3506741
......@@ -145,6 +145,8 @@ class ContextManager:
def _initialize_internal_model(self):
"""Initialize the internal HuggingFace model for condensation (lazy loading)"""
import logging
import json
from pathlib import Path
logger = logging.getLogger(__name__)
if self._internal_model is not None:
......@@ -156,7 +158,33 @@ class ContextManager:
import threading
logger.info("=== INITIALIZING INTERNAL CONDENSATION MODEL ===")
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
# Load model name from config
config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not config_path.exists():
# Try installed locations
installed_dirs = [
Path('/usr/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
test_path = installed_dir / 'aisbf.json'
if test_path.exists():
config_path = test_path
break
else:
# Fallback to source tree
config_path = Path(__file__).parent.parent / 'config' / 'aisbf.json'
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3" # Default
if config_path.exists():
try:
with open(config_path) as f:
aisbf_config = json.load(f)
model_name = aisbf_config.get('internal_model', {}).get('condensation_model_id', model_name)
except Exception as e:
logger.warning(f"Error loading condensation model config: {e}, using default")
logger.info(f"Model: {model_name}")
# Check for GPU availability
......@@ -700,7 +728,13 @@ def get_context_config_for_model(
rotation_model_config: Optional[Dict] = None
) -> Dict:
"""
Get context configuration for a specific model.
Get context configuration for a specific model with cascading fallback.
Priority order for each field:
1. Rotation model config (if explicitly set)
2. Provider model-specific config (if exists)
3. Provider default config (fallback)
4. System default (0 for condense_context, None for others)
Args:
model_name: Name of the model
......@@ -716,19 +750,47 @@ def get_context_config_for_model(
'condense_method': None
}
# Check rotation model config first (highest priority)
if rotation_model_config:
context_config['context_size'] = rotation_model_config.get('context_size')
context_config['condense_context'] = rotation_model_config.get('condense_context', 0)
context_config['condense_method'] = rotation_model_config.get('condense_method')
# Step 1: Get provider-level defaults and model-specific config
model_specific_config = None
if provider_config:
# Try to find model-specific config in provider
if hasattr(provider_config, 'models') and provider_config.models:
for model in provider_config.models:
if model.get('name') == model_name:
model_specific_config = model
break
# Build base config from provider (model-specific > provider defaults)
# context_size
if model_specific_config and model_specific_config.get('context_size') is not None:
context_config['context_size'] = model_specific_config.get('context_size')
elif hasattr(provider_config, 'default_context_size') and provider_config.default_context_size is not None:
context_config['context_size'] = provider_config.default_context_size
# condense_context
if model_specific_config and model_specific_config.get('condense_context') is not None:
context_config['condense_context'] = model_specific_config.get('condense_context')
elif hasattr(provider_config, 'default_condense_context') and provider_config.default_condense_context is not None:
context_config['condense_context'] = provider_config.default_condense_context
else:
context_config['condense_context'] = 0
# condense_method
if model_specific_config and model_specific_config.get('condense_method') is not None:
context_config['condense_method'] = model_specific_config.get('condense_method')
elif hasattr(provider_config, 'default_condense_method') and provider_config.default_condense_method is not None:
context_config['condense_method'] = provider_config.default_condense_method
# Fall back to provider config
elif provider_config and hasattr(provider_config, 'models'):
for model in provider_config.models:
if model.get('name') == model_name:
context_config['context_size'] = model.get('context_size')
context_config['condense_context'] = model.get('condense_context', 0)
context_config['condense_method'] = model.get('condense_method')
break
# Step 2: Override with rotation model config (only if explicitly set)
if rotation_model_config:
# Only override if the field is explicitly set in rotation config
if 'context_size' in rotation_model_config and rotation_model_config['context_size'] is not None:
context_config['context_size'] = rotation_model_config['context_size']
if 'condense_context' in rotation_model_config and rotation_model_config['condense_context'] is not None:
context_config['condense_context'] = rotation_model_config['condense_context']
if 'condense_method' in rotation_model_config and rotation_model_config['condense_method'] is not None:
context_config['condense_method'] = rotation_model_config['condense_method']
return context_config
\ No newline at end of file
aisbf/handlers.py
View file @ a3506741
......@@ -1366,6 +1366,11 @@ class RotationHandler:
logger.info(f" [AVAILABLE] Provider {provider_id} is active and ready")
# Check if provider-level weight is specified
provider_weight = provider.get('weight', 1) # Default to 1 if not specified
if provider.get('weight') is not None:
logger.info(f" Provider-level weight: {provider_weight}")
# Check if models are specified in rotation config
# If not, use models from provider config
rotation_models = provider.get('models')
......@@ -1375,17 +1380,17 @@ class RotationHandler:
# Get models from provider config
if provider_config.models:
# Use models from provider config with default weight of 1
# Use models from provider config with provider-level weight
rotation_models = []
for provider_model in provider_config.models:
model_dict = {
'name': provider_model.name,
'weight': 1, # Default weight
'weight': provider_weight, # Use provider-level weight
'rate_limit': provider_model.rate_limit,
'max_request_tokens': provider_model.max_request_tokens
}
rotation_models.append(model_dict)
logger.info(f" Loaded {len(rotation_models)} model(s) from provider config")
logger.info(f" Loaded {len(rotation_models)} model(s) from provider config with weight {provider_weight}")
else:
logger.warning(f" No models defined in provider config for {provider_id}")
logger.warning(f" Skipping this provider")
......@@ -2530,6 +2535,8 @@ class AutoselectHandler:
def _initialize_internal_model(self):
"""Initialize the internal HuggingFace model for selection (lazy loading)"""
import logging
import json
from pathlib import Path
logger = logging.getLogger(__name__)
if self._internal_model is not None:
......@@ -2541,7 +2548,33 @@ class AutoselectHandler:
import threading
logger.info("=== INITIALIZING INTERNAL SELECTION MODEL ===")
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
# Load model name from config
config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not config_path.exists():
# Try installed locations
installed_dirs = [
Path('/usr/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
test_path = installed_dir / 'aisbf.json'
if test_path.exists():
config_path = test_path
break
else:
# Fallback to source tree
config_path = Path(__file__).parent.parent / 'config' / 'aisbf.json'
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3" # Default
if config_path.exists():
try:
with open(config_path) as f:
aisbf_config = json.load(f)
model_name = aisbf_config.get('internal_model', {}).get('autoselect_model_id', model_name)
except Exception as e:
logger.warning(f"Error loading autoselect model config: {e}, using default")
logger.info(f"Model: {model_name}")
# Check for GPU availability
......
aisbf/utils.py
View file @ a3506741
......@@ -177,4 +177,335 @@ def get_max_request_tokens_for_model(
return max_tokens
logger.debug(f"No max_request_tokens configured for model {model_name}")
return None
\ No newline at end of file
return None
def generate_self_signed_certificate(cert_path: str, key_path: str) -> bool:
"""
Generate a self-signed SSL certificate and private key.
Args:
cert_path: Path where the certificate should be saved
key_path: Path where the private key should be saved
Returns:
True if successful, False otherwise
"""
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
try:
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization
import datetime
logger.info(f"Generating self-signed SSL certificate...")
logger.info(f"Certificate path: {cert_path}")
logger.info(f"Key path: {key_path}")
# Generate private key
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
)
# Create certificate
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, "US"),
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "State"),
x509.NameAttribute(NameOID.LOCALITY_NAME, "City"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "AISBF"),
x509.NameAttribute(NameOID.COMMON_NAME, "localhost"),
])
cert = x509.CertificateBuilder().subject_name(
subject
).issuer_name(
issuer
).public_key(
private_key.public_key()
).serial_number(
x509.random_serial_number()
).not_valid_before(
datetime.datetime.utcnow()
).not_valid_after(
datetime.datetime.utcnow() + datetime.timedelta(days=365)
).add_extension(
x509.SubjectAlternativeName([
x509.DNSName("localhost"),
x509.DNSName("127.0.0.1"),
]),
critical=False,
).sign(private_key, hashes.SHA256())
# Ensure directories exist
Path(cert_path).parent.mkdir(parents=True, exist_ok=True)
Path(key_path).parent.mkdir(parents=True, exist_ok=True)
# Write certificate to file
with open(cert_path, "wb") as f:
f.write(cert.public_bytes(serialization.Encoding.PEM))
# Write private key to file
with open(key_path, "wb") as f:
f.write(private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption()
))
logger.info(f"Successfully generated self-signed SSL certificate")
logger.info(f"Certificate valid for 365 days")
return True
except ImportError:
logger.error("cryptography library not installed. Cannot generate self-signed certificate.")
logger.error("Install with: pip install cryptography")
return False
except Exception as e:
logger.error(f"Error generating self-signed certificate: {e}", exc_info=True)
return False
def check_certificate_expiry(cert_path: str) -> Optional[int]:
"""
Check how many days until certificate expires.
Args:
cert_path: Path to certificate file
Returns:
Number of days until expiry, or None if cannot be determined
"""
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
try:
from cryptography import x509
from cryptography.hazmat.backends import default_backend
import datetime
if not Path(cert_path).exists():
return None
with open(cert_path, 'rb') as f:
cert_data = f.read()
cert = x509.load_pem_x509_certificate(cert_data, default_backend())
expiry_date = cert.not_valid_after
days_until_expiry = (expiry_date - datetime.datetime.utcnow()).days
logger.info(f"Certificate expires in {days_until_expiry} days")
return days_until_expiry
except Exception as e:
logger.error(f"Error checking certificate expiry: {e}")
return None
def generate_letsencrypt_certificate(domain: str, cert_path: str, key_path: str, email: Optional[str] = None) -> bool:
"""
Generate Let's Encrypt SSL certificate using certbot.
Args:
domain: Public domain name
cert_path: Path where certificate should be saved
key_path: Path where private key should be saved
email: Optional email for Let's Encrypt notifications
Returns:
True if successful, False otherwise
"""
import logging
import subprocess
from pathlib import Path
logger = logging.getLogger(__name__)
try:
logger.info(f"Generating Let's Encrypt certificate for domain: {domain}")
# Check if certbot is installed
try:
subprocess.run(['certbot', '--version'], capture_output=True, check=True)
except (subprocess.CalledProcessError, FileNotFoundError):
logger.error("certbot not found. Please install certbot:")
logger.error(" Ubuntu/Debian: sudo apt-get install certbot")
logger.error(" CentOS/RHEL: sudo yum install certbot")
logger.error(" macOS: brew install certbot")
return False
# Prepare certbot command
config_dir = Path.home() / '.aisbf' / 'letsencrypt'
config_dir.mkdir(parents=True, exist_ok=True)
cmd = [
'certbot', 'certonly',
'--standalone',
'--non-interactive',
'--agree-tos',
'--domain', domain,
'--config-dir', str(config_dir),
'--work-dir', str(config_dir / 'work'),
'--logs-dir', str(config_dir / 'logs'),
]
if email:
cmd.extend(['--email', email])
else:
cmd.append('--register-unsafely-without-email')
logger.info(f"Running certbot command...")
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
logger.error(f"certbot failed: {result.stderr}")
return False
# Copy certificates to specified paths
le_cert = config_dir / 'live' / domain / 'fullchain.pem'
le_key = config_dir / 'live' / domain / 'privkey.pem'
if le_cert.exists() and le_key.exists():
import shutil
shutil.copy2(le_cert, cert_path)
shutil.copy2(le_key, key_path)
logger.info(f"Let's Encrypt certificate generated successfully")
logger.info(f"Certificate valid for 90 days")
return True
else:
logger.error(f"Certificate files not found after certbot execution")
return False
except Exception as e:
logger.error(f"Error generating Let's Encrypt certificate: {e}", exc_info=True)
return False
def renew_letsencrypt_certificate(domain: str, cert_path: str, key_path: str) -> bool:
"""
Renew Let's Encrypt SSL certificate.
Args:
domain: Public domain name
cert_path: Path where certificate should be saved
key_path: Path where private key should be saved
Returns:
True if successful, False otherwise
"""
import logging
import subprocess
from pathlib import Path
logger = logging.getLogger(__name__)
try:
logger.info(f"Renewing Let's Encrypt certificate for domain: {domain}")
config_dir = Path.home() / '.aisbf' / 'letsencrypt'
cmd = [
'certbot', 'renew',
'--config-dir', str(config_dir),
'--work-dir', str(config_dir / 'work'),
'--logs-dir', str(config_dir / 'logs'),
'--non-interactive',
]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
logger.error(f"certbot renew failed: {result.stderr}")
return False
# Copy renewed certificates
le_cert = config_dir / 'live' / domain / 'fullchain.pem'
le_key = config_dir / 'live' / domain / 'privkey.pem'
if le_cert.exists() and le_key.exists():
import shutil
shutil.copy2(le_cert, cert_path)
shutil.copy2(le_key, key_path)
logger.info(f"Let's Encrypt certificate renewed successfully")
return True
else:
logger.error(f"Renewed certificate files not found")
return False
except Exception as e:
logger.error(f"Error renewing Let's Encrypt certificate: {e}", exc_info=True)
return False
def ensure_ssl_certificates(cert_path: Optional[str] = None, key_path: Optional[str] = None,
public_domain: Optional[str] = None, email: Optional[str] = None) -> tuple:
"""
Ensure SSL certificates exist, generating them if necessary.
Uses Let's Encrypt if public_domain is provided, otherwise self-signed.
Also checks expiry and renews if needed.
Args:
cert_path: Path to certificate file (None for default)
key_path: Path to key file (None for default)
public_domain: Public domain for Let's Encrypt (None for self-signed)
email: Email for Let's Encrypt notifications (optional)
Returns:
Tuple of (cert_path, key_path) with resolved paths
"""
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
# Use default paths if not specified
config_dir = Path.home() / '.aisbf'
if not cert_path:
cert_path = str(config_dir / 'cert.pem')
if not key_path:
key_path = str(config_dir / 'key.pem')
# Expand user paths
cert_path = str(Path(cert_path).expanduser())
key_path = str(Path(key_path).expanduser())
# Check if certificates exist
cert_exists = Path(cert_path).exists()
key_exists = Path(key_path).exists()
# Check certificate expiry if it exists
needs_renewal = False
if cert_exists:
days_until_expiry = check_certificate_expiry(cert_path)
if days_until_expiry is not None and days_until_expiry < 30:
logger.warning(f"Certificate expires in {days_until_expiry} days, renewal needed")
needs_renewal = True
# Generate or renew certificate
if not cert_exists or not key_exists or needs_renewal:
if public_domain:
# Try Let's Encrypt
logger.info(f"Using Let's Encrypt for domain: {public_domain}")
if needs_renewal:
success = renew_letsencrypt_certificate(public_domain, cert_path, key_path)
else:
success = generate_letsencrypt_certificate(public_domain, cert_path, key_path, email)
if not success:
logger.warning(f"Let's Encrypt failed, falling back to self-signed certificate")
if not generate_self_signed_certificate(cert_path, key_path):
raise FileNotFoundError(f"Failed to generate SSL certificates")
else:
# Generate self-signed certificate
logger.info(f"Generating self-signed SSL certificate...")
if not generate_self_signed_certificate(cert_path, key_path):
raise FileNotFoundError(f"Failed to generate self-signed SSL certificate")
else:
logger.info(f"Using existing SSL certificate: {cert_path}")
logger.info(f"Using existing SSL key: {key_path}")
return cert_path, key_path
\ No newline at end of file
config/aisbf.json
View file @ a3506741
......@@ -19,6 +19,7 @@
"password": "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
},
"internal_model": {
"model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
"condensation_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
"autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
}
}
config/autoselect.json
View file @ a3506741
......@@ -15,4 +15,4 @@
}
]
}
}
\ No newline at end of file
}
config/rotations.json
View file @ a3506741
......@@ -176,6 +176,27 @@
]
}
]
},
"simple-gemini": {
"_comment": "SIMPLE FORMAT: Specify provider_id only, models will be loaded from providers.json automatically",
"model_name": "simple-gemini",
"notifyerrors": false,
"providers": [
{
"provider_id": "gemini"
}
]
},
"simple-openai": {
"_comment": "SIMPLE FORMAT: Provider with custom weight, models loaded from providers.json",
"model_name": "simple-openai",
"notifyerrors": false,
"providers": [
{
"provider_id": "openai",
"weight": 2
}
]
}
}
}
\ No newline at end of file
main.py
View file @ a3506741
......@@ -31,6 +31,8 @@ from aisbf.models import ChatCompletionRequest, ChatCompletionResponse
from aisbf.handlers import RequestHandler, RotationHandler, AutoselectHandler
from aisbf.database import initialize_database
from starlette.middleware.sessions import SessionMiddleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.datastructures import Headers
from itsdangerous import URLSafeTimedSerializer
import time
import logging
......@@ -39,11 +41,14 @@ import os
import argparse
import secrets
import hashlib
import asyncio
from logging.handlers import RotatingFileHandler
from datetime import datetime, timedelta
from collections import defaultdict
from pathlib import Path
import json
import markdown
from urllib.parse import urljoin
# Global variable to store custom config directory
_custom_config_dir = None
......@@ -346,34 +351,413 @@ def setup_logging():
# Configure logging
logger = setup_logging()
class ProxyHeadersMiddleware(BaseHTTPMiddleware):
"""
Middleware to handle proxy headers and make the application proxy-aware.
Supports standard proxy headers:
- X-Forwarded-Proto: Original protocol (http/https)
- X-Forwarded-Host: Original host
- X-Forwarded-Port: Original port
- X-Forwarded-Prefix or X-Script-Name: URL prefix/subpath
- X-Forwarded-For: Client IP address
"""
async def dispatch(self, request: Request, call_next):
# Get proxy headers
forwarded_proto = request.headers.get("X-Forwarded-Proto")
forwarded_host = request.headers.get("X-Forwarded-Host")
forwarded_port = request.headers.get("X-Forwarded-Port")
forwarded_prefix = request.headers.get("X-Forwarded-Prefix") or request.headers.get("X-Script-Name")
forwarded_for = request.headers.get("X-Forwarded-For")
# Update request scope with proxy information
if forwarded_proto:
request.scope["scheme"] = forwarded_proto
if forwarded_host:
# Handle host:port format
if ":" in forwarded_host and not forwarded_port:
host_parts = forwarded_host.split(":", 1)
request.scope["server"] = (host_parts[0], int(host_parts[1]))
else:
port = int(forwarded_port) if forwarded_port else (443 if forwarded_proto == "https" else 80)
request.scope["server"] = (forwarded_host, port)
elif forwarded_port:
# Only port was forwarded, keep existing host
current_host = request.scope.get("server", ("localhost", 80))[0]
request.scope["server"] = (current_host, int(forwarded_port))
# Handle URL prefix/subpath
if forwarded_prefix:
# Remove trailing slash from prefix
forwarded_prefix = forwarded_prefix.rstrip("/")
request.scope["root_path"] = forwarded_prefix
# Update path to remove prefix if present
original_path = request.scope.get("path", "")
if original_path.startswith(forwarded_prefix):
request.scope["path"] = original_path[len(forwarded_prefix):] or "/"
# Store client IP from X-Forwarded-For
if forwarded_for:
# X-Forwarded-For can contain multiple IPs, take the first one (original client)
client_ip = forwarded_for.split(",")[0].strip()
request.scope["client"] = (client_ip, request.scope.get("client", ("", 0))[1])
response = await call_next(request)
return response
def get_base_url(request: Request) -> str:
"""
Get the base URL for the application, respecting proxy headers.
Returns the full base URL including scheme, host, port, and prefix.
Example: https://example.com:8443/aisbf
"""
scheme = request.scope.get("scheme", "http")
server = request.scope.get("server", ("localhost", 80))
host = server[0]
port = server[1]
root_path = request.scope.get("root_path", "")
# Don't include port in URL if it's the default for the scheme
if (scheme == "http" and port == 80) or (scheme == "https" and port == 443):
base_url = f"{scheme}://{host}{root_path}"
else:
base_url = f"{scheme}://{host}:{port}{root_path}"
return base_url
def url_for(request: Request, path: str) -> str:
"""
Generate a proxy-aware URL for the given path.
Args:
request: The current request object
path: The path to generate URL for (should start with /)
Returns:
Full URL respecting proxy configuration
"""
base_url = get_base_url(request)
# Ensure path starts with /
if not path.startswith("/"):
path = "/" + path
return f"{base_url}{path}"
# Note: config will be imported after parsing CLI args if --config is provided
# For now, we'll delay the import and initialization
app = FastAPI(title="AI Proxy Server")
# Initialize Jinja2 templates
# Add proxy headers middleware (must be added before other middleware)
app.add_middleware(ProxyHeadersMiddleware)
# Initialize Jinja2 templates with custom globals for proxy-aware URLs
templates = Jinja2Templates(directory="templates")
# Add session middleware (will be configured with secret key in main())
# Placeholder - will be added in main() after we have a secret key
session_secret_key = None
# Add custom template globals for proxy-aware URL generation
def setup_template_globals():
"""Setup Jinja2 template globals for proxy-aware URLs"""
templates.env.globals['url_for'] = url_for
templates.env.globals['get_base_url'] = get_base_url
# Call setup after templates are initialized
setup_template_globals()
# These will be initialized in main() after config is loaded
# Add session middleware at module level with a temporary secret key
# This is needed for uvicorn import (when main() doesn't run)
_default_session_secret = secrets.token_urlsafe(32)
app.add_middleware(SessionMiddleware, secret_key=_default_session_secret)
# These will be initialized in startup event or main() after config is loaded
request_handler = None
rotation_handler = None
autoselect_handler = None
server_config = None
config = None
_initialized = False
# Model cache for dynamically fetched provider models
_model_cache = {}
_model_cache_timestamps = {}
_cache_refresh_interval = 4 * 3600 # 4 hours in seconds
_cache_refresh_task = None
def initialize_app(custom_config_dir=None):
"""Initialize app globals. Called by startup event or main()."""
global config, request_handler, rotation_handler, autoselect_handler, server_config, _initialized
if _initialized:
return
# Set custom config directory if provided
if custom_config_dir:
set_config_dir(custom_config_dir)
logger.info(f"Using custom config directory: {custom_config_dir}")
# Import config
from aisbf.config import config as cfg
from aisbf.handlers import RequestHandler, RotationHandler, AutoselectHandler
config = cfg
request_handler = RequestHandler()
rotation_handler = RotationHandler()
autoselect_handler = AutoselectHandler()
# Load server configuration
server_config = load_server_config(custom_config_dir)
# Load dashboard config
aisbf_config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not aisbf_config_path.exists():
if custom_config_dir:
aisbf_config_path = Path(custom_config_dir) / 'aisbf.json'
else:
# Try installed location first
installed_path = Path(__file__).parent / 'aisbf.json'
if installed_path.exists():
aisbf_config_path = installed_path
else:
# Fall back to config subdirectory
aisbf_config_path = Path(__file__).parent / 'config' / 'aisbf.json'
if aisbf_config_path.exists():
with open(aisbf_config_path) as f:
aisbf_config = json.load(f)
server_config['dashboard_config'] = aisbf_config.get('dashboard', {})
else:
# Default with hashed password for 'admin'
server_config['dashboard_config'] = {
'username': 'admin',
'password': '8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918'
}
_initialized = True
logger.info("App initialization complete")
async def fetch_provider_models(provider_id: str) -> list:
"""Fetch models from provider API and cache them"""
global _model_cache, _model_cache_timestamps
try:
logger.debug(f"Fetching models from provider: {provider_id}")
# Create a dummy request object for the handler
from starlette.requests import Request
from starlette.datastructures import Headers
scope = {
"type": "http",
"method": "GET",
"headers": [],
"query_string": b"",
"path": f"/api/{provider_id}/models",
}
dummy_request = Request(scope)
# Fetch models from provider API
models = await request_handler.handle_model_list(dummy_request, provider_id)
# Cache the results
_model_cache[provider_id] = models
_model_cache_timestamps[provider_id] = time.time()
logger.info(f"Cached {len(models)} models from provider: {provider_id}")
return models
except Exception as e:
logger.warning(f"Failed to fetch models from provider {provider_id}: {e}")
return []
async def refresh_model_cache():
"""Background task to refresh model cache periodically"""
global _model_cache, _model_cache_timestamps
while True:
try:
await asyncio.sleep(_cache_refresh_interval)
logger.info("Starting periodic model cache refresh...")
# Refresh cache for all providers without local model config
for provider_id, provider_config in config.providers.items():
if not (hasattr(provider_config, 'models') and provider_config.models):
await fetch_provider_models(provider_id)
logger.info("Model cache refresh complete")
except Exception as e:
logger.error(f"Error in model cache refresh task: {e}")
async def get_provider_models(provider_id: str, provider_config) -> list:
"""Get models for a provider from local config or cache"""
global _model_cache, _model_cache_timestamps
# Check if provider requires API key and if it's configured
api_key_required = getattr(provider_config, 'api_key_required', False)
api_key = getattr(provider_config, 'api_key', None)
# If API key is required but not configured or is placeholder, skip this provider
if api_key_required:
if not api_key or api_key.startswith('YOUR_'):
logger.debug(f"Skipping provider {provider_id}: API key required but not configured")
return []
# If provider has local model config, use it
if hasattr(provider_config, 'models') and provider_config.models:
models = []
for model in provider_config.models:
model_id = f"{provider_id}/{model.name}"
models.append({
'id': model_id,
'object': 'model',
'created': int(time.time()),
'owned_by': provider_config.name,
'provider': provider_id,
'type': 'provider',
'model_name': model.name,
'context_size': getattr(model, 'context_size', None),
'capabilities': getattr(model, 'capabilities', []),
'source': 'local_config'
})
return models
# Check if we have cached models
if provider_id in _model_cache:
cache_age = time.time() - _model_cache_timestamps.get(provider_id, 0)
if cache_age < _cache_refresh_interval:
# Cache is still fresh, use it
cached_models = _model_cache[provider_id]
if cached_models: # Only return if we have actual models
# Add provider prefix to model IDs
models = []
for model in cached_models:
model_copy = model.copy()
model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
model_copy['provider'] = provider_id
model_copy['type'] = 'provider'
model_copy['source'] = 'api_cache'
models.append(model_copy)
return models
# No local config and no cache, try to fetch from API (only if API key is valid or not required)
if not api_key_required or (api_key and not api_key.startswith('YOUR_')):
try:
fetched_models = await fetch_provider_models(provider_id)
if fetched_models:
# Add provider prefix to model IDs
models = []
for model in fetched_models:
model_copy = model.copy()
model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
model_copy['provider'] = provider_id
model_copy['type'] = 'provider'
model_copy['source'] = 'api_cache'
models.append(model_copy)
return models
except Exception as e:
logger.debug(f"Failed to fetch models for provider {provider_id}: {e}")
# No models available - return empty list (don't show generic fallback)
return []
@app.on_event("startup")
async def startup_event():
"""Initialize app on startup (for uvicorn import case)."""
global config, server_config, _cache_refresh_task
if not _initialized:
# Use environment variable for config dir if set
custom_config_dir = get_config_dir()
initialize_app(custom_config_dir)
# Log configuration files loaded
if config and hasattr(config, '_loaded_files'):
logger.info("")
logger.info("=" * 80)
logger.info("=== CONFIGURATION FILES LOADED ===")
logger.info("=" * 80)
if 'providers' in config._loaded_files:
logger.info(f"Providers: {config._loaded_files['providers']}")
if 'rotations' in config._loaded_files:
logger.info(f"Rotations: {config._loaded_files['rotations']}")
if 'autoselect' in config._loaded_files:
logger.info(f"Autoselect: {config._loaded_files['autoselect']}")
if 'condensation' in config._loaded_files:
logger.info(f"Condensation: {config._loaded_files['condensation']}")
logger.info("=" * 80)
logger.info("")
# Start background task for model cache refresh
if _cache_refresh_task is None:
_cache_refresh_task = asyncio.create_task(refresh_model_cache())
logger.info(f"Started model cache refresh task (interval: {_cache_refresh_interval/3600} hours)")
# In debug mode, validate provider configurations
AISBF_DEBUG = os.environ.get('AISBF_DEBUG', '').lower() in ('true', '1', 'yes')
if AISBF_DEBUG and config:
logger.info("")
logger.info("=" * 80)
logger.info("=== PROVIDER CONFIGURATION VALIDATION (DEBUG MODE) ===")
logger.info("=" * 80)
for provider_id, provider_config in config.providers.items():
logger.info(f"")
logger.info(f"Provider: {provider_id}")
logger.info(f" Type: {provider_config.type}")
logger.info(f" Endpoint: {provider_config.endpoint}")
logger.info(f" API Key Required: {provider_config.api_key_required}")
# Check if API key is configured
if provider_config.api_key_required:
if provider_config.api_key:
logger.info(f" API Key: Configured ✓")
logger.info(f" Status: Ready to use")
else:
logger.warning(f" API Key: NOT CONFIGURED ✗")
logger.warning(f" Status: WILL BE SKIPPED - API key required but not provided")
logger.warning(f" Action: Add api_key to provider configuration in providers.json")
else:
logger.info(f" API Key: Not required")
logger.info(f" Status: Ready to use")
# Show model count if available
if provider_config.models:
logger.info(f" Models Configured: {len(provider_config.models)}")
for model in provider_config.models[:3]: # Show first 3 models
logger.info(f" - {model.name}")
if len(provider_config.models) > 3:
logger.info(f" ... and {len(provider_config.models) - 3} more")
else:
logger.info(f" Models Configured: None (will use provider's default models)")
logger.info("")
logger.info("=" * 80)
logger.info("")
logger.info(f"=== AISBF Server Started ===")
logger.info(f"Available providers: {list(config.providers.keys()) if config else []}")
logger.info(f"Available rotations: {list(config.rotations.keys()) if config else []}")
logger.info(f"Available autoselect: {list(config.autoselect.keys()) if config else []}")
# Authentication middleware
@app.middleware("http")
async def auth_middleware(request: Request, call_next):
"""Check API token authentication if enabled"""
if server_config.get('auth_enabled', False):
if server_config and server_config.get('auth_enabled', False):
# Skip auth for root endpoint and dashboard routes
if request.url.path == "/" or request.url.path.startswith("/dashboard"):
response = await call_next(request)
return response
# Skip auth for public models endpoints (GET only)
if request.method == "GET" and request.url.path in ["/api/models", "/api/v1/models"]:
response = await call_next(request)
return response
# Check for Authorization header
auth_header = request.headers.get('Authorization', '')
......@@ -447,12 +831,12 @@ app.add_middleware(
@app.get("/dashboard/login", response_class=HTMLResponse)
async def dashboard_login_page(request: Request):
"""Show dashboard login page"""
return templates.TemplateResponse("dashboard/login.html", {"request": request})
return templates.TemplateResponse("dashboard/login.html", {"request": request, "session": request.session})
@app.post("/dashboard/login")
async def dashboard_login(request: Request, username: str = Form(...), password: str = Form(...)):
"""Handle dashboard login"""
dashboard_config = server_config.get('dashboard_config', {})
dashboard_config = server_config.get('dashboard_config', {}) if server_config else {}
stored_username = dashboard_config.get('username', 'admin')
stored_password_hash = dashboard_config.get('password', '8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918')
......@@ -463,19 +847,19 @@ async def dashboard_login(request: Request, username: str = Form(...), password:
if username == stored_username and password_hash == stored_password_hash:
request.session['logged_in'] = True
request.session['username'] = username
return RedirectResponse(url="/dashboard", status_code=303)
return templates.TemplateResponse("dashboard/login.html", {"request": request, "error": "Invalid credentials"})
return RedirectResponse(url=url_for(request, "/dashboard"), status_code=303)
return templates.TemplateResponse("dashboard/login.html", {"request": request, "session": request.session, "error": "Invalid credentials"})
@app.get("/dashboard/logout")
async def dashboard_logout(request: Request):
"""Handle dashboard logout"""
request.session.clear()
return RedirectResponse(url="/dashboard/login", status_code=303)
return RedirectResponse(url=url_for(request, "/dashboard/login"), status_code=303)
def require_dashboard_auth(request: Request):
"""Check if user is logged in to dashboard"""
if not request.session.get('logged_in'):
return RedirectResponse(url="/dashboard/login", status_code=303)
return RedirectResponse(url=url_for(request, "/dashboard/login"), status_code=303)
return None
@app.get("/dashboard", response_class=HTMLResponse)
......@@ -507,13 +891,23 @@ async def dashboard_providers(request: Request):
config_path = Path(__file__).parent / 'config' / 'providers.json'
with open(config_path) as f:
config_content = f.read()
full_config = json.load(f)
# Extract just the providers object (handle both nested and flat structures)
if 'providers' in full_config and isinstance(full_config['providers'], dict):
providers_data = full_config['providers']
else:
# Fallback for flat structure (backward compatibility)
providers_data = {k: v for k, v in full_config.items() if k != 'condensation'}
return templates.TemplateResponse("dashboard/edit_config.html", {
# Check for success parameter
success = request.query_params.get('success')
return templates.TemplateResponse("dashboard/providers.html", {
"request": request,
"session": request.session,
"title": "Providers Configuration",
"config_content": config_content
"providers_json": json.dumps(providers_data),
"success": "Configuration saved successfully! Restart server for changes to take effect." if success else None
})
@app.post("/dashboard/providers")
......@@ -525,27 +919,58 @@ async def dashboard_providers_save(request: Request, config: str = Form(...)):
try:
# Validate JSON
json.loads(config)
providers_data = json.loads(config)
# Apply defaults: if condense_method is set but condense_context is not, default to 80
for provider_key, provider in providers_data.items():
if 'models' in provider and isinstance(provider['models'], list):
for model in provider['models']:
if 'condense_method' in model and model.get('condense_method'):
if 'condense_context' not in model or model.get('condense_context') is None:
model['condense_context'] = 80
# Save to file
# Load existing config to preserve structure
config_path = Path.home() / '.aisbf' / 'providers.json'
config_path.parent.mkdir(parents=True, exist_ok=True)
with open(config_path, 'w') as f:
f.write(config)
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / 'providers.json'
# Read existing config to preserve condensation settings
with open(config_path) as f:
full_config = json.load(f)
# Update providers section while preserving other keys
full_config['providers'] = providers_data
return templates.TemplateResponse("dashboard/edit_config.html", {
# Save to file with full structure
save_path = Path.home() / '.aisbf' / 'providers.json'
save_path.parent.mkdir(parents=True, exist_ok=True)
with open(save_path, 'w') as f:
json.dump(full_config, f, indent=2)
return templates.TemplateResponse("dashboard/providers.html", {
"request": request,
"session": request.session,
"title": "Providers Configuration",
"config_content": config,
"providers_json": json.dumps(providers_data),
"success": "Configuration saved successfully! Restart server for changes to take effect."
})
except json.JSONDecodeError as e:
return templates.TemplateResponse("dashboard/edit_config.html", {
# Reload current config on error
config_path = Path.home() / '.aisbf' / 'providers.json'
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / 'providers.json'
with open(config_path) as f:
full_config = json.load(f)
# Extract providers
if 'providers' in full_config and isinstance(full_config['providers'], dict):
providers_data = full_config['providers']
else:
providers_data = {k: v for k, v in full_config.items() if k != 'condensation'}
return templates.TemplateResponse("dashboard/providers.html", {
"request": request,
"session": request.session,
"title": "Providers Configuration",
"config_content": config,
"providers_json": json.dumps(providers_data),
"error": f"Invalid JSON: {str(e)}"
})
......@@ -561,13 +986,20 @@ async def dashboard_rotations(request: Request):
config_path = Path(__file__).parent / 'config' / 'rotations.json'
with open(config_path) as f:
config_content = f.read()
rotations_data = json.load(f)
# Get available providers
available_providers = list(config.providers.keys()) if config else []
# Check for success parameter
success = request.query_params.get('success')
return templates.TemplateResponse("dashboard/edit_config.html", {
return templates.TemplateResponse("dashboard/rotations.html", {
"request": request,
"session": request.session,
"title": "Rotations Configuration",
"config_content": config_content
"rotations_json": json.dumps(rotations_data),
"available_providers": json.dumps(available_providers),
"success": "Configuration saved successfully! Restart server for changes to take effect." if success else None
})
@app.post("/dashboard/rotations")
......@@ -578,25 +1010,48 @@ async def dashboard_rotations_save(request: Request, config: str = Form(...)):
return auth_check
try:
json.loads(config)
rotations_data = json.loads(config)
# Apply defaults: if condense_method is set but condense_context is not, default to 80
if 'rotations' in rotations_data:
for rotation_key, rotation in rotations_data['rotations'].items():
if 'providers' in rotation and isinstance(rotation['providers'], list):
for provider in rotation['providers']:
if 'models' in provider and isinstance(provider['models'], list):
for model in provider['models']:
if 'condense_method' in model and model.get('condense_method'):
if 'condense_context' not in model or model.get('condense_context') is None:
model['condense_context'] = 80
config_path = Path.home() / '.aisbf' / 'rotations.json'
config_path.parent.mkdir(parents=True, exist_ok=True)
with open(config_path, 'w') as f:
f.write(config)
json.dump(rotations_data, f, indent=2)
return templates.TemplateResponse("dashboard/edit_config.html", {
available_providers = list(config.providers.keys()) if config else []
return templates.TemplateResponse("dashboard/rotations.html", {
"request": request,
"session": request.session,
"title": "Rotations Configuration",
"config_content": config,
"rotations_json": json.dumps(rotations_data),
"available_providers": json.dumps(available_providers),
"success": "Configuration saved successfully! Restart server for changes to take effect."
})
except json.JSONDecodeError as e:
return templates.TemplateResponse("dashboard/edit_config.html", {
# Reload current config on error
config_path = Path.home() / '.aisbf' / 'rotations.json'
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / 'rotations.json'
with open(config_path) as f:
rotations_data = json.load(f)
available_providers = list(config.providers.keys()) if config else []
return templates.TemplateResponse("dashboard/rotations.html", {
"request": request,
"session": request.session,
"title": "Rotations Configuration",
"config_content": config,
"rotations_json": json.dumps(rotations_data),
"available_providers": json.dumps(available_providers),
"error": f"Invalid JSON: {str(e)}"
})
......@@ -612,13 +1067,52 @@ async def dashboard_autoselect(request: Request):
config_path = Path(__file__).parent / 'config' / 'autoselect.json'
with open(config_path) as f:
config_content = f.read()
autoselect_data = json.load(f)
# Get available rotations
available_rotations = list(config.rotations.keys()) if config else []
return templates.TemplateResponse("dashboard/edit_config.html", {
# Get available provider models
available_models = []
# Add rotation IDs
for rotation_id in available_rotations:
available_models.append({
'id': rotation_id,
'name': f'{rotation_id} (rotation)',
'type': 'rotation'
})
# Add provider models
providers_path = Path.home() / '.aisbf' / 'providers.json'
if not providers_path.exists():
providers_path = Path(__file__).parent / 'config' / 'providers.json'
if providers_path.exists():
with open(providers_path) as f:
providers_config = json.load(f)
providers_data = providers_config.get('providers', {})
for provider_id, provider in providers_data.items():
if 'models' in provider and isinstance(provider['models'], list):
for model in provider['models']:
model_id = f"{provider_id}/{model['name']}"
available_models.append({
'id': model_id,
'name': f"{model_id} (provider model)",
'type': 'provider'
})
# Check for success parameter
success = request.query_params.get('success')
return templates.TemplateResponse("dashboard/autoselect.html", {
"request": request,
"session": request.session,
"title": "Autoselect Configuration",
"config_content": config_content
"autoselect_json": json.dumps(autoselect_data),
"available_rotations": json.dumps(available_rotations),
"available_models": json.dumps(available_models),
"success": "Configuration saved successfully! Restart server for changes to take effect." if success else None
})
@app.post("/dashboard/autoselect")
......@@ -629,53 +1123,120 @@ async def dashboard_autoselect_save(request: Request, config: str = Form(...)):
return auth_check
try:
json.loads(config)
autoselect_data = json.loads(config)
config_path = Path.home() / '.aisbf' / 'autoselect.json'
config_path.parent.mkdir(parents=True, exist_ok=True)
with open(config_path, 'w') as f:
f.write(config)
json.dump(autoselect_data, f, indent=2)
available_rotations = list(config.rotations.keys()) if config else []
return templates.TemplateResponse("dashboard/edit_config.html", {
return templates.TemplateResponse("dashboard/autoselect.html", {
"request": request,
"session": request.session,
"title": "Autoselect Configuration",
"config_content": config,
"autoselect_json": json.dumps(autoselect_data),
"available_rotations": json.dumps(available_rotations),
"success": "Configuration saved successfully! Restart server for changes to take effect."
})
except json.JSONDecodeError as e:
return templates.TemplateResponse("dashboard/edit_config.html", {
# Reload current config on error
config_path = Path.home() / '.aisbf' / 'autoselect.json'
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / 'autoselect.json'
with open(config_path) as f:
autoselect_data = json.load(f)
available_rotations = list(config.rotations.keys()) if config else []
return templates.TemplateResponse("dashboard/autoselect.html", {
"request": request,
"session": request.session,
"title": "Autoselect Configuration",
"config_content": config,
"autoselect_json": json.dumps(autoselect_data),
"available_rotations": json.dumps(available_rotations),
"error": f"Invalid JSON: {str(e)}"
})
@app.get("/dashboard/condensation", response_class=HTMLResponse)
async def dashboard_condensation(request: Request):
"""Edit condensation prompts"""
@app.get("/dashboard/prompts", response_class=HTMLResponse)
async def dashboard_prompts(request: Request):
"""Edit prompt templates"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
# Load condensation_conversational.md
config_path = Path.home() / '.aisbf' / 'condensation_conversational.md'
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / 'condensation_conversational.md'
# Define available prompts
prompt_files = [
{'key': 'condensation_conversational', 'name': 'Condensation - Conversational', 'filename': 'condensation_conversational.md'},
{'key': 'condensation_semantic', 'name': 'Condensation - Semantic', 'filename': 'condensation_semantic.md'},
{'key': 'autoselect', 'name': 'Autoselect - Model Selection', 'filename': 'autoselect.md'},
]
prompts_data = []
for prompt_file in prompt_files:
config_path = Path.home() / '.aisbf' / prompt_file['filename']
if not config_path.exists():
config_path = Path(__file__).parent / 'config' / prompt_file['filename']
if config_path.exists():
with open(config_path) as f:
content = f.read()
prompts_data.append({
'key': prompt_file['key'],
'name': prompt_file['name'],
'filename': prompt_file['filename'],
'content': content
})
with open(config_path) as f:
config_content = f.read()
# Check for success parameter
success = request.query_params.get('success')
return templates.TemplateResponse("dashboard/edit_config.html", {
return templates.TemplateResponse("dashboard/prompts.html", {
"request": request,
"session": request.session,
"title": "Condensation Prompts (Conversational)",
"config_content": config_content
"prompts": prompt_files,
"prompts_data": json.dumps(prompts_data),
"success": "Prompt saved successfully!" if success else None
})
@app.post("/dashboard/prompts")
async def dashboard_prompts_save(request: Request, prompt_key: str = Form(...), prompt_content: str = Form(...)):
"""Save prompt template"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
# Map prompt keys to filenames
prompt_map = {
'condensation_conversational': 'condensation_conversational.md',
'condensation_semantic': 'condensation_semantic.md',
'autoselect': 'autoselect.md',
}
if prompt_key not in prompt_map:
return templates.TemplateResponse("dashboard/prompts.html", {
"request": request,
"session": request.session,
"prompts": [],
"prompts_data": "[]",
"error": "Invalid prompt key"
})
filename = prompt_map[prompt_key]
config_path = Path.home() / '.aisbf' / filename
config_path.parent.mkdir(parents=True, exist_ok=True)
with open(config_path, 'w') as f:
f.write(prompt_content)
return RedirectResponse(url=url_for(request, "/dashboard/prompts?success=1"), status_code=303)
@app.get("/dashboard/condensation", response_class=HTMLResponse)
async def dashboard_condensation(request: Request):
"""Redirect to prompts page for backward compatibility"""
return RedirectResponse(url=url_for(request, "/dashboard/prompts"), status_code=303)
@app.post("/dashboard/condensation")
async def dashboard_condensation_save(request: Request, config: str = Form(...)):
"""Save condensation prompts"""
"""Save condensation prompts - backward compatibility"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
......@@ -685,13 +1246,7 @@ async def dashboard_condensation_save(request: Request, config: str = Form(...))
with open(config_path, 'w') as f:
f.write(config)
return templates.TemplateResponse("dashboard/edit_config.html", {
"request": request,
"session": request.session,
"title": "Condensation Prompts (Conversational)",
"config_content": config,
"success": "Prompts saved successfully!"
})
return RedirectResponse(url=url_for(request, "/dashboard/prompts?success=1"), status_code=303)
@app.get("/dashboard/settings", response_class=HTMLResponse)
async def dashboard_settings(request: Request):
......@@ -724,7 +1279,8 @@ async def dashboard_settings_save(
auth_tokens: str = Form(""),
dashboard_username: str = Form(...),
dashboard_password: str = Form(""),
internal_model_id: str = Form(...)
condensation_model_id: str = Form(...),
autoselect_model_id: str = Form(...)
):
"""Save server settings"""
auth_check = require_dashboard_auth(request)
......@@ -749,7 +1305,8 @@ async def dashboard_settings_save(
if dashboard_password: # Only update if provided - hash the password
password_hash = hashlib.sha256(dashboard_password.encode()).hexdigest()
aisbf_config['dashboard']['password'] = password_hash
aisbf_config['internal_model']['model_id'] = internal_model_id
aisbf_config['internal_model']['condensation_model_id'] = condensation_model_id
aisbf_config['internal_model']['autoselect_model_id'] = autoselect_model_id
# Save config
config_path = Path.home() / '.aisbf' / 'aisbf.json'
......@@ -788,6 +1345,93 @@ async def dashboard_restart(request: Request):
return JSONResponse({"message": "Server is restarting..."})
@app.get("/dashboard/docs", response_class=HTMLResponse)
async def dashboard_docs(request: Request):
"""Display documentation"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
# Try to find DOCUMENTATION.md
doc_path = Path(__file__).parent / 'DOCUMENTATION.md'
if not doc_path.exists():
doc_path = Path.home() / '.aisbf' / 'DOCUMENTATION.md'
if doc_path.exists():
with open(doc_path, encoding='utf-8') as f:
markdown_content = f.read()
# Convert markdown to HTML with extensions for better formatting
html_content = markdown.markdown(
markdown_content,
extensions=['fenced_code', 'tables', 'nl2br', 'sane_lists']
)
else:
html_content = "<p>Documentation file not found.</p>"
return templates.TemplateResponse("dashboard/docs.html", {
"request": request,
"session": request.session,
"content": html_content,
"title": "Documentation"
})
@app.get("/dashboard/about", response_class=HTMLResponse)
async def dashboard_about(request: Request):
"""Display README/About"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
# Try to find README.md
readme_path = Path(__file__).parent / 'README.md'
if not readme_path.exists():
readme_path = Path.home() / '.aisbf' / 'README.md'
if readme_path.exists():
with open(readme_path, encoding='utf-8') as f:
markdown_content = f.read()
# Convert markdown to HTML with extensions for better formatting
html_content = markdown.markdown(
markdown_content,
extensions=['fenced_code', 'tables', 'nl2br', 'sane_lists']
)
else:
html_content = "<p>README file not found.</p>"
return templates.TemplateResponse("dashboard/docs.html", {
"request": request,
"session": request.session,
"content": html_content,
"title": "About"
})
@app.get("/dashboard/license", response_class=HTMLResponse)
async def dashboard_license(request: Request):
"""Display License"""
auth_check = require_dashboard_auth(request)
if auth_check:
return auth_check
# Try to find LICENSE.txt
license_path = Path(__file__).parent / 'LICENSE.txt'
if not license_path.exists():
license_path = Path.home() / '.aisbf' / 'LICENSE.txt'
if license_path.exists():
with open(license_path, encoding='utf-8') as f:
content = f.read()
# Convert to HTML with pre tags to preserve formatting
html_content = f"<pre style='white-space: pre-wrap; word-wrap: break-word; background: #0f3460; padding: 20px; border-radius: 6px; color: #e0e0e0; font-family: inherit;'>{content}</pre>"
else:
html_content = "<p>License file not found.</p>"
return templates.TemplateResponse("dashboard/docs.html", {
"request": request,
"session": request.session,
"content": html_content,
"title": "License"
})
def parse_provider_from_model(model: str) -> tuple[str, str]:
"""
Parse provider and model from model field.
......@@ -814,6 +1458,58 @@ async def root():
"autoselect": list(config.autoselect.keys())
}
@app.get("/health")
async def health():
return {"status": "ok"}
@app.get("/api/v1/models/{model_id}")
async def v1_get_model(model_id: str, request: Request):
"""Get a specific model by ID (OpenAI-compatible endpoint)"""
# First try to find in all models
all_models_response = await v1_list_all_models(request)
all_models = all_models_response.get("data", [])
for model in all_models:
if model.get("id") == model_id:
return model
from fastapi import HTTPException
raise HTTPException(status_code=404, detail=f"Model '{model_id}' not found")
@app.post("/api/v1/completions")
async def v1_completions(request: Request):
"""
Legacy text completions endpoint (OpenAI-compatible)
Maps to chat/completions for compatibility
"""
# Get the request body
body = await request.body()
import json
data = json.loads(body) if body else {}
# Convert completion request to chat completion
prompt = data.get("prompt", "")
model = data.get("model", "")
max_tokens = data.get("max_tokens", 2048)
temperature = data.get("temperature", 1.0)
# Build chat messages from prompt
messages = [{"role": "user", "content": prompt}]
# Create a new request for chat/completions
chat_request = {
"model": model,
"messages": messages,
"max_tokens": max_tokens,
"temperature": temperature
}
# Call chat completions handler
return await v1_chat_completions(chat_request, request)
# Standard OpenAI-compatible v1 endpoints
@app.post("/api/v1/chat/completions")
async def v1_chat_completions(request: Request, body: ChatCompletionRequest):
......@@ -827,85 +1523,152 @@ async def v1_chat_completions(request: Request, body: ChatCompletionRequest):
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model' (e.g., 'openai/gpt-4')"
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
logger.info(f"Parsed provider: {provider_id}, model: {actual_model}")
# Update body with actual model name
body_dict = body.model_dump()
body_dict['model'] = actual_model
# Check if it's an autoselect
if provider_id in config.autoselect:
# PATH 3: Check if it's an autoselect (format: autoselect/{name})
if provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
body_dict['model'] = actual_model
if body.stream:
return await autoselect_handler.handle_autoselect_streaming_request(provider_id, body_dict)
return await autoselect_handler.handle_autoselect_streaming_request(actual_model, body_dict)
else:
return await autoselect_handler.handle_autoselect_request(provider_id, body_dict)
# Check if it's a rotation
if provider_id in config.rotations:
return await rotation_handler.handle_rotation_request(provider_id, body_dict)
return await autoselect_handler.handle_autoselect_request(actual_model, body_dict)
# PATH 2: Check if it's a rotation (format: rotation/{name})
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
body_dict['model'] = actual_model
return await rotation_handler.handle_rotation_request(actual_model, body_dict)
# Check if it's a provider
# PATH 1: Direct provider model (format: {provider}/{model})
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
detail=f"Provider '{provider_id}' not found. Available providers: {list(config.providers.keys())}, or use 'rotation/name' or 'autoselect/name'"
)
# Handle as direct provider request
body_dict['model'] = actual_model
if body.stream:
return await request_handler.handle_streaming_chat_completion(request, provider_id, body_dict)
else:
return await request_handler.handle_chat_completion(request, provider_id, body_dict)
@app.get("/api/models")
async def list_all_models(request: Request):
"""List all available models from all providers (public endpoint)"""
logger.info("=== LIST ALL MODELS REQUEST ===")
all_models = []
# PATH 1: Add provider models (from local config or cached API results)
for provider_id, provider_config in config.providers.items():
try:
provider_models = await get_provider_models(provider_id, provider_config)
all_models.extend(provider_models)
except Exception as e:
logger.warning(f"Error listing models for provider {provider_id}: {e}")
# PATH 2: Add rotations as rotation/{rotation_name}
for rotation_id, rotation_config in config.rotations.items():
try:
all_models.append({
'id': f"rotation/{rotation_id}",
'object': 'model',
'created': int(time.time()),
'owned_by': 'aisbf-rotation',
'type': 'rotation',
'rotation_id': rotation_id,
'model_name': rotation_config.model_name
})
except Exception as e:
logger.warning(f"Error listing rotation {rotation_id}: {e}")
# PATH 3: Add autoselect as autoselect/{autoselect_name}
for autoselect_id, autoselect_config in config.autoselect.items():
try:
all_models.append({
'id': f"autoselect/{autoselect_id}",
'object': 'model',
'created': int(time.time()),
'owned_by': 'aisbf-autoselect',
'type': 'autoselect',
'autoselect_id': autoselect_id,
'model_name': autoselect_config.model_name,
'description': autoselect_config.description
})
except Exception as e:
logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
logger.info(f"Returning {len(all_models)} total models")
return {"object": "list", "data": all_models}
@app.get("/api/v1/models")
async def v1_list_all_models(request: Request):
"""List all available models from all providers"""
"""List all available models from all providers (OpenAI-compatible endpoint)"""
logger.info("=== V1 LIST ALL MODELS REQUEST ===")
all_models = []
# Add provider models
for provider_id in config.providers.keys():
# PATH 1: Add provider models (from local config or cached API results)
for provider_id, provider_config in config.providers.items():
try:
models = await request_handler.handle_model_list(request, provider_id)
for model in models:
# Prepend provider to model ID
model['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
model['provider'] = provider_id
all_models.append(model)
provider_models = await get_provider_models(provider_id, provider_config)
all_models.extend(provider_models)
except Exception as e:
logger.warning(f"Error listing models for provider {provider_id}: {e}")
# Add rotation models
for rotation_id in config.rotations.keys():
# PATH 2: Add rotations as rotation/{rotation_name}
for rotation_id, rotation_config in config.rotations.items():
try:
models = await rotation_handler.handle_rotation_model_list(rotation_id)
for model in models:
model['id'] = f"{rotation_id}/{model.get('name', '')}"
model['type'] = 'rotation'
all_models.append(model)
all_models.append({
'id': f"rotation/{rotation_id}",
'object': 'model',
'created': int(time.time()),
'owned_by': 'aisbf-rotation',
'type': 'rotation',
'rotation_id': rotation_id,
'model_name': rotation_config.model_name
})
except Exception as e:
logger.warning(f"Error listing models for rotation {rotation_id}: {e}")
logger.warning(f"Error listing rotation {rotation_id}: {e}")
# Add autoselect models
for autoselect_id in config.autoselect.keys():
# PATH 3: Add autoselect as autoselect/{autoselect_name}
for autoselect_id, autoselect_config in config.autoselect.items():
try:
models = await autoselect_handler.handle_autoselect_model_list(autoselect_id)
for model in models:
model['id'] = f"{autoselect_id}/{model.get('name', model.get('id', ''))}"
model['type'] = 'autoselect'
all_models.append(model)
all_models.append({
'id': f"autoselect/{autoselect_id}",
'object': 'model',
'created': int(time.time()),
'owned_by': 'aisbf-autoselect',
'type': 'autoselect',
'autoselect_id': autoselect_id,
'model_name': autoselect_config.model_name,
'description': autoselect_config.description
})
except Exception as e:
logger.warning(f"Error listing models for autoselect {autoselect_id}: {e}")
logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
logger.info(f"Returning {len(all_models)} total models")
return {"object": "list", "data": all_models}
@app.post("/api/v1/audio/transcriptions")
async def v1_audio_transcriptions(request: Request):
"""Standard audio transcription endpoint"""
"""Standard audio transcription endpoint (supports all three proxy paths)"""
logger.info("=== V1 AUDIO TRANSCRIPTION REQUEST ===")
form = await request.form()
......@@ -916,7 +1679,46 @@ async def v1_audio_transcriptions(request: Request):
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model' (e.g., 'openai/whisper-1')"
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
# Create new form data with updated model
......@@ -932,7 +1734,7 @@ async def v1_audio_transcriptions(request: Request):
@app.post("/api/v1/audio/speech")
async def v1_audio_speech(request: Request, body: dict):
"""Standard text-to-speech endpoint"""
"""Standard text-to-speech endpoint (supports all three proxy paths)"""
logger.info("=== V1 TEXT-TO-SPEECH REQUEST ===")
model = body.get('model', '')
......@@ -941,7 +1743,46 @@ async def v1_audio_speech(request: Request, body: dict):
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model' (e.g., 'openai/tts-1')"
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
......@@ -949,7 +1790,7 @@ async def v1_audio_speech(request: Request, body: dict):
@app.post("/api/v1/images/generations")
async def v1_image_generations(request: Request, body: dict):
"""Standard image generation endpoint"""
"""Standard image generation endpoint (supports all three proxy paths)"""
logger.info("=== V1 IMAGE GENERATION REQUEST ===")
model = body.get('model', '')
......@@ -958,7 +1799,46 @@ async def v1_image_generations(request: Request, body: dict):
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model' (e.g., 'openai/dall-e-3')"
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
......@@ -966,7 +1846,7 @@ async def v1_image_generations(request: Request, body: dict):
@app.post("/api/v1/embeddings")
async def v1_embeddings(request: Request, body: dict):
"""Standard embeddings endpoint"""
"""Standard embeddings endpoint (supports all three proxy paths)"""
logger.info("=== V1 EMBEDDINGS REQUEST ===")
model = body.get('model', '')
......@@ -975,7 +1855,46 @@ async def v1_embeddings(request: Request, body: dict):
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model' (e.g., 'openai/text-embedding-ada-002')"
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
......@@ -1254,63 +2173,247 @@ async def list_models(request: Request, provider_id: str):
logger.error(f"Error handling list_models: {str(e)}", exc_info=True)
raise
# Audio endpoints
@app.post("/api/{provider_id}/audio/transcriptions")
async def audio_transcriptions(provider_id: str, request: Request):
"""Handle audio transcription requests"""
logger.info(f"=== AUDIO TRANSCRIPTION REQUEST ===")
logger.info(f"Provider ID: {provider_id}")
# Audio endpoints (model specified in request as provider/model, rotation/name, or autoselect/name)
@app.post("/api/audio/transcriptions")
async def audio_transcriptions(request: Request):
"""Handle audio transcription requests (supports all three proxy paths)"""
logger.info("=== AUDIO TRANSCRIPTION REQUEST ===")
# Get form data (audio file upload)
form = await request.form()
model = form.get('model', '')
try:
result = await request_handler.handle_audio_transcription(request, provider_id, form)
return result
except Exception as e:
logger.error(f"Error handling audio transcription: {str(e)}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
provider_id, actual_model = parse_provider_from_model(model)
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
# Select a provider from the rotation using weighted random selection
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
# Use the fallback model from autoselect config
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
# Parse the fallback to get provider and model
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
# Fallback is a rotation, select from it
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
# Validate provider exists
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
# Create new form data with updated model
from starlette.datastructures import FormData
updated_form = FormData()
for key, value in form.items():
if key == 'model':
updated_form[key] = actual_model
else:
updated_form[key] = value
return await request_handler.handle_audio_transcription(request, provider_id, updated_form)
@app.post("/api/{provider_id}/audio/speech")
async def audio_speech(provider_id: str, request: Request, body: dict):
"""Handle text-to-speech requests"""
logger.info(f"=== TEXT-TO-SPEECH REQUEST ===")
logger.info(f"Provider ID: {provider_id}")
@app.post("/api/audio/speech")
async def audio_speech(request: Request, body: dict):
"""Handle text-to-speech requests (supports all three proxy paths)"""
logger.info("=== TEXT-TO-SPEECH REQUEST ===")
try:
result = await request_handler.handle_text_to_speech(request, provider_id, body)
return result
except Exception as e:
logger.error(f"Error handling text-to-speech: {str(e)}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
model = body.get('model', '')
provider_id, actual_model = parse_provider_from_model(model)
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
return await request_handler.handle_text_to_speech(request, provider_id, body)
# Image endpoints
@app.post("/api/{provider_id}/images/generations")
async def image_generations(provider_id: str, request: Request, body: dict):
"""Handle image generation requests"""
logger.info(f"=== IMAGE GENERATION REQUEST ===")
logger.info(f"Provider ID: {provider_id}")
# Image endpoints (supports all three proxy paths)
@app.post("/api/images/generations")
async def image_generations(request: Request, body: dict):
"""Handle image generation requests (supports all three proxy paths)"""
logger.info("=== IMAGE GENERATION REQUEST ===")
try:
result = await request_handler.handle_image_generation(request, provider_id, body)
return result
except Exception as e:
logger.error(f"Error handling image generation: {str(e)}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
model = body.get('model', '')
provider_id, actual_model = parse_provider_from_model(model)
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
return await request_handler.handle_image_generation(request, provider_id, body)
# Embeddings endpoint
@app.post("/api/{provider_id}/embeddings")
async def embeddings(provider_id: str, request: Request, body: dict):
"""Handle embeddings requests"""
logger.info(f"=== EMBEDDINGS REQUEST ===")
logger.info(f"Provider ID: {provider_id}")
# Embeddings endpoint (supports all three proxy paths)
@app.post("/api/embeddings")
async def embeddings(request: Request, body: dict):
"""Handle embeddings requests (supports all three proxy paths)"""
logger.info("=== EMBEDDINGS REQUEST ===")
try:
result = await request_handler.handle_embeddings(request, provider_id, body)
return result
except Exception as e:
logger.error(f"Error handling embeddings: {str(e)}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
model = body.get('model', '')
provider_id, actual_model = parse_provider_from_model(model)
if not provider_id:
raise HTTPException(
status_code=400,
detail="Model must be in format 'provider/model', 'rotation/name', or 'autoselect/name'"
)
# Handle rotation
if provider_id == "rotation":
if actual_model not in config.rotations:
raise HTTPException(
status_code=400,
detail=f"Rotation '{actual_model}' not found. Available: {list(config.rotations.keys())}"
)
selected_provider, selected_model = rotation_handler._select_provider_and_model(actual_model)
provider_id = selected_provider
actual_model = selected_model
# Handle autoselect
elif provider_id == "autoselect":
if actual_model not in config.autoselect:
raise HTTPException(
status_code=400,
detail=f"Autoselect '{actual_model}' not found. Available: {list(config.autoselect.keys())}"
)
autoselect_config = config.autoselect[actual_model]
fallback = autoselect_config.fallback
if '/' in fallback:
provider_id, actual_model = fallback.split('/', 1)
else:
if fallback in config.rotations:
selected_provider, selected_model = rotation_handler._select_provider_and_model(fallback)
provider_id = selected_provider
actual_model = selected_model
else:
raise HTTPException(
status_code=400,
detail=f"Invalid fallback configuration for autoselect '{actual_model}'"
)
if provider_id not in config.providers:
raise HTTPException(
status_code=400,
detail=f"Provider '{provider_id}' not found. Available: {list(config.providers.keys())}"
)
body['model'] = actual_model
return await request_handler.handle_embeddings(request, provider_id, body)
# Content proxy endpoint
@app.get("/api/proxy/{content_id}")
......@@ -1382,42 +2485,11 @@ Examples:
global _original_argv
_original_argv = sys.argv.copy()
# Set custom config directory if provided
if args.config:
set_config_dir(args.config)
logger.info(f"Using custom config directory: {args.config}")
# Import config after setting custom directory
global config, request_handler, rotation_handler, autoselect_handler, server_config
from aisbf.config import config
from aisbf.handlers import RequestHandler, RotationHandler, AutoselectHandler
# Initialize handlers
request_handler = RequestHandler()
rotation_handler = RotationHandler()
autoselect_handler = AutoselectHandler()
# Load server configuration
server_config = load_server_config(args.config)
# Add session middleware for dashboard
secret_key = secrets.token_urlsafe(32)
app.add_middleware(SessionMiddleware, secret_key=secret_key)
# Initialize app (this sets config, handlers, server_config)
initialize_app(args.config)
# Load dashboard config
aisbf_config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not aisbf_config_path.exists():
if args.config:
aisbf_config_path = Path(args.config) / 'aisbf.json'
else:
aisbf_config_path = Path(__file__).parent / 'config' / 'aisbf.json'
if aisbf_config_path.exists():
with open(aisbf_config_path) as f:
aisbf_config = json.load(f)
server_config['dashboard_config'] = aisbf_config.get('dashboard', {})
else:
server_config['dashboard_config'] = {'username': 'admin', 'password': 'admin'}
# Get the now-initialized server_config
global server_config
# CLI arguments take precedence over config file
host = args.host if args.host else server_config['host']
......
requirements.txt
View file @ a3506741
......@@ -16,4 +16,5 @@ transformers
jinja2
itsdangerous
bs4
protobuf>=3.20,<4
\ No newline at end of file
protobuf>=3.20,<4
markdown
\ No newline at end of file
setup.py
View file @ a3506741
......@@ -108,6 +108,11 @@ setup(
'templates/dashboard/index.html',
'templates/dashboard/edit_config.html',
'templates/dashboard/settings.html',
'templates/dashboard/providers.html',
'templates/dashboard/rotations.html',
'templates/dashboard/autoselect.html',
'templates/dashboard/prompts.html',
'templates/dashboard/docs.html',
]),
],
entry_points={
......
templates/base.html
View file @ a3506741
......@@ -22,36 +22,36 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<title>{% block title %}AISBF Dashboard{% endblock %}</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; background: #f5f5f5; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; background: #1a1a2e; color: #e0e0e0; }
.container { max-width: 1200px; margin: 0 auto; padding: 20px; }
.header { background: #2c3e50; color: white; padding: 20px 0; margin-bottom: 30px; }
.header { background: #16213e; color: white; padding: 20px 0; margin-bottom: 30px; border-bottom: 2px solid #0f3460; }
.header h1 { font-size: 24px; font-weight: 600; display: inline-block; }
.header-actions { float: right; }
.nav { background: white; padding: 15px; border-radius: 8px; margin-bottom: 20px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
.nav a { color: #2c3e50; text-decoration: none; margin-right: 20px; padding: 8px 12px; border-radius: 4px; }
.nav a:hover { background: #ecf0f1; }
.nav a.active { background: #3498db; color: white; }
.content { background: white; padding: 30px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
.nav { background: #16213e; padding: 15px; border-radius: 8px; margin-bottom: 20px; box-shadow: 0 2px 4px rgba(0,0,0,0.3); }
.nav a { color: #a0a0a0; text-decoration: none; margin-right: 20px; padding: 8px 12px; border-radius: 4px; }
.nav a:hover { background: #0f3460; color: #e0e0e0; }
.nav a.active { background: #e94560; color: white; }
.content { background: #16213e; padding: 30px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.3); }
.form-group { margin-bottom: 20px; }
.form-group label { display: block; margin-bottom: 5px; font-weight: 500; color: #2c3e50; }
.form-group input, .form-group textarea, .form-group select { width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 4px; font-size: 14px; }
.form-group label { display: block; margin-bottom: 5px; font-weight: 500; color: #e0e0e0; }
.form-group input, .form-group textarea, .form-group select { width: 100%; padding: 10px; border: 1px solid #0f3460; border-radius: 4px; font-size: 14px; background: #1a1a2e; color: #e0e0e0; }
.form-group textarea { min-height: 200px; font-family: 'Courier New', monospace; }
.btn { padding: 10px 20px; background: #3498db; color: white; border: none; border-radius: 4px; cursor: pointer; font-size: 14px; text-decoration: none; display: inline-block; margin-left: 10px; }
.btn:hover { background: #2980b9; }
.btn-secondary { background: #95a5a6; }
.btn-secondary:hover { background: #7f8c8d; }
.btn { padding: 10px 20px; background: #e94560; color: white; border: none; border-radius: 4px; cursor: pointer; font-size: 14px; text-decoration: none; display: inline-block; margin-left: 10px; }
.btn:hover { background: #c73e54; }
.btn-secondary { background: #0f3460; }
.btn-secondary:hover { background: #1a4a7a; }
.btn-danger { background: #e74c3c; }
.btn-danger:hover { background: #c0392b; }
.btn-warning { background: #f39c12; }
.btn-warning:hover { background: #e67e22; }
.alert { padding: 15px; border-radius: 4px; margin-bottom: 20px; }
.alert-success { background: #d4edda; color: #155724; border: 1px solid #c3e6cb; }
.alert-error { background: #f8d7da; color: #721c24; border: 1px solid #f5c6cb; }
.alert-info { background: #d1ecf1; color: #0c5460; border: 1px solid #bee5eb; }
.alert-success { background: #1a3a2e; color: #4ade80; border: 1px solid #22c55e; }
.alert-error { background: #3a1a1a; color: #f87171; border: 1px solid #ef4444; }
.alert-info { background: #1a2a3a; color: #60a5fa; border: 1px solid #3b82f6; }
table { width: 100%; border-collapse: collapse; margin-top: 20px; }
th, td { padding: 12px; text-align: left; border-bottom: 1px solid #ddd; }
th { background: #f8f9fa; font-weight: 600; }
.code { background: #f8f9fa; padding: 15px; border-radius: 4px; font-family: 'Courier New', monospace; font-size: 13px; overflow-x: auto; }
th, td { padding: 12px; text-align: left; border-bottom: 1px solid #0f3460; color: #e0e0e0; }
th { background: #0f3460; font-weight: 600; color: #e0e0e0; }
.code { background: #0f3460; padding: 15px; border-radius: 4px; font-family: 'Courier New', monospace; font-size: 13px; overflow-x: auto; color: #e0e0e0; }
</style>
<script>
/*
......@@ -72,7 +72,7 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
function restartServer() {
if (confirm('Are you sure you want to restart the server? This will disconnect all active connections.')) {
fetch('/dashboard/restart', {
fetch('{{ url_for(request, "/dashboard/restart") }}', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
})
......@@ -95,7 +95,7 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
{% if session.logged_in %}
<div class="header-actions">
<button onclick="restartServer()" class="btn btn-warning">Restart Server</button>
<a href="/dashboard/logout" class="btn btn-secondary">Logout</a>
<a href="{{ url_for(request, '/dashboard/logout') }}" class="btn btn-secondary">Logout</a>
</div>
{% endif %}
</div>
......@@ -104,12 +104,15 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
{% if session.logged_in %}
<div class="container">
<div class="nav">
<a href="/dashboard" {% if request.path == '/dashboard' %}class="active"{% endif %}>Overview</a>
<a href="/dashboard/providers" {% if '/providers' in request.path %}class="active"{% endif %}>Providers</a>
<a href="/dashboard/rotations" {% if '/rotations' in request.path %}class="active"{% endif %}>Rotations</a>
<a href="/dashboard/autoselect" {% if '/autoselect' in request.path %}class="active"{% endif %}>Autoselect</a>
<a href="/dashboard/condensation" {% if '/condensation' in request.path %}class="active"{% endif %}>Condensation</a>
<a href="/dashboard/settings" {% if '/settings' in request.path %}class="active"{% endif %}>Settings</a>
<a href="{{ url_for(request, '/dashboard') }}" {% if request.path == '/dashboard' %}class="active"{% endif %}>Overview</a>
<a href="{{ url_for(request, '/dashboard/providers') }}" {% if '/providers' in request.path %}class="active"{% endif %}>Providers</a>
<a href="{{ url_for(request, '/dashboard/rotations') }}" {% if '/rotations' in request.path %}class="active"{% endif %}>Rotations</a>
<a href="{{ url_for(request, '/dashboard/autoselect') }}" {% if '/autoselect' in request.path %}class="active"{% endif %}>Autoselect</a>
<a href="{{ url_for(request, '/dashboard/prompts') }}" {% if '/prompts' in request.path %}class="active"{% endif %}>Prompts</a>
<a href="{{ url_for(request, '/dashboard/settings') }}" {% if '/settings' in request.path %}class="active"{% endif %}>Settings</a>
<a href="{{ url_for(request, '/dashboard/docs') }}" {% if '/docs' in request.path %}class="active"{% endif %}>Docs</a>
<a href="{{ url_for(request, '/dashboard/about') }}" {% if '/about' in request.path %}class="active"{% endif %}>About</a>
<a href="{{ url_for(request, '/dashboard/license') }}" {% if '/license' in request.path %}class="active"{% endif %}>License</a>
</div>
</div>
{% endif %}
......
templates/dashboard/autoselect.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}Autoselect - AISBF Dashboard{% endblock %}
{% block content %}
<h2 style="margin-bottom: 30px;">Autoselect Configuration</h2>
{% if success %}
<div class="alert alert-success">{{ success }}</div>
{% endif %}
{% if error %}
<div class="alert alert-error">{{ error }}</div>
{% endif %}
<div id="autoselect-list" style="margin-bottom: 20px;">
<!-- Autoselect list will be rendered here -->
</div>
<button type="button" class="btn" onclick="addAutoselect()" style="margin-top: 20px;">Add Autoselect</button>
<div style="display: flex; gap: 10px; margin-top: 20px;">
<button type="button" class="btn" onclick="saveAutoselect()">Save Configuration</button>
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
<script>
let autoselectConfig = {{ autoselect_json | safe }};
let availableRotations = {{ available_rotations | safe }};
let availableModels = {{ available_models | safe }};
let expandedAutoselects = new Set();
function renderAutoselectList() {
const container = document.getElementById('autoselect-list');
container.innerHTML = '';
if (!autoselectConfig || Object.keys(autoselectConfig).length === 0) {
container.innerHTML = '<p style="color: #a0a0a0;">No autoselect configurations defined</p>';
return;
}
Object.entries(autoselectConfig).forEach(([key, autoselect]) => {
const autoselectItem = document.createElement('div');
autoselectItem.className = 'autoselect-item';
autoselectItem.style.cssText = 'border: 1px solid #0f3460; margin-bottom: 10px; border-radius: 5px; background: #1a1a2e;';
const isExpanded = expandedAutoselects.has(key);
const modelCount = autoselect.available_models ? autoselect.available_models.length : 0;
autoselectItem.innerHTML = `
<div class="autoselect-header" onclick="toggleAutoselect('${key}')" style="padding: 15px; cursor: pointer; display: flex; justify-content: space-between; align-items: center; user-select: none;">
<div style="display: flex; align-items: center; gap: 10px;">
<span style="font-size: 18px;">${isExpanded ? '▼' : '▶'}</span>
<strong style="font-size: 16px;">${autoselect.model_name || key}</strong>
<span style="color: #a0a0a0; font-size: 14px;">(${modelCount} available model${modelCount !== 1 ? 's' : ''})</span>
</div>
<button type="button" class="btn btn-secondary" onclick="event.stopPropagation(); removeAutoselect('${key}')" style="background: #dc3545; padding: 5px 15px;">Remove</button>
</div>
<div id="autoselect-details-${key}" style="display: ${isExpanded ? 'block' : 'none'}; padding: 20px; border-top: 1px solid #0f3460; background: #16213e;">
<!-- Details will be rendered here -->
</div>
`;
container.appendChild(autoselectItem);
if (isExpanded) {
renderAutoselectDetails(key);
}
});
}
function toggleAutoselect(key) {
if (expandedAutoselects.has(key)) {
expandedAutoselects.delete(key);
} else {
expandedAutoselects.add(key);
}
renderAutoselectList();
}
function renderAutoselectDetails(autoselectKey) {
const container = document.getElementById(`autoselect-details-${autoselectKey}`);
const autoselect = autoselectConfig[autoselectKey];
// Default to "internal" if selection_model is not set
const selectionValue = autoselect.selection_model || 'internal';
// Build selection model options: internal + rotations + provider models
let selectionOptions = `<option value="internal" ${selectionValue === 'internal' ? 'selected' : ''}>internal (Use configured internal model)</option>`;
selectionOptions += '<optgroup label="Rotations">';
selectionOptions += availableRotations.map(r =>
`<option value="${r}" ${selectionValue === r ? 'selected' : ''}>${r}</option>`
).join('');
selectionOptions += '</optgroup>';
selectionOptions += '<optgroup label="Provider Models">';
selectionOptions += availableModels.map(m =>
`<option value="${m.id}" ${selectionValue === m.id ? 'selected' : ''}>${m.name}</option>`
).join('');
selectionOptions += '</optgroup>';
// Build fallback options: rotations + provider models
let fallbackOptions = '<optgroup label="Rotations">';
fallbackOptions += availableRotations.map(r =>
`<option value="${r}" ${autoselect.fallback === r ? 'selected' : ''}>${r}</option>`
).join('');
fallbackOptions += '</optgroup>';
fallbackOptions += '<optgroup label="Provider Models">';
fallbackOptions += availableModels.map(m =>
`<option value="${m.id}" ${autoselect.fallback === m.id ? 'selected' : ''}>${m.name}</option>`
).join('');
fallbackOptions += '</optgroup>';
container.innerHTML = `
<div class="form-group">
<label>Model Name</label>
<input type="text" value="${autoselect.model_name}" onchange="updateAutoselect('${autoselectKey}', 'model_name', this.value)" required>
</div>
<div class="form-group">
<label>Description</label>
<textarea onchange="updateAutoselect('${autoselectKey}', 'description', this.value)" style="min-height: 60px;">${autoselect.description || ''}</textarea>
</div>
<div class="form-group">
<label>Selection Model (Model to use for analysis)</label>
<select onchange="updateAutoselect('${autoselectKey}', 'selection_model', this.value)" required>
<option value="">Select model...</option>
${selectionOptions}
</select>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Choose "internal" to use the configured internal model, or select a rotation/provider model</small>
</div>
<div class="form-group">
<label>Fallback Model (Default if selection fails)</label>
<select onchange="updateAutoselect('${autoselectKey}', 'fallback', this.value)" required>
<option value="">Select model...</option>
${fallbackOptions}
</select>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Choose from rotations or provider models</small>
</div>
<h4 style="margin-top: 20px; margin-bottom: 10px;">Available Models</h4>
<p style="color: #a0a0a0; font-size: 14px; margin-bottom: 10px;">Define which models can be selected and their descriptions for AI analysis</p>
<div id="models-${autoselectKey}"></div>
<button type="button" class="btn btn-secondary" onclick="addAutoselectModel('${autoselectKey}')" style="margin-top: 10px;">Add Model</button>
`;
renderAutoselectModels(autoselectKey);
}
function renderAutoselectModels(autoselectKey) {
const container = document.getElementById(`models-${autoselectKey}`);
const autoselect = autoselectConfig[autoselectKey];
if (!autoselect.available_models || autoselect.available_models.length === 0) {
container.innerHTML = '<p style="color: #a0a0a0;">No models configured</p>';
return;
}
container.innerHTML = '';
autoselect.available_models.forEach((model, index) => {
const modelDiv = document.createElement('div');
modelDiv.style.cssText = 'border: 1px solid #0f3460; padding: 15px; margin-bottom: 10px; border-radius: 3px; background: #1a1a2e;';
const modelOptions = availableModels.map(m =>
`<option value="${m.id}" ${model.model_id === m.id ? 'selected' : ''}>${m.name}</option>`
).join('');
modelDiv.innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<strong>Model ${index + 1}</strong>
<button type="button" class="btn btn-secondary" onclick="removeAutoselectModel('${autoselectKey}', ${index})" style="background: #dc3545; padding: 5px 10px; font-size: 12px;">Remove</button>
</div>
<div class="form-group">
<label>Model ID (Rotation or Provider Model)</label>
<select onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'model_id', this.value)" required>
<option value="">Select model...</option>
${modelOptions}
</select>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Choose from rotations or provider models</small>
</div>
<div class="form-group">
<label>Description (Used by AI to select appropriate model)</label>
<textarea onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'description', this.value)" style="min-height: 80px;" required>${model.description || ''}</textarea>
<small style="color: #666; font-size: 12px;">Be specific about when this model should be used (e.g., "Best for programming, code generation, debugging")</small>
</div>
`;
container.appendChild(modelDiv);
});
}
function addAutoselect() {
const key = prompt('Enter autoselect key (e.g., "autoselect", "smart-select"):');
if (!key) {
return;
}
if (autoselectConfig[key]) {
alert('Autoselect key already exists');
return;
}
autoselectConfig[key] = {
model_name: key,
description: '',
selection_model: '',
fallback: '',
available_models: []
};
expandedAutoselects.add(key);
renderAutoselectList();
}
function removeAutoselect(key) {
if (confirm(`Remove autoselect "${key}"?`)) {
delete autoselectConfig[key];
expandedAutoselects.delete(key);
renderAutoselectList();
}
}
function updateAutoselect(key, field, value) {
autoselectConfig[key][field] = value;
}
function addAutoselectModel(autoselectKey) {
if (!autoselectConfig[autoselectKey].available_models) {
autoselectConfig[autoselectKey].available_models = [];
}
autoselectConfig[autoselectKey].available_models.push({
model_id: '',
description: ''
});
renderAutoselectModels(autoselectKey);
}
function removeAutoselectModel(autoselectKey, index) {
if (confirm('Remove this model?')) {
autoselectConfig[autoselectKey].available_models.splice(index, 1);
renderAutoselectModels(autoselectKey);
}
}
function updateAutoselectModel(autoselectKey, index, field, value) {
autoselectConfig[autoselectKey].available_models[index][field] = value;
}
async function saveAutoselect() {
try {
const response = await fetch('/dashboard/autoselect', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: 'config=' + encodeURIComponent(JSON.stringify(autoselectConfig, null, 2))
});
if (response.ok) {
window.location.href = '/dashboard/autoselect?success=1';
} else {
alert('Error saving configuration');
}
} catch (error) {
alert('Error: ' + error.message);
}
}
// Initial render
renderAutoselectList();
</script>
<style>
.autoselect-item {
animation: fadeIn 0.3s;
}
.autoselect-header:hover {
background: #0f3460;
}
@keyframes fadeIn {
from { opacity: 0; transform: translateY(-10px); }
to { opacity: 1; transform: translateY(0); }
}
.form-group {
margin-bottom: 15px;
}
.form-group label {
display: block;
margin-bottom: 5px;
font-weight: 500;
}
.form-group input[type="text"],
.form-group textarea,
.form-group select {
width: 100%;
padding: 8px;
border: 1px solid #ddd;
border-radius: 3px;
font-size: 14px;
}
.form-group input[type="checkbox"] {
margin-right: 5px;
}
</style>
{% endblock %}
templates/dashboard/docs.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}{{ title }} - AISBF Dashboard{% endblock %}
{% block content %}
<style>
.markdown-content {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
line-height: 1.6;
color: #e0e0e0;
max-width: 100%;
overflow-x: auto;
}
.markdown-content pre {
background: #0f3460;
padding: 16px;
border-radius: 6px;
overflow-x: auto;
border: 1px solid #1a4a7a;
font-family: 'Courier New', Consolas, Monaco, monospace;
font-size: 13px;
line-height: 1.45;
color: #e0e0e0;
}
.markdown-content code {
background: #0f3460;
padding: 2px 6px;
border-radius: 3px;
font-family: 'Courier New', Consolas, Monaco, monospace;
font-size: 13px;
color: #e0e0e0;
}
.markdown-content h1 {
font-size: 2em;
border-bottom: 2px solid #e94560;
padding-bottom: 0.3em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h2 {
font-size: 1.5em;
border-bottom: 1px solid #0f3460;
padding-bottom: 0.3em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h3 {
font-size: 1.25em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h4, .markdown-content h5, .markdown-content h6 {
color: #ffffff;
}
.markdown-content ul, .markdown-content ol {
padding-left: 2em;
margin-bottom: 16px;
}
.markdown-content li {
margin-bottom: 4px;
}
.markdown-content p {
margin-bottom: 16px;
}
.markdown-content a {
color: #e94560;
text-decoration: none;
}
.markdown-content a:hover {
text-decoration: underline;
}
.markdown-content blockquote {
padding: 0 1em;
color: #a0a0a0;
border-left: 4px solid #e94560;
margin-bottom: 16px;
}
.markdown-content table {
border-collapse: collapse;
width: 100%;
margin-bottom: 16px;
}
.markdown-content table th,
.markdown-content table td {
padding: 6px 13px;
border: 1px solid #0f3460;
}
.markdown-content table th {
background: #0f3460;
font-weight: 600;
}
.markdown-content table tr:nth-child(2n) {
background: #0f346055;
}
.markdown-content hr {
border: none;
border-top: 1px solid #0f3460;
margin: 24px 0;
}
.markdown-content img {
max-width: 100%;
border-radius: 4px;
}
/* Inline code that isn't in a pre block */
.markdown-content p > code,
.markdown-content li > code {
background: #0f3460;
padding: 2px 6px;
border-radius: 3px;
color: #e94560;
}
</style>
<h2>{{ title }}</h2>
<div class="markdown-content">
{{ content|safe }}
</div>
{% endblock %}
templates/dashboard/index.html
View file @ a3506741
......@@ -61,9 +61,11 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
</table>
<h3 style="margin-top: 30px; margin-bottom: 15px;">Quick Actions</h3>
<div style="display: flex; gap: 10px;">
<div style="display: flex; gap: 10px; flex-wrap: wrap;">
<a href="/dashboard/providers" class="btn">Manage Providers</a>
<a href="/dashboard/rotations" class="btn">Manage Rotations</a>
<a href="/dashboard/autoselect" class="btn">Manage Autoselect</a>
<a href="/dashboard/prompts" class="btn">Manage Prompts</a>
<a href="/dashboard/settings" class="btn btn-secondary">Server Settings</a>
</div>
{% endblock %}
templates/dashboard/prompts.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}Prompts - AISBF Dashboard{% endblock %}
{% block content %}
<h2 style="margin-bottom: 30px;">Prompts Configuration</h2>
{% if success %}
<div class="alert alert-success">{{ success }}</div>
{% endif %}
{% if error %}
<div class="alert alert-error">{{ error }}</div>
{% endif %}
<div style="margin-bottom: 20px;">
<label style="font-weight: 500; margin-bottom: 10px; display: block;">Select Prompt File:</label>
<select id="prompt-selector" onchange="switchPrompt()" style="width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 3px; font-size: 14px;">
{% for prompt in prompts %}
<option value="{{ prompt.key }}" {% if loop.first %}selected{% endif %}>{{ prompt.name }}</option>
{% endfor %}
</select>
</div>
<form method="POST" id="prompt-form">
<input type="hidden" name="prompt_key" id="prompt_key" value="">
<div class="form-group">
<label for="prompt_content" style="font-weight: 500; margin-bottom: 10px; display: block;">Prompt Content:</label>
<textarea id="prompt_content" name="prompt_content" style="width: 100%; min-height: 400px; padding: 10px; border: 1px solid #ddd; border-radius: 3px; font-family: monospace; font-size: 13px; line-height: 1.5;"></textarea>
<small style="color: #666; display: block; margin-top: 5px;">Edit the prompt template. Use markdown formatting as needed.</small>
</div>
<div style="display: flex; gap: 10px; margin-top: 20px;">
<button type="submit" class="btn">Save Prompt</button>
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
</form>
<script>
const prompts = {{ prompts_data | safe }};
let currentPromptKey = '';
function switchPrompt() {
const selector = document.getElementById('prompt-selector');
const key = selector.value;
currentPromptKey = key;
const prompt = prompts.find(p => p.key === key);
if (prompt) {
document.getElementById('prompt_content').value = prompt.content;
document.getElementById('prompt_key').value = key;
}
}
// Initialize with first prompt
if (prompts.length > 0) {
currentPromptKey = prompts[0].key;
document.getElementById('prompt_content').value = prompts[0].content;
document.getElementById('prompt_key').value = prompts[0].key;
}
</script>
<style>
.form-group {
margin-bottom: 20px;
}
textarea {
resize: vertical;
}
.btn {
padding: 10px 20px;
border: none;
border-radius: 3px;
cursor: pointer;
font-size: 14px;
text-decoration: none;
display: inline-block;
}
.btn-secondary {
background: #6c757d;
color: white;
}
.btn-secondary:hover {
background: #5a6268;
}
</style>
{% endblock %}
templates/dashboard/providers.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}Providers - AISBF Dashboard{% endblock %}
{% block content %}
<h2 style="margin-bottom: 30px;">Providers Configuration</h2>
{% if success %}
<div class="alert alert-success">{{ success }}</div>
{% endif %}
{% if error %}
<div class="alert alert-error">{{ error }}</div>
{% endif %}
<!-- New Provider Form (hidden by default) -->
<div id="new-provider-form" style="display: none; background: #16213e; border: 2px solid #4a9eff; border-radius: 8px; padding: 20px; margin-bottom: 20px;">
<h3 style="margin: 0 0 20px 0; color: #4a9eff;">Add New Provider</h3>
<div class="form-group">
<label>Provider Key (unique identifier, e.g., "gemini", "openai", "kiro")</label>
<input type="text" id="new-provider-key" placeholder="Enter provider key" style="width: 100%;">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">This will be used as the provider ID in the configuration and API endpoints</small>
</div>
<div class="form-group">
<label>Provider Type</label>
<select id="new-provider-type" onchange="updateNewProviderDefaults()">
<option value="openai">OpenAI</option>
<option value="google">Google</option>
<option value="anthropic">Anthropic</option>
<option value="ollama">Ollama</option>
<option value="kiro">Kiro (Amazon Q Developer)</option>
</select>
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Select the type of provider to configure appropriate settings</small>
</div>
<div id="new-provider-type-info" style="background: #0f2840; padding: 15px; border-radius: 5px; margin-bottom: 15px; border-left: 3px solid #4a9eff;">
<p style="margin: 0; color: #a0a0a0;" id="new-provider-type-description">
OpenAI-compatible provider. Uses standard API key authentication.
</p>
</div>
<div style="display: flex; gap: 10px;">
<button type="button" class="btn" onclick="confirmAddProvider()">Create Provider</button>
<button type="button" class="btn btn-secondary" onclick="cancelAddProvider()">Cancel</button>
</div>
</div>
<div id="providers-list" style="margin-bottom: 20px;">
<!-- Provider list will be rendered here -->
</div>
<button type="button" class="btn" id="add-provider-btn" onclick="showAddProviderForm()" style="margin-top: 20px;">Add Provider</button>
<div style="display: flex; gap: 10px; margin-top: 20px;">
<button type="button" class="btn" onclick="saveProviders()">Save Configuration</button>
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
<script>
let providersData = {{ providers_json | safe }};
let expandedProviders = new Set();
function renderProvidersList() {
const container = document.getElementById('providers-list');
container.innerHTML = '';
Object.entries(providersData).forEach(([key, provider]) => {
const providerItem = document.createElement('div');
providerItem.className = 'provider-item';
providerItem.style.cssText = 'border: 1px solid #0f3460; margin-bottom: 10px; border-radius: 5px; background: #1a1a2e;';
const isExpanded = expandedProviders.has(key);
providerItem.innerHTML = `
<div class="provider-header" onclick="toggleProvider('${key}')" style="padding: 15px; cursor: pointer; display: flex; justify-content: space-between; align-items: center; user-select: none;">
<div style="display: flex; align-items: center; gap: 10px;">
<span style="font-size: 18px;">${isExpanded ? '▼' : '▶'}</span>
<strong style="font-size: 16px;">${key}</strong>
<span style="color: #a0a0a0; font-size: 14px;">(${provider.name || key})</span>
</div>
<button type="button" class="btn btn-secondary" onclick="event.stopPropagation(); removeProvider('${key}')" style="background: #dc3545; padding: 5px 15px;">Remove</button>
</div>
<div id="provider-details-${key}" style="display: ${isExpanded ? 'block' : 'none'}; padding: 20px; border-top: 1px solid #0f3460; background: #16213e;">
<!-- Details will be rendered here -->
</div>
`;
container.appendChild(providerItem);
if (isExpanded) {
renderProviderDetails(key);
}
});
}
function toggleProvider(key) {
if (expandedProviders.has(key)) {
expandedProviders.delete(key);
} else {
expandedProviders.add(key);
}
renderProvidersList();
}
function renderProviderDetails(key) {
const container = document.getElementById(`provider-details-${key}`);
const provider = providersData[key];
const isKiroProvider = provider.type === 'kiro';
// Initialize kiro_config if this is a kiro provider and doesn't have it
if (isKiroProvider && !provider.kiro_config) {
provider.kiro_config = {
region: 'us-east-1',
creds_file: '',
sqlite_db: '',
refresh_token: '',
profile_arn: '',
client_id: '',
client_secret: ''
};
}
const kiroConfig = provider.kiro_config || {};
// Build authentication fields based on provider type
let authFieldsHtml = '';
if (isKiroProvider) {
// Kiro-specific authentication fields
authFieldsHtml = `
<div style="background: #0f2840; padding: 15px; border-radius: 5px; margin-bottom: 15px; border-left: 3px solid #4a9eff;">
<h4 style="margin: 0 0 15px 0; color: #4a9eff;">Kiro Authentication</h4>
<small style="color: #a0a0a0; display: block; margin-bottom: 15px;">
Choose one authentication method: Kiro IDE credentials (creds_file), kiro-cli database (sqlite_db), or direct credentials (refresh_token + client_id/secret).
</small>
<div class="form-group">
<label>AWS Region</label>
<input type="text" value="${kiroConfig.region || 'us-east-1'}" onchange="updateKiroConfig('${key}', 'region', this.value)" placeholder="us-east-1">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">AWS region for Kiro API (default: us-east-1)</small>
</div>
<h5 style="margin: 20px 0 10px 0; color: #8ec8ff;">Option 1: Kiro IDE Credentials</h5>
<div class="form-group">
<label>Credentials File Path</label>
<input type="text" value="${kiroConfig.creds_file || ''}" onchange="updateKiroConfig('${key}', 'creds_file', this.value)" placeholder="~/.config/Code/User/globalStorage/amazon.q/credentials.json">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Path to Kiro IDE credentials JSON file</small>
</div>
<h5 style="margin: 20px 0 10px 0; color: #8ec8ff;">Option 2: kiro-cli Database</h5>
<div class="form-group">
<label>SQLite Database Path</label>
<input type="text" value="${kiroConfig.sqlite_db || ''}" onchange="updateKiroConfig('${key}', 'sqlite_db', this.value)" placeholder="~/.local/share/kiro-cli/data.sqlite3">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Path to kiro-cli SQLite database</small>
</div>
<h5 style="margin: 20px 0 10px 0; color: #8ec8ff;">Option 3: Direct Credentials</h5>
<div class="form-group">
<label>Refresh Token</label>
<input type="password" value="${kiroConfig.refresh_token || ''}" onchange="updateKiroConfig('${key}', 'refresh_token', this.value)" placeholder="Enter refresh token">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Kiro refresh token for direct authentication</small>
</div>
<div class="form-group">
<label>Profile ARN</label>
<input type="text" value="${kiroConfig.profile_arn || ''}" onchange="updateKiroConfig('${key}', 'profile_arn', this.value)" placeholder="arn:aws:codewhisperer:us-east-1:...">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">AWS CodeWhisperer profile ARN (optional)</small>
</div>
<div class="form-group">
<label>Client ID (for AWS SSO OIDC)</label>
<input type="text" value="${kiroConfig.client_id || ''}" onchange="updateKiroConfig('${key}', 'client_id', this.value)" placeholder="OAuth client ID">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">OAuth client ID for AWS SSO OIDC authentication</small>
</div>
<div class="form-group">
<label>Client Secret (for AWS SSO OIDC)</label>
<input type="password" value="${kiroConfig.client_secret || ''}" onchange="updateKiroConfig('${key}', 'client_secret', this.value)" placeholder="OAuth client secret">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">OAuth client secret for AWS SSO OIDC authentication</small>
</div>
</div>
`;
} else {
// Standard API key authentication fields
authFieldsHtml = `
<div class="form-group">
<label>
<input type="checkbox" ${provider.api_key_required ? 'checked' : ''} onchange="updateProvider('${key}', 'api_key_required', this.checked)">
API Key Required
</label>
</div>
<div class="form-group">
<label>API Key</label>
<input type="password" value="${provider.api_key || ''}" onchange="updateProvider('${key}', 'api_key', this.value)" placeholder="Enter API key">
</div>
`;
}
container.innerHTML = `
<div class="form-group">
<label>Provider ID</label>
<input type="text" value="${provider.id}" onchange="updateProvider('${key}', 'id', this.value)" required>
</div>
<div class="form-group">
<label>Name</label>
<input type="text" value="${provider.name}" onchange="updateProvider('${key}', 'name', this.value)" required>
</div>
<div class="form-group">
<label>Endpoint</label>
<input type="text" value="${provider.endpoint}" onchange="updateProvider('${key}', 'endpoint', this.value)" required>
${isKiroProvider ? '<small style="color: #a0a0a0; display: block; margin-top: 5px;">Typically: https://q.us-east-1.amazonaws.com</small>' : ''}
</div>
<div class="form-group">
<label>Type</label>
<select onchange="updateProviderType('${key}', this.value)" required>
<option value="google" ${provider.type === 'google' ? 'selected' : ''}>Google</option>
<option value="openai" ${provider.type === 'openai' ? 'selected' : ''}>OpenAI</option>
<option value="anthropic" ${provider.type === 'anthropic' ? 'selected' : ''}>Anthropic</option>
<option value="ollama" ${provider.type === 'ollama' ? 'selected' : ''}>Ollama</option>
<option value="kiro" ${provider.type === 'kiro' ? 'selected' : ''}>Kiro (Amazon Q Developer)</option>
</select>
</div>
${authFieldsHtml}
<div class="form-group">
<label>Rate Limit (seconds)</label>
<input type="number" value="${provider.rate_limit || 0}" onchange="updateProvider('${key}', 'rate_limit', parseFloat(this.value))" step="0.1">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Time delay between requests to this provider</small>
</div>
<div class="form-group">
<label>Default Rate Limit TPM (Tokens Per Minute)</label>
<input type="number" value="${provider.default_rate_limit_TPM || ''}" onchange="updateProvider('${key}', 'default_rate_limit_TPM', this.value ? parseInt(this.value) : null)" placeholder="Optional">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Default token limit per minute for models in this provider</small>
</div>
<div class="form-group">
<label>Default Rate Limit TPH (Tokens Per Hour)</label>
<input type="number" value="${provider.default_rate_limit_TPH || ''}" onchange="updateProvider('${key}', 'default_rate_limit_TPH', this.value ? parseInt(this.value) : null)" placeholder="Optional">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Default token limit per hour for models in this provider</small>
</div>
<div class="form-group">
<label>Default Rate Limit TPD (Tokens Per Day)</label>
<input type="number" value="${provider.default_rate_limit_TPD || ''}" onchange="updateProvider('${key}', 'default_rate_limit_TPD', this.value ? parseInt(this.value) : null)" placeholder="Optional">
<small style="color: #a0a0a0; display: block; margin-top: 5px;">Default token limit per day for models in this provider</small>
</div>
<div class="form-group">
<label>Default Max Request Tokens</label>
<input type="number" value="${provider.default_max_request_tokens || ''}" onchange="updateProvider('${key}', 'default_max_request_tokens', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Default Context Size</label>
<input type="number" value="${provider.default_context_size || ''}" onchange="updateProvider('${key}', 'default_context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Default Condense Context (%)</label>
<input type="number" value="${provider.default_condense_context || ''}" onchange="updateProvider('${key}', 'default_condense_context', this.value ? parseInt(this.value) : null)" placeholder="Optional (e.g., 80)">
</div>
<div class="form-group">
<label>Default Condense Method (conversational, semantic, hierarchical, algorithmic)</label>
<input type="text" value="${Array.isArray(provider.default_condense_method) ? provider.default_condense_method.join(', ') : (provider.default_condense_method || '')}" onchange="updateProviderCondenseMethod('${key}', this.value)" placeholder="e.g., semantic, conversational">
</div>
<h4 style="margin-top: 20px; margin-bottom: 10px;">Models</h4>
<div id="models-${key}"></div>
<button type="button" class="btn btn-secondary" onclick="addModel('${key}')" style="margin-top: 10px;">Add Model</button>
`;
renderModels(key);
}
function renderModels(providerKey) {
const container = document.getElementById(`models-${providerKey}`);
const provider = providersData[providerKey];
if (!provider.models || provider.models.length === 0) {
container.innerHTML = '<p style="color: #a0a0a0;">No models configured</p>';
return;
}
container.innerHTML = '';
provider.models.forEach((model, index) => {
const modelDiv = document.createElement('div');
modelDiv.style.cssText = 'border: 1px solid #0f3460; padding: 15px; margin-bottom: 10px; border-radius: 3px; background: #1a1a2e;';
modelDiv.innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<strong>Model ${index + 1}</strong>
<button type="button" class="btn btn-secondary" onclick="removeModel('${providerKey}', ${index})" style="background: #dc3545; padding: 5px 10px; font-size: 12px;">Remove</button>
</div>
<div class="form-group">
<label>Model Name</label>
<input type="text" value="${model.name}" onchange="updateModel('${providerKey}', ${index}, 'name', this.value)" required>
</div>
<div class="form-group">
<label>Rate Limit (seconds)</label>
<input type="number" value="${model.rate_limit || 0}" onchange="updateModel('${providerKey}', ${index}, 'rate_limit', parseFloat(this.value))" step="0.1">
</div>
<div class="form-group">
<label>Max Request Tokens</label>
<input type="number" value="${model.max_request_tokens || ''}" onchange="updateModel('${providerKey}', ${index}, 'max_request_tokens', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Context Size</label>
<input type="number" value="${model.context_size || ''}" onchange="updateModel('${providerKey}', ${index}, 'context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Rate Limit TPM (Tokens Per Minute)</label>
<input type="number" value="${model.rate_limit_TPM || ''}" onchange="updateModel('${providerKey}', ${index}, 'rate_limit_TPM', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Rate Limit TPH (Tokens Per Hour)</label>
<input type="number" value="${model.rate_limit_TPH || ''}" onchange="updateModel('${providerKey}', ${index}, 'rate_limit_TPH', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Rate Limit TPD (Tokens Per Day)</label>
<input type="number" value="${model.rate_limit_TPD || ''}" onchange="updateModel('${providerKey}', ${index}, 'rate_limit_TPD', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<div class="form-group">
<label>Condense Context (%)</label>
<input type="number" value="${model.condense_context || ''}" onchange="updateModel('${providerKey}', ${index}, 'condense_context', this.value ? parseInt(this.value) : null)" placeholder="Optional (default: 80)">
</div>
<div class="form-group">
<label>Condense Method (conversational, semantic, hierarchical, algorithmic)</label>
<input type="text" value="${Array.isArray(model.condense_method) ? model.condense_method.join(', ') : (model.condense_method || '')}" onchange="updateModelCondenseMethod('${providerKey}', ${index}, this.value)" placeholder="e.g., semantic, conversational">
</div>
`;
container.appendChild(modelDiv);
});
}
function showAddProviderForm() {
document.getElementById('new-provider-form').style.display = 'block';
document.getElementById('add-provider-btn').style.display = 'none';
document.getElementById('new-provider-key').value = '';
document.getElementById('new-provider-type').value = 'openai';
updateNewProviderDefaults();
document.getElementById('new-provider-key').focus();
}
function cancelAddProvider() {
document.getElementById('new-provider-form').style.display = 'none';
document.getElementById('add-provider-btn').style.display = 'inline-block';
}
function updateNewProviderDefaults() {
const providerType = document.getElementById('new-provider-type').value;
const descriptionEl = document.getElementById('new-provider-type-description');
const descriptions = {
'openai': 'OpenAI-compatible provider. Uses standard API key authentication. Endpoint: https://api.openai.com/v1',
'google': 'Google AI provider (Gemini). Uses API key authentication. Endpoint: https://generativelanguage.googleapis.com/v1beta',
'anthropic': 'Anthropic provider (Claude). Uses API key authentication. Endpoint: https://api.anthropic.com/v1',
'ollama': 'Ollama local provider. No API key required by default. Endpoint: http://localhost:11434/api',
'kiro': 'Kiro (Amazon Q Developer) provider. Uses Kiro credentials (IDE, CLI, or direct tokens). Endpoint: https://q.us-east-1.amazonaws.com'
};
descriptionEl.textContent = descriptions[providerType] || 'Standard provider configuration.';
}
function confirmAddProvider() {
const key = document.getElementById('new-provider-key').value.trim();
const providerType = document.getElementById('new-provider-type').value;
// Validate key
if (!key) {
showFormError('Please enter a provider key');
document.getElementById('new-provider-key').focus();
return;
}
if (providersData[key]) {
showFormError('A provider with this key already exists');
document.getElementById('new-provider-key').focus();
return;
}
// Validate key format (alphanumeric, dashes, underscores only)
if (!/^[a-zA-Z0-9_-]+$/.test(key)) {
showFormError('Provider key can only contain letters, numbers, dashes, and underscores');
document.getElementById('new-provider-key').focus();
return;
}
// Create base provider config
const newProvider = {
id: key,
name: key,
endpoint: '',
type: providerType,
api_key_required: providerType !== 'kiro' && providerType !== 'ollama',
rate_limit: 0,
default_rate_limit: 0,
models: []
};
// Set type-specific defaults
if (providerType === 'kiro') {
newProvider.endpoint = 'https://q.us-east-1.amazonaws.com';
newProvider.name = key + ' (Amazon Q Developer)';
newProvider.kiro_config = {
region: 'us-east-1',
creds_file: '',
sqlite_db: '',
refresh_token: '',
profile_arn: '',
client_id: '',
client_secret: ''
};
} else if (providerType === 'openai') {
newProvider.endpoint = 'https://api.openai.com/v1';
newProvider.api_key = '';
} else if (providerType === 'google') {
newProvider.endpoint = 'https://generativelanguage.googleapis.com/v1beta';
newProvider.api_key = '';
} else if (providerType === 'anthropic') {
newProvider.endpoint = 'https://api.anthropic.com/v1';
newProvider.api_key = '';
} else if (providerType === 'ollama') {
newProvider.endpoint = 'http://localhost:11434/api';
newProvider.api_key_required = false;
newProvider.api_key = '';
} else {
newProvider.api_key = '';
}
providersData[key] = newProvider;
expandedProviders.add(key);
// Hide the form and show the button
cancelAddProvider();
renderProvidersList();
// Scroll to the new provider
setTimeout(() => {
const providerElement = document.querySelector(`[onclick*="toggleProvider('${key}')"]`);
if (providerElement) {
providerElement.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
}, 100);
}
function showFormError(message) {
// Check if error element already exists
let errorEl = document.getElementById('new-provider-error');
if (!errorEl) {
errorEl = document.createElement('div');
errorEl.id = 'new-provider-error';
errorEl.className = 'alert alert-error';
errorEl.style.marginBottom = '15px';
const form = document.getElementById('new-provider-form');
form.insertBefore(errorEl, form.querySelector('.form-group'));
}
errorEl.textContent = message;
errorEl.style.display = 'block';
// Auto-hide after 5 seconds
setTimeout(() => {
if (errorEl) {
errorEl.style.display = 'none';
}
}, 5000);
}
// Legacy function - kept for compatibility but redirects to new form
function addProvider() {
showAddProviderForm();
}
function removeProvider(key) {
if (confirm(`Remove provider "${key}"?`)) {
delete providersData[key];
expandedProviders.delete(key);
renderProvidersList();
}
}
function updateProvider(key, field, value) {
providersData[key][field] = value;
}
function updateProviderType(key, newType) {
const oldType = providersData[key].type;
providersData[key].type = newType;
// Handle transition to/from kiro type
if (newType === 'kiro' && oldType !== 'kiro') {
// Transitioning TO kiro: initialize kiro_config, set api_key_required to false
providersData[key].api_key_required = false;
providersData[key].kiro_config = {
region: 'us-east-1',
creds_file: '',
sqlite_db: '',
refresh_token: '',
profile_arn: '',
client_id: '',
client_secret: ''
};
// Set default endpoint for kiro
if (!providersData[key].endpoint || providersData[key].endpoint === '') {
providersData[key].endpoint = 'https://q.us-east-1.amazonaws.com';
}
} else if (newType !== 'kiro' && oldType === 'kiro') {
// Transitioning FROM kiro: remove kiro_config, set api_key_required to true
providersData[key].api_key_required = true;
delete providersData[key].kiro_config;
}
// Re-render to show appropriate fields
renderProviderDetails(key);
}
function updateKiroConfig(key, field, value) {
if (!providersData[key].kiro_config) {
providersData[key].kiro_config = {};
}
providersData[key].kiro_config[field] = value;
}
function addModel(providerKey) {
if (!providersData[providerKey].models) {
providersData[providerKey].models = [];
}
providersData[providerKey].models.push({
name: '',
rate_limit: 0
});
renderModels(providerKey);
}
function removeModel(providerKey, index) {
if (confirm('Remove this model?')) {
providersData[providerKey].models.splice(index, 1);
renderModels(providerKey);
}
}
function updateModel(providerKey, index, field, value) {
providersData[providerKey].models[index][field] = value;
}
function updateModelCondenseMethod(providerKey, index, value) {
const trimmed = value.trim();
if (!trimmed) {
providersData[providerKey].models[index].condense_method = null;
return;
}
// Check if it's a comma-separated list
if (trimmed.includes(',')) {
providersData[providerKey].models[index].condense_method =
trimmed.split(',').map(s => s.trim()).filter(s => s);
} else {
providersData[providerKey].models[index].condense_method = trimmed;
}
}
function updateProviderCondenseMethod(providerKey, value) {
const trimmed = value.trim();
if (!trimmed) {
providersData[providerKey].default_condense_method = null;
return;
}
// Check if it's a comma-separated list
if (trimmed.includes(',')) {
providersData[providerKey].default_condense_method =
trimmed.split(',').map(s => s.trim()).filter(s => s);
} else {
providersData[providerKey].default_condense_method = trimmed;
}
}
async function saveProviders() {
try {
const response = await fetch('/dashboard/providers', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: 'config=' + encodeURIComponent(JSON.stringify(providersData, null, 2))
});
if (response.ok) {
window.location.href = '/dashboard/providers?success=1';
} else {
alert('Error saving configuration');
}
} catch (error) {
alert('Error: ' + error.message);
}
}
// Initial render
renderProvidersList();
</script>
<style>
.provider-item {
animation: fadeIn 0.3s;
}
.provider-header:hover {
background: #0f3460;
}
@keyframes fadeIn {
from { opacity: 0; transform: translateY(-10px); }
to { opacity: 1; transform: translateY(0); }
}
.form-group {
margin-bottom: 15px;
}
.form-group label {
display: block;
margin-bottom: 5px;
font-weight: 500;
color: #e0e0e0;
}
.form-group input[type="text"],
.form-group input[type="password"],
.form-group input[type="number"],
.form-group select {
width: 100%;
padding: 8px;
border: 1px solid #0f3460;
border-radius: 3px;
font-size: 14px;
background: #1a1a2e;
color: #e0e0e0;
}
.form-group input[type="checkbox"] {
margin-right: 5px;
}
</style>
{% endblock %}
templates/dashboard/rotations.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}Rotations - AISBF Dashboard{% endblock %}
{% block content %}
<h2 style="margin-bottom: 30px;">Rotations Configuration</h2>
{% if success %}
<div class="alert alert-success">{{ success }}</div>
{% endif %}
{% if error %}
<div class="alert alert-error">{{ error }}</div>
{% endif %}
<div class="form-group" style="margin-bottom: 20px;">
<label>
<input type="checkbox" id="global-notify" onchange="updateGlobalNotify(this.checked)">
Global Notify Errors
</label>
</div>
<div id="rotations-list" style="margin-bottom: 20px;">
<!-- Rotations list will be rendered here -->
</div>
<button type="button" class="btn" onclick="addRotation()" style="margin-top: 20px;">Add Rotation</button>
<div style="display: flex; gap: 10px; margin-top: 20px;">
<button type="button" class="btn" onclick="saveRotations()">Save Configuration</button>
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
<script>
let rotationsConfig = {{ rotations_json | safe }};
let availableProviders = {{ available_providers | safe }};
let expandedRotations = new Set();
document.getElementById('global-notify').checked = rotationsConfig.notifyerrors || false;
function renderRotationsList() {
const container = document.getElementById('rotations-list');
container.innerHTML = '';
Object.entries(rotationsConfig.rotations || {}).forEach(([key, rotation]) => {
const rotationItem = document.createElement('div');
rotationItem.className = 'rotation-item';
rotationItem.style.cssText = 'border: 1px solid #0f3460; margin-bottom: 10px; border-radius: 5px; background: #1a1a2e;';
const isExpanded = expandedRotations.has(key);
const providerCount = rotation.providers ? rotation.providers.length : 0;
rotationItem.innerHTML = `
<div class="rotation-header" onclick="toggleRotation('${key}')" style="padding: 15px; cursor: pointer; display: flex; justify-content: space-between; align-items: center; user-select: none;">
<div style="display: flex; align-items: center; gap: 10px;">
<span style="font-size: 18px;">${isExpanded ? '▼' : '▶'}</span>
<strong style="font-size: 16px;">${key}</strong>
<span style="color: #a0a0a0; font-size: 14px;">(${providerCount} provider${providerCount !== 1 ? 's' : ''})</span>
</div>
<button type="button" class="btn btn-secondary" onclick="event.stopPropagation(); removeRotation('${key}')" style="background: #dc3545; padding: 5px 15px;">Remove</button>
</div>
<div id="rotation-details-${key}" style="display: ${isExpanded ? 'block' : 'none'}; padding: 20px; border-top: 1px solid #0f3460; background: #16213e;">
<!-- Details will be rendered here -->
</div>
`;
container.appendChild(rotationItem);
if (isExpanded) {
renderRotationDetails(key);
}
});
}
function toggleRotation(key) {
if (expandedRotations.has(key)) {
expandedRotations.delete(key);
} else {
expandedRotations.add(key);
}
renderRotationsList();
}
function renderRotationDetails(rotationKey) {
const container = document.getElementById(`rotation-details-${rotationKey}`);
const rotation = rotationsConfig.rotations[rotationKey];
container.innerHTML = `
<div class="form-group">
<label>Model Name</label>
<input type="text" value="${rotation.model_name}" onchange="updateRotation('${rotationKey}', 'model_name', this.value)" required>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${rotation.notifyerrors ? 'checked' : ''} onchange="updateRotation('${rotationKey}', 'notifyerrors', this.checked)">
Notify Errors
</label>
</div>
<div class="form-group">
<label>Default Rate Limit (seconds)</label>
<input type="number" value="${rotation.default_rate_limit || ''}" onchange="updateRotation('${rotationKey}', 'default_rate_limit', this.value ? parseFloat(this.value) : null)" step="0.1" placeholder="Optional">
</div>
<div class="form-group">
<label>Default Context Size</label>
<input type="number" value="${rotation.default_context_size || ''}" onchange="updateRotation('${rotationKey}', 'default_context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<h4 style="margin-top: 20px; margin-bottom: 10px;">Providers</h4>
<div id="providers-${rotationKey}"></div>
<button type="button" class="btn btn-secondary" onclick="addRotationProvider('${rotationKey}')" style="margin-top: 10px;">Add Provider</button>
`;
renderRotationProviders(rotationKey);
}
function renderRotationProviders(rotationKey) {
const container = document.getElementById(`providers-${rotationKey}`);
const rotation = rotationsConfig.rotations[rotationKey];
if (!rotation.providers || rotation.providers.length === 0) {
container.innerHTML = '<p style="color: #a0a0a0;">No providers configured</p>';
return;
}
container.innerHTML = '';
rotation.providers.forEach((provider, providerIndex) => {
const providerDiv = document.createElement('div');
providerDiv.style.cssText = 'border: 1px solid #0f3460; padding: 15px; margin-bottom: 10px; border-radius: 3px; background: #1a1a2e;';
const providerOptions = availableProviders.map(p =>
`<option value="${p}" ${provider.provider_id === p ? 'selected' : ''}>${p}</option>`
).join('');
providerDiv.innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 10px;">
<strong>Provider ${providerIndex + 1}</strong>
<button type="button" class="btn btn-secondary" onclick="removeRotationProvider('${rotationKey}', ${providerIndex})" style="background: #dc3545; padding: 5px 10px; font-size: 12px;">Remove</button>
</div>
<div class="form-group">
<label>Provider ID</label>
<select onchange="updateRotationProvider('${rotationKey}', ${providerIndex}, 'provider_id', this.value)" required>
<option value="">Select provider...</option>
${providerOptions}
</select>
</div>
<div class="form-group">
<label>Weight (optional, for provider-level weight)</label>
<input type="number" value="${provider.weight || ''}" onchange="updateRotationProvider('${rotationKey}', ${providerIndex}, 'weight', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div>
<h5 style="margin-top: 15px; margin-bottom: 10px;">Models</h5>
<div id="models-${rotationKey}-${providerIndex}"></div>
<button type="button" class="btn btn-secondary" onclick="addRotationModel('${rotationKey}', ${providerIndex})" style="margin-top: 10px; font-size: 12px;">Add Model</button>
<p style="font-size: 12px; color: #a0a0a0; margin-top: 5px;">Leave models empty to use all models from provider config</p>
`;
container.appendChild(providerDiv);
renderRotationModels(rotationKey, providerIndex);
});
}
function renderRotationModels(rotationKey, providerIndex) {
const container = document.getElementById(`models-${rotationKey}-${providerIndex}`);
const provider = rotationsConfig.rotations[rotationKey].providers[providerIndex];
if (!provider.models || provider.models.length === 0) {
container.innerHTML = '<p style="color: #a0a0a0; font-size: 12px;">No models specified (will use all from provider)</p>';
return;
}
container.innerHTML = '';
provider.models.forEach((model, modelIndex) => {
const modelDiv = document.createElement('div');
modelDiv.style.cssText = 'border: 1px solid #0f3460; padding: 10px; margin-bottom: 8px; border-radius: 3px; background: #16213e;';
modelDiv.innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong style="font-size: 13px;">Model ${modelIndex + 1}</strong>
<button type="button" onclick="removeRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex})" style="background: #dc3545; color: white; border: none; padding: 3px 8px; border-radius: 3px; cursor: pointer; font-size: 11px;">Remove</button>
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Model Name</label>
<input type="text" value="${model.name}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'name', this.value)" required style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Weight</label>
<input type="number" value="${model.weight || 1}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'weight', parseInt(this.value))" style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Rate Limit (seconds)</label>
<input type="number" value="${model.rate_limit || 0}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'rate_limit', parseFloat(this.value))" step="0.1" style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Max Request Tokens</label>
<input type="number" value="${model.max_request_tokens || ''}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'max_request_tokens', this.value ? parseInt(this.value) : null)" placeholder="Optional" style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Context Size</label>
<input type="number" value="${model.context_size || ''}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional" style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 8px;">
<label style="font-size: 12px;">Condense Context (%)</label>
<input type="number" value="${model.condense_context || ''}" onchange="updateRotationModel('${rotationKey}', ${providerIndex}, ${modelIndex}, 'condense_context', this.value ? parseInt(this.value) : null)" placeholder="Optional (default: 80)" style="font-size: 12px; padding: 5px;">
</div>
<div class="form-group" style="margin-bottom: 0;">
<label style="font-size: 12px;">Condense Method</label>
<input type="text" value="${Array.isArray(model.condense_method) ? model.condense_method.join(', ') : (model.condense_method || '')}" onchange="updateRotationModelCondenseMethod('${rotationKey}', ${providerIndex}, ${modelIndex}, this.value)" placeholder="e.g., semantic, conversational, hierarchical" style="font-size: 12px; padding: 5px;">
</div>
`;
container.appendChild(modelDiv);
});
}
function updateGlobalNotify(value) {
rotationsConfig.notifyerrors = value;
}
function addRotation() {
const key = prompt('Enter rotation key (e.g., "coding", "general"):');
if (!key || rotationsConfig.rotations[key]) {
alert('Invalid or duplicate rotation key');
return;
}
if (!rotationsConfig.rotations) {
rotationsConfig.rotations = {};
}
rotationsConfig.rotations[key] = {
model_name: key,
notifyerrors: false,
providers: []
};
expandedRotations.add(key);
renderRotationsList();
}
function removeRotation(key) {
if (confirm(`Remove rotation "${key}"?`)) {
delete rotationsConfig.rotations[key];
expandedRotations.delete(key);
renderRotationsList();
}
}
function updateRotation(key, field, value) {
rotationsConfig.rotations[key][field] = value;
}
function addRotationProvider(rotationKey) {
if (!rotationsConfig.rotations[rotationKey].providers) {
rotationsConfig.rotations[rotationKey].providers = [];
}
rotationsConfig.rotations[rotationKey].providers.push({
provider_id: '',
models: []
});
renderRotationProviders(rotationKey);
}
function removeRotationProvider(rotationKey, providerIndex) {
if (confirm('Remove this provider?')) {
rotationsConfig.rotations[rotationKey].providers.splice(providerIndex, 1);
renderRotationProviders(rotationKey);
}
}
function updateRotationProvider(rotationKey, providerIndex, field, value) {
rotationsConfig.rotations[rotationKey].providers[providerIndex][field] = value;
}
function addRotationModel(rotationKey, providerIndex) {
const provider = rotationsConfig.rotations[rotationKey].providers[providerIndex];
if (!provider.models) {
provider.models = [];
}
provider.models.push({
name: '',
weight: 1,
rate_limit: 0
});
renderRotationModels(rotationKey, providerIndex);
}
function removeRotationModel(rotationKey, providerIndex, modelIndex) {
if (confirm('Remove this model?')) {
rotationsConfig.rotations[rotationKey].providers[providerIndex].models.splice(modelIndex, 1);
renderRotationModels(rotationKey, providerIndex);
}
}
function updateRotationModel(rotationKey, providerIndex, modelIndex, field, value) {
rotationsConfig.rotations[rotationKey].providers[providerIndex].models[modelIndex][field] = value;
}
function updateRotationModelCondenseMethod(rotationKey, providerIndex, modelIndex, value) {
const trimmed = value.trim();
if (!trimmed) {
rotationsConfig.rotations[rotationKey].providers[providerIndex].models[modelIndex].condense_method = null;
return;
}
// Check if it's a comma-separated list
if (trimmed.includes(',')) {
rotationsConfig.rotations[rotationKey].providers[providerIndex].models[modelIndex].condense_method =
trimmed.split(',').map(s => s.trim()).filter(s => s);
} else {
rotationsConfig.rotations[rotationKey].providers[providerIndex].models[modelIndex].condense_method = trimmed;
}
}
async function saveRotations() {
try {
const response = await fetch('/dashboard/rotations', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: 'config=' + encodeURIComponent(JSON.stringify(rotationsConfig, null, 2))
});
if (response.ok) {
window.location.href = '/dashboard/rotations?success=1';
} else {
alert('Error saving configuration');
}
} catch (error) {
alert('Error: ' + error.message);
}
}
// Initial render
renderRotationsList();
</script>
<style>
.rotation-item {
animation: fadeIn 0.3s;
}
.rotation-header:hover {
background: #0f3460;
}
@keyframes fadeIn {
from { opacity: 0; transform: translateY(-10px); }
to { opacity: 1; transform: translateY(0); }
}
.form-group {
margin-bottom: 15px;
}
.form-group label {
display: block;
margin-bottom: 5px;
font-weight: 500;
color: #e0e0e0;
}
.form-group input[type="text"],
.form-group input[type="number"],
.form-group select {
width: 100%;
padding: 8px;
border: 1px solid #0f3460;
border-radius: 3px;
font-size: 14px;
background: #1a1a2e;
color: #e0e0e0;
}
.form-group input[type="checkbox"] {
margin-right: 5px;
}
</style>
{% endblock %}
templates/dashboard/settings.html
View file @ a3506741
......@@ -44,12 +44,32 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<div class="form-group">
<label for="protocol">Protocol</label>
<select id="protocol" name="protocol">
<select id="protocol" name="protocol" onchange="toggleSSLFields()">
<option value="http" {% if config.server.protocol == 'http' %}selected{% endif %}>HTTP</option>
<option value="https" {% if config.server.protocol == 'https' %}selected{% endif %}>HTTPS</option>
</select>
</div>
<div id="ssl-fields" style="display: {% if config.server.protocol == 'https' %}block{% else %}none{% endif %};">
<div class="form-group">
<label for="public_domain">Public Domain (for Let's Encrypt)</label>
<input type="text" id="public_domain" name="public_domain" value="{{ config.server.public_domain or '' }}" placeholder="example.com (optional)">
<small style="color: #666; display: block; margin-top: 5px;">If specified, Let's Encrypt will be used to generate valid SSL certificates. Leave empty to use self-signed certificates.</small>
</div>
<div class="form-group">
<label for="ssl_cert">SSL Certificate Path</label>
<input type="text" id="ssl_cert" name="ssl_cert" value="{{ config.server.ssl_cert or '' }}" placeholder="~/.aisbf/cert.pem (default)">
<small style="color: #666; display: block; margin-top: 5px;">Path to SSL certificate file. Leave empty to use default (~/.aisbf/cert.pem). Auto-generated using Let's Encrypt if public domain is set, otherwise self-signed.</small>
</div>
<div class="form-group">
<label for="ssl_key">SSL Key Path</label>
<input type="text" id="ssl_key" name="ssl_key" value="{{ config.server.ssl_key or '' }}" placeholder="~/.aisbf/key.pem (default)">
<small style="color: #666; display: block; margin-top: 5px;">Path to SSL private key file. Leave empty to use default (~/.aisbf/key.pem). Auto-generated with certificate.</small>
</div>
</div>
<h3 style="margin: 30px 0 20px;">Authentication</h3>
<div class="form-group">
......@@ -76,11 +96,18 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<input type="password" id="dashboard_password" name="dashboard_password" placeholder="Leave blank to keep current">
</div>
<h3 style="margin: 30px 0 20px;">Internal Model</h3>
<h3 style="margin: 30px 0 20px;">Internal Models</h3>
<div class="form-group">
<label for="internal_model_id">Model ID</label>
<input type="text" id="internal_model_id" name="internal_model_id" value="{{ config.internal_model.model_id }}" required>
<label for="condensation_model_id">Condensation Model ID</label>
<input type="text" id="condensation_model_id" name="condensation_model_id" value="{{ config.internal_model.condensation_model_id }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Used when condensation model is set to "internal"</small>
</div>
<div class="form-group">
<label for="autoselect_model_id">Autoselect Model ID</label>
<input type="text" id="autoselect_model_id" name="autoselect_model_id" value="{{ config.internal_model.autoselect_model_id }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Used when autoselect selection_model is set to "internal"</small>
</div>
<div style="display: flex; gap: 10px; margin-top: 30px;">
......@@ -88,4 +115,17 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
</form>
<script>
function toggleSSLFields() {
const protocol = document.getElementById('protocol').value;
const sslFields = document.getElementById('ssl-fields');
if (protocol === 'https') {
sslFields.style.display = 'block';
} else {
sslFields.style.display = 'none';
}
}
</script>
{% endblock %}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment