Skip to content

A
aisbf
Commit a3506741 authored by Your Name's avatar Your Name
Browse files
Options

All update up to proxy

parent 36fc3e9f
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 4618 additions and 451 deletions
AI.PROMPT
View file @ a3506741
......@@ -383,7 +383,9 @@ Kiro Gateway is a third-party proxy gateway that provides OpenAI and Anthropic-c
**Configuration:**
In `config/providers.json`:
**IMPORTANT:** Kiro providers use a different configuration structure than other providers. Instead of an `api_key` field, they use a `kiro_config` object with authentication settings specific to Kiro/Amazon Q Developer.
**Standard Provider Configuration (Google, OpenAI, Anthropic, etc.):**
```json
{
"gemini": {
......@@ -405,18 +407,32 @@ In `config/providers.json`:
"context_size": 1000000
}
]
},
}
}
```
**Kiro Provider Configuration (uses kiro_config instead of api_key):**
```json
{
"kiro": {
"id": "kiro",
"name": "Kiro Gateway (Amazon Q Developer)",
"endpoint": "http://localhost:8000/v1",
"name": "Kiro (Amazon Q Developer)",
"endpoint": "https://q.us-east-1.amazonaws.com",
"type": "kiro",
"api_key_required": true,
"api_key": "YOUR_KIRO_API_KEY",
"api_key_required": false,
"rate_limit": 0,
"kiro_config": {
"region": "us-east-1",
"creds_file": "~/.config/Code/User/globalStorage/amazon.q/credentials.json",
"sqlite_db": "",
"refresh_token": "",
"profile_arn": "",
"client_id": "",
"client_secret": ""
},
"models": [
{
"name": "claude-sonnet-4-5",
"name": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"rate_limit": 0,
"max_request_tokens": 200000,
"context_size": 200000
......@@ -426,6 +442,20 @@ In `config/providers.json`:
}
```
**Kiro Configuration Fields (kiro_config):**
- `region`: AWS region (default: "us-east-1")
- `creds_file`: Path to Kiro IDE credentials JSON file (Option 1)
- `sqlite_db`: Path to kiro-cli SQLite database (Option 2)
- `refresh_token`: Direct refresh token for authentication (Option 3)
- `profile_arn`: AWS CodeWhisperer profile ARN (optional)
- `client_id`: OAuth client ID for AWS SSO OIDC (used with refresh_token)
- `client_secret`: OAuth client secret for AWS SSO OIDC (used with refresh_token)
Choose one authentication method:
1. **Kiro IDE credentials**: Set `creds_file` to the path of your Kiro IDE credentials
2. **kiro-cli database**: Set `sqlite_db` to the path of your kiro-cli SQLite database
3. **Direct credentials**: Set `refresh_token` and optionally `client_id`/`client_secret` for AWS SSO OIDC
In `config/rotations.json` (API keys are now referenced from providers.json):
```json
{
......@@ -590,6 +620,25 @@ This AI.PROMPT file is automatically updated when significant changes are made t
### Recent Updates
**2026-03-23 - Proxy-Awareness Implementation**
- Added ProxyHeadersMiddleware to handle reverse proxy deployments
- Implemented automatic detection of proxy headers (X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port, X-Forwarded-Prefix, X-Forwarded-For)
- Updated all RedirectResponse calls to use proxy-aware url_for() function
- Updated templates to use proxy-aware URL generation
- Added comprehensive nginx proxy configuration examples to DOCUMENTATION.md
- Application now fully supports deployment behind reverse proxies with subpaths
- Dashboard URLs, redirects, and API responses automatically adjust based on proxy configuration
**2026-03-23 - Kiro Provider Dashboard Support**
- Updated providers.html dashboard to show kiro-specific configuration fields
- Kiro providers now display `kiro_config` options (region, creds_file, sqlite_db, refresh_token, profile_arn, client_id, client_secret) instead of standard api_key fields
- Added `updateProviderType()` function to handle switching between kiro and other provider types
- Added `updateKiroConfig()` function to manage kiro-specific configuration fields
- Updated `addProvider()` function to ask for provider type and set appropriate defaults for kiro providers
- Dashboard automatically initializes kiro_config when provider type is changed to "kiro"
- Dashboard automatically removes kiro_config when provider type is changed from "kiro" to another type
- Updated AI.PROMPT documentation to clarify kiro provider configuration differences
**2026-03-22 - Configuration Refactoring**
- Centralized API key storage in providers.json
- API keys are now stored only in provider definitions, not in rotation/autoselect configs
......@@ -841,4 +890,118 @@ When making changes that affect the project structure or functionality:
### Version Control
AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.
\ No newline at end of file
AI.PROMPT should be committed to version control along with other project files to ensure that documentation stays synchronized with the codebase.
## Proxy-Awareness and Reverse Proxy Support
### Overview
AISBF is fully proxy-aware and can be deployed behind reverse proxies like nginx, Apache, or Caddy. The application automatically detects and respects standard proxy headers to ensure correct URL generation and routing.
### Implementation
**Proxy Headers Middleware** ([`main.py`](main.py:349-408)):
- `ProxyHeadersMiddleware` class handles proxy header detection
- Automatically updates request scope with proxy information
- Processes standard proxy headers before other middleware
- Must be added before other middleware to ensure proper request scope modification
**Supported Proxy Headers**:
- `X-Forwarded-Proto`: Original protocol (http/https)
- `X-Forwarded-Host`: Original hostname
- `X-Forwarded-Port`: Original port number
- `X-Forwarded-Prefix` or `X-Script-Name`: URL subpath/prefix
- `X-Forwarded-For`: Original client IP address
**Helper Functions** ([`main.py`](main.py:410-445)):
- `get_base_url(request)`: Returns full base URL respecting proxy configuration
- `url_for(request, path)`: Generates proxy-aware URLs for any path
- `setup_template_globals()`: Configures Jinja2 templates with proxy-aware URL functions
### URL Generation
**In Python Code**:
```python
# Use url_for() for all redirects
return RedirectResponse(url=url_for(request, "/dashboard"), status_code=303)
# Get base URL
base_url = get_base_url(request) # e.g., "https://example.com:8443/aisbf"
```
**In Jinja2 Templates**:
```html
<!-- Use url_for() function in templates -->
<a href="{{ url_for(request, '/dashboard/providers') }}">Providers</a>
<!-- For JavaScript fetch calls -->
fetch('{{ url_for(request, "/dashboard/restart") }}', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
})
```
### Deployment Scenarios
**Scenario 1: Root Path Deployment**
- URL: `https://aisbf.example.com/`
- No special configuration needed
- Proxy sets: `X-Forwarded-Proto: https`, `X-Forwarded-Host: aisbf.example.com`
**Scenario 2: Subpath Deployment**
- URL: `https://example.com/aisbf/`
- Proxy must set: `X-Forwarded-Prefix: /aisbf`
- AISBF automatically adjusts all URLs to include `/aisbf` prefix
**Scenario 3: Custom Port**
- URL: `https://example.com:8443/`
- Proxy sets: `X-Forwarded-Port: 8443`
- URLs include port number when non-standard
### Nginx Configuration
See [`DOCUMENTATION.md`](DOCUMENTATION.md) "Proxy Deployment" section for complete nginx configuration examples including:
- Simple reverse proxy (root path)
- Reverse proxy with subpath
- Load balancing multiple instances
- API-only deployment with rate limiting
### Testing Proxy Configuration
After deploying behind a proxy:
1. Access the dashboard and verify all links work correctly
2. Test redirects (login, logout) work properly
3. Verify the restart button uses correct URL
4. Test API endpoints through the proxy
5. Check that streaming responses work correctly
### Common Issues and Solutions
**Issue**: Dashboard links return 404 errors
**Solution**: Ensure `X-Forwarded-Prefix` header is set correctly in nginx
**Issue**: Redirect loops after login
**Solution**: Verify `X-Forwarded-Proto` is set to `https` for HTTPS deployments
**Issue**: URLs missing subpath prefix
**Solution**: Check that `X-Forwarded-Prefix` or `X-Script-Name` header is configured
**Issue**: Streaming responses not working
**Solution**: Ensure `proxy_buffering off` is set in nginx configuration
### Files Modified for Proxy-Awareness
- [`main.py`](main.py): Added ProxyHeadersMiddleware, url_for(), get_base_url() functions
- [`templates/base.html`](templates/base.html): Updated all URLs to use url_for() function
- All dashboard route handlers in [`main.py`](main.py): Updated RedirectResponse calls to use url_for()
- [`DOCUMENTATION.md`](DOCUMENTATION.md): Added comprehensive nginx proxy configuration examples
### Security Considerations
When deploying behind a proxy:
- AISBF should bind to localhost only (default: `127.0.0.1:17765`)
- Proxy should handle SSL/TLS termination
- Enable authentication in AISBF configuration
- Configure rate limiting in the proxy
- Set security headers in proxy configuration
- Monitor access logs for suspicious activity
\ No newline at end of file
API_EXAMPLES.md
View file @ a3506741
This diff is collapsed.
DOCUMENTATION.md
View file @ a3506741
This diff is collapsed.
README.md
View file @ a3506741
This diff is collapsed.
aisbf.sh
View file @ a3506741
#!/bin/bash
#!/bin/bash
########################################################
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
......@@ -91,15 +91,22 @@ ensure_venv() {
fi
}
# Function to update venv packages silently
# Function to update venv packages (only install missing ones, no forced upgrades)
update_venv() {
# Update requirements if requirements.txt exists
# Only update if requirements file exists
if [ -f "$SHARE_DIR/requirements.txt" ]; then
"$VENV_DIR/bin/pip" install --upgrade -r "$SHARE_DIR/requirements.txt" -q 2>/dev/null
# Check if there are any new packages to install (not already satisfied)
"$VENV_DIR/bin/pip" install -r "$SHARE_DIR/requirements.txt" 2>&1 | grep -q "Requirement already satisfied"
ALREADY_SATISFIED=$?
if [ $ALREADY_SATISFIED -ne 0 ]; then
echo "Installing new requirements (this will take a while!) ... "
"$VENV_DIR/bin/pip" install -r "$SHARE_DIR/requirements.txt"
echo "[OK]"
else
echo "Virtual env already up to date"
fi
fi
# Update aisbf package silently
"$VENV_DIR/bin/pip" install --upgrade aisbf -q 2>/dev/null
}
# Function to start the server
......@@ -121,14 +128,19 @@ start_server() {
echo "Starting AISBF on port $PORT..."
# Start the proxy server with logging
# Redirect stderr to suppress BrokenPipeError during shutdown
uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | while IFS= read -r line; do
# Filter out BrokenPipeError logging errors
if [[ "$line" != *"--- Logging error ---"* ]] && [[ "$line" != *"BrokenPipeError"* ]] && [[ "$line" != *"Call stack:"* ]] && [[ "$line" != *"File "*"/python"* ]] && [[ "$line" != *"Message:"* ]] && [[ "$line" != *"Arguments:"* ]]; then
echo "$line" | tee -a "$LOG_DIR/aisbf_stdout.log"
fi
done
# Check if debug mode is enabled
if [ "$DEBUG" = "true" ]; then
echo "Debug mode enabled - showing all debug messages"
export AISBF_DEBUG=true
fi
# Start the proxy server - runs in foreground
# Use exec to replace the shell process so signals are properly handled
if [ "$DEBUG" = "true" ]; then
exec uvicorn main:app --host 127.0.0.1 --port $PORT --log-level debug 2>&1 | tee -a "$LOG_DIR/aisbf_stdout.log"
else
exec uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v -E "(--- Logging error ---|BrokenPipeError|Call stack:|Message:|Arguments:)" | tee -a "$LOG_DIR/aisbf_stdout.log"
fi
}
# Function to start as daemon
......@@ -156,9 +168,19 @@ start_daemon() {
echo "Starting AISBF on port $PORT in background..."
# Check if debug mode is enabled
if [ "$DEBUG" = "true" ]; then
echo "Debug mode enabled - showing all debug messages"
export AISBF_DEBUG=true
fi
# Start in background with nohup and logging
# Filter out BrokenPipeError logging errors
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v '--- Logging error ---' | grep -v 'BrokenPipeError' | grep -v 'Call stack:' | grep -v 'File .*python' | grep -v 'Message:' | grep -v 'Arguments:'" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
if [ "$DEBUG" = "true" ]; then
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT --log-level debug 2>&1" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
else
nohup bash -c "source $VENV_DIR/bin/activate && cd $SHARE_DIR && uvicorn main:app --host 127.0.0.1 --port $PORT 2>&1 | grep -v '--- Logging error ---' | grep -v 'BrokenPipeError' | grep -v 'Call stack:' | grep -v 'File .*python' | grep -v 'Message:' | grep -v 'Arguments:'" >> "$LOG_DIR/aisbf_stdout.log" 2>&1 &
fi
PID=$!
echo $PID > "$PIDFILE"
echo "AISBF started in background (PID: $PID)"
......@@ -200,8 +222,59 @@ stop_daemon() {
fi
}
# Function to show help
show_help() {
echo "AISBF - AI Service Broker Framework"
echo ""
echo "Usage: aisbf.sh [OPTIONS] [COMMAND]"
echo ""
echo "Options:"
echo " --debug Enable debug mode with verbose logging"
echo " -h, --help Show this help message"
echo ""
echo "Commands:"
echo " daemon Start AISBF in background (daemon mode)"
echo " status Check if AISBF is running"
echo " stop Stop the AISBF daemon"
echo ""
echo "Examples:"
echo " aisbf.sh # Start in foreground"
echo " aisbf.sh --debug # Start with debug logging"
echo " aisbf.sh daemon # Start in background"
echo " aisbf.sh --debug daemon # Start in background with debug"
echo " aisbf.sh status # Check status"
echo " aisbf.sh stop # Stop the server"
}
# Parse command line arguments
DEBUG="false"
COMMAND=""
while [[ $# -gt 0 ]]; do
case $1 in
--debug|-d)
DEBUG="true"
shift
;;
-h|--help)
show_help
exit 0
;;
daemon|status|stop)
COMMAND="$1"
shift
;;
*)
echo "Unknown option: $1"
echo ""
show_help
exit 1
;;
esac
done
# Main command handling
case "$1" in
case "$COMMAND" in
daemon)
start_daemon
;;
......
aisbf/config.py
View file @ a3506741
......@@ -66,6 +66,7 @@ class ProviderConfig(BaseModel):
default_condense_method: Optional[Union[str, List[str]]] = None
class RotationConfig(BaseModel):
model_name: str
providers: List[Dict]
notifyerrors: bool = False
# Default settings for models in this rotation
......@@ -104,12 +105,16 @@ class Config:
if custom_dir:
self._custom_config_dir = Path(custom_dir)
# Track loaded file paths for summary
self._loaded_files = {}
self._ensure_config_directory()
self._load_providers()
self._load_rotations()
self._load_autoselect()
self._load_condensation()
self._initialize_error_tracking()
self._log_configuration_summary()
def _get_config_source_dir(self):
"""Get the directory containing default config files"""
......@@ -187,6 +192,7 @@ class Config:
with open(providers_path) as f:
data = json.load(f)
self.providers = {k: ProviderConfig(**v) for k, v in data['providers'].items()}
self._loaded_files['providers'] = str(providers_path.absolute())
logger.info(f"Loaded {len(self.providers)} providers: {list(self.providers.keys())}")
for provider_id, provider_config in self.providers.items():
logger.info(f" - {provider_id}: type={provider_config.type}, endpoint={provider_config.endpoint}")
......@@ -214,6 +220,7 @@ class Config:
logger.info(f"Loading rotations from: {rotations_path}")
with open(rotations_path) as f:
data = json.load(f)
self._loaded_files['rotations'] = str(rotations_path.absolute())
# Extract global notifyerrors setting (top-level, outside rotations)
self.global_notifyerrors = data.get('notifyerrors', False)
......@@ -254,18 +261,31 @@ class Config:
logger.info(f"=== Config._load_rotations END ===")
def _load_autoselect(self):
import logging
logger = logging.getLogger(__name__)
logger.info(f"=== Config._load_autoselect START ===")
autoselect_path = Path.home() / '.aisbf' / 'autoselect.json'
logger.info(f"Looking for autoselect at: {autoselect_path}")
if not autoselect_path.exists():
logger.info(f"User config not found, falling back to source config")
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
autoselect_path = source_dir / 'autoselect.json'
logger.info(f"Using source config at: {autoselect_path}")
except FileNotFoundError:
logger.error("Could not find autoselect.json configuration file")
raise FileNotFoundError("Could not find autoselect.json configuration file")
logger.info(f"Loading autoselect from: {autoselect_path}")
with open(autoselect_path) as f:
data = json.load(f)
self.autoselect = {k: AutoselectConfig(**v) for k, v in data.items()}
self._loaded_files['autoselect'] = str(autoselect_path.absolute())
logger.info(f"Loaded {len(self.autoselect)} autoselect configurations: {list(self.autoselect.keys())}")
logger.info(f"=== Config._load_autoselect END ===")
def _load_condensation(self):
"""Load condensation configuration from providers.json"""
......@@ -274,20 +294,26 @@ class Config:
logger.info(f"=== Config._load_condensation START ===")
providers_path = Path.home() / '.aisbf' / 'providers.json'
logger.info(f"Looking for condensation config in: {providers_path}")
if not providers_path.exists():
logger.info(f"User config not found, falling back to source config")
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
providers_path = source_dir / 'providers.json'
logger.info(f"Using source config at: {providers_path}")
except FileNotFoundError:
logger.warning("Could not find providers.json for condensation config")
self.condensation = CondensationConfig()
return
logger.info(f"Loading condensation config from: {providers_path}")
with open(providers_path) as f:
data = json.load(f)
condensation_data = data.get('condensation', {})
self.condensation = CondensationConfig(**condensation_data)
self._loaded_files['condensation'] = str(providers_path.absolute())
logger.info(f"Loaded condensation config: provider_id={self.condensation.provider_id}, model={self.condensation.model}, enabled={self.condensation.enabled}")
logger.info(f"=== Config._load_condensation END ===")
......@@ -299,6 +325,31 @@ class Config:
'last_failure': None,
'disabled_until': None
}
def _log_configuration_summary(self):
"""Log a summary of all loaded configuration files"""
import logging
logger = logging.getLogger(__name__)
logger.info("")
logger.info("=" * 80)
logger.info("=== CONFIGURATION FILES LOADED ===")
logger.info("=" * 80)
if 'providers' in self._loaded_files:
logger.info(f"Providers: {self._loaded_files['providers']}")
if 'rotations' in self._loaded_files:
logger.info(f"Rotations: {self._loaded_files['rotations']}")
if 'autoselect' in self._loaded_files:
logger.info(f"Autoselect: {self._loaded_files['autoselect']}")
if 'condensation' in self._loaded_files:
logger.info(f"Condensation: {self._loaded_files['condensation']}")
logger.info("=" * 80)
logger.info("")
def get_provider(self, provider_id: str) -> ProviderConfig:
import logging
......
aisbf/context.py
View file @ a3506741
......@@ -145,6 +145,8 @@ class ContextManager:
def _initialize_internal_model(self):
"""Initialize the internal HuggingFace model for condensation (lazy loading)"""
import logging
import json
from pathlib import Path
logger = logging.getLogger(__name__)
if self._internal_model is not None:
......@@ -156,7 +158,33 @@ class ContextManager:
import threading
logger.info("=== INITIALIZING INTERNAL CONDENSATION MODEL ===")
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
# Load model name from config
config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not config_path.exists():
# Try installed locations
installed_dirs = [
Path('/usr/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
test_path = installed_dir / 'aisbf.json'
if test_path.exists():
config_path = test_path
break
else:
# Fallback to source tree
config_path = Path(__file__).parent.parent / 'config' / 'aisbf.json'
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3" # Default
if config_path.exists():
try:
with open(config_path) as f:
aisbf_config = json.load(f)
model_name = aisbf_config.get('internal_model', {}).get('condensation_model_id', model_name)
except Exception as e:
logger.warning(f"Error loading condensation model config: {e}, using default")
logger.info(f"Model: {model_name}")
# Check for GPU availability
......@@ -700,7 +728,13 @@ def get_context_config_for_model(
rotation_model_config: Optional[Dict] = None
) -> Dict:
"""
Get context configuration for a specific model.
Get context configuration for a specific model with cascading fallback.
Priority order for each field:
1. Rotation model config (if explicitly set)
2. Provider model-specific config (if exists)
3. Provider default config (fallback)
4. System default (0 for condense_context, None for others)
Args:
model_name: Name of the model
......@@ -716,19 +750,47 @@ def get_context_config_for_model(
'condense_method': None
}
# Check rotation model config first (highest priority)
if rotation_model_config:
context_config['context_size'] = rotation_model_config.get('context_size')
context_config['condense_context'] = rotation_model_config.get('condense_context', 0)
context_config['condense_method'] = rotation_model_config.get('condense_method')
# Step 1: Get provider-level defaults and model-specific config
model_specific_config = None
if provider_config:
# Try to find model-specific config in provider
if hasattr(provider_config, 'models') and provider_config.models:
for model in provider_config.models:
if model.get('name') == model_name:
model_specific_config = model
break
# Build base config from provider (model-specific > provider defaults)
# context_size
if model_specific_config and model_specific_config.get('context_size') is not None:
context_config['context_size'] = model_specific_config.get('context_size')
elif hasattr(provider_config, 'default_context_size') and provider_config.default_context_size is not None:
context_config['context_size'] = provider_config.default_context_size
# condense_context
if model_specific_config and model_specific_config.get('condense_context') is not None:
context_config['condense_context'] = model_specific_config.get('condense_context')
elif hasattr(provider_config, 'default_condense_context') and provider_config.default_condense_context is not None:
context_config['condense_context'] = provider_config.default_condense_context
else:
context_config['condense_context'] = 0
# condense_method
if model_specific_config and model_specific_config.get('condense_method') is not None:
context_config['condense_method'] = model_specific_config.get('condense_method')
elif hasattr(provider_config, 'default_condense_method') and provider_config.default_condense_method is not None:
context_config['condense_method'] = provider_config.default_condense_method
# Fall back to provider config
elif provider_config and hasattr(provider_config, 'models'):
for model in provider_config.models:
if model.get('name') == model_name:
context_config['context_size'] = model.get('context_size')
context_config['condense_context'] = model.get('condense_context', 0)
context_config['condense_method'] = model.get('condense_method')
break
# Step 2: Override with rotation model config (only if explicitly set)
if rotation_model_config:
# Only override if the field is explicitly set in rotation config
if 'context_size' in rotation_model_config and rotation_model_config['context_size'] is not None:
context_config['context_size'] = rotation_model_config['context_size']
if 'condense_context' in rotation_model_config and rotation_model_config['condense_context'] is not None:
context_config['condense_context'] = rotation_model_config['condense_context']
if 'condense_method' in rotation_model_config and rotation_model_config['condense_method'] is not None:
context_config['condense_method'] = rotation_model_config['condense_method']
return context_config
\ No newline at end of file
aisbf/handlers.py
View file @ a3506741
......@@ -1366,6 +1366,11 @@ class RotationHandler:
logger.info(f" [AVAILABLE] Provider {provider_id} is active and ready")
# Check if provider-level weight is specified
provider_weight = provider.get('weight', 1) # Default to 1 if not specified
if provider.get('weight') is not None:
logger.info(f" Provider-level weight: {provider_weight}")
# Check if models are specified in rotation config
# If not, use models from provider config
rotation_models = provider.get('models')
......@@ -1375,17 +1380,17 @@ class RotationHandler:
# Get models from provider config
if provider_config.models:
# Use models from provider config with default weight of 1
# Use models from provider config with provider-level weight
rotation_models = []
for provider_model in provider_config.models:
model_dict = {
'name': provider_model.name,
'weight': 1, # Default weight
'weight': provider_weight, # Use provider-level weight
'rate_limit': provider_model.rate_limit,
'max_request_tokens': provider_model.max_request_tokens
}
rotation_models.append(model_dict)
logger.info(f" Loaded {len(rotation_models)} model(s) from provider config")
logger.info(f" Loaded {len(rotation_models)} model(s) from provider config with weight {provider_weight}")
else:
logger.warning(f" No models defined in provider config for {provider_id}")
logger.warning(f" Skipping this provider")
......@@ -2530,6 +2535,8 @@ class AutoselectHandler:
def _initialize_internal_model(self):
"""Initialize the internal HuggingFace model for selection (lazy loading)"""
import logging
import json
from pathlib import Path
logger = logging.getLogger(__name__)
if self._internal_model is not None:
......@@ -2541,7 +2548,33 @@ class AutoselectHandler:
import threading
logger.info("=== INITIALIZING INTERNAL SELECTION MODEL ===")
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
# Load model name from config
config_path = Path.home() / '.aisbf' / 'aisbf.json'
if not config_path.exists():
# Try installed locations
installed_dirs = [
Path('/usr/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
test_path = installed_dir / 'aisbf.json'
if test_path.exists():
config_path = test_path
break
else:
# Fallback to source tree
config_path = Path(__file__).parent.parent / 'config' / 'aisbf.json'
model_name = "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3" # Default
if config_path.exists():
try:
with open(config_path) as f:
aisbf_config = json.load(f)
model_name = aisbf_config.get('internal_model', {}).get('autoselect_model_id', model_name)
except Exception as e:
logger.warning(f"Error loading autoselect model config: {e}, using default")
logger.info(f"Model: {model_name}")
# Check for GPU availability
......
aisbf/utils.py
View file @ a3506741
This diff is collapsed.
config/aisbf.json
View file @ a3506741
......@@ -19,6 +19,7 @@
"password": "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
},
"internal_model": {
"model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
"condensation_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
"autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
}
}
config/autoselect.json
View file @ a3506741
......@@ -15,4 +15,4 @@
}
]
}
}
\ No newline at end of file
}
config/rotations.json
View file @ a3506741
......@@ -176,6 +176,27 @@
]
}
]
},
"simple-gemini": {
"_comment": "SIMPLE FORMAT: Specify provider_id only, models will be loaded from providers.json automatically",
"model_name": "simple-gemini",
"notifyerrors": false,
"providers": [
{
"provider_id": "gemini"
}
]
},
"simple-openai": {
"_comment": "SIMPLE FORMAT: Provider with custom weight, models loaded from providers.json",
"model_name": "simple-openai",
"notifyerrors": false,
"providers": [
{
"provider_id": "openai",
"weight": 2
}
]
}
}
}
\ No newline at end of file
main.py
View file @ a3506741
This diff is collapsed.
requirements.txt
View file @ a3506741
......@@ -16,4 +16,5 @@ transformers
jinja2
itsdangerous
bs4
protobuf>=3.20,<4
\ No newline at end of file
protobuf>=3.20,<4
markdown
\ No newline at end of file
setup.py
View file @ a3506741
......@@ -108,6 +108,11 @@ setup(
'templates/dashboard/index.html',
'templates/dashboard/edit_config.html',
'templates/dashboard/settings.html',
'templates/dashboard/providers.html',
'templates/dashboard/rotations.html',
'templates/dashboard/autoselect.html',
'templates/dashboard/prompts.html',
'templates/dashboard/docs.html',
]),
],
entry_points={
......
templates/base.html
View file @ a3506741
......@@ -22,36 +22,36 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<title>{% block title %}AISBF Dashboard{% endblock %}</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; background: #f5f5f5; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif; background: #1a1a2e; color: #e0e0e0; }
.container { max-width: 1200px; margin: 0 auto; padding: 20px; }
.header { background: #2c3e50; color: white; padding: 20px 0; margin-bottom: 30px; }
.header { background: #16213e; color: white; padding: 20px 0; margin-bottom: 30px; border-bottom: 2px solid #0f3460; }
.header h1 { font-size: 24px; font-weight: 600; display: inline-block; }
.header-actions { float: right; }
.nav { background: white; padding: 15px; border-radius: 8px; margin-bottom: 20px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
.nav a { color: #2c3e50; text-decoration: none; margin-right: 20px; padding: 8px 12px; border-radius: 4px; }
.nav a:hover { background: #ecf0f1; }
.nav a.active { background: #3498db; color: white; }
.content { background: white; padding: 30px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
.nav { background: #16213e; padding: 15px; border-radius: 8px; margin-bottom: 20px; box-shadow: 0 2px 4px rgba(0,0,0,0.3); }
.nav a { color: #a0a0a0; text-decoration: none; margin-right: 20px; padding: 8px 12px; border-radius: 4px; }
.nav a:hover { background: #0f3460; color: #e0e0e0; }
.nav a.active { background: #e94560; color: white; }
.content { background: #16213e; padding: 30px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.3); }
.form-group { margin-bottom: 20px; }
.form-group label { display: block; margin-bottom: 5px; font-weight: 500; color: #2c3e50; }
.form-group input, .form-group textarea, .form-group select { width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 4px; font-size: 14px; }
.form-group label { display: block; margin-bottom: 5px; font-weight: 500; color: #e0e0e0; }
.form-group input, .form-group textarea, .form-group select { width: 100%; padding: 10px; border: 1px solid #0f3460; border-radius: 4px; font-size: 14px; background: #1a1a2e; color: #e0e0e0; }
.form-group textarea { min-height: 200px; font-family: 'Courier New', monospace; }
.btn { padding: 10px 20px; background: #3498db; color: white; border: none; border-radius: 4px; cursor: pointer; font-size: 14px; text-decoration: none; display: inline-block; margin-left: 10px; }
.btn:hover { background: #2980b9; }
.btn-secondary { background: #95a5a6; }
.btn-secondary:hover { background: #7f8c8d; }
.btn { padding: 10px 20px; background: #e94560; color: white; border: none; border-radius: 4px; cursor: pointer; font-size: 14px; text-decoration: none; display: inline-block; margin-left: 10px; }
.btn:hover { background: #c73e54; }
.btn-secondary { background: #0f3460; }
.btn-secondary:hover { background: #1a4a7a; }
.btn-danger { background: #e74c3c; }
.btn-danger:hover { background: #c0392b; }
.btn-warning { background: #f39c12; }
.btn-warning:hover { background: #e67e22; }
.alert { padding: 15px; border-radius: 4px; margin-bottom: 20px; }
.alert-success { background: #d4edda; color: #155724; border: 1px solid #c3e6cb; }
.alert-error { background: #f8d7da; color: #721c24; border: 1px solid #f5c6cb; }
.alert-info { background: #d1ecf1; color: #0c5460; border: 1px solid #bee5eb; }
.alert-success { background: #1a3a2e; color: #4ade80; border: 1px solid #22c55e; }
.alert-error { background: #3a1a1a; color: #f87171; border: 1px solid #ef4444; }
.alert-info { background: #1a2a3a; color: #60a5fa; border: 1px solid #3b82f6; }
table { width: 100%; border-collapse: collapse; margin-top: 20px; }
th, td { padding: 12px; text-align: left; border-bottom: 1px solid #ddd; }
th { background: #f8f9fa; font-weight: 600; }
.code { background: #f8f9fa; padding: 15px; border-radius: 4px; font-family: 'Courier New', monospace; font-size: 13px; overflow-x: auto; }
th, td { padding: 12px; text-align: left; border-bottom: 1px solid #0f3460; color: #e0e0e0; }
th { background: #0f3460; font-weight: 600; color: #e0e0e0; }
.code { background: #0f3460; padding: 15px; border-radius: 4px; font-family: 'Courier New', monospace; font-size: 13px; overflow-x: auto; color: #e0e0e0; }
</style>
<script>
/*
......@@ -72,7 +72,7 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
function restartServer() {
if (confirm('Are you sure you want to restart the server? This will disconnect all active connections.')) {
fetch('/dashboard/restart', {
fetch('{{ url_for(request, "/dashboard/restart") }}', {
method: 'POST',
headers: {'Content-Type': 'application/json'}
})
......@@ -95,7 +95,7 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
{% if session.logged_in %}
<div class="header-actions">
<button onclick="restartServer()" class="btn btn-warning">Restart Server</button>
<a href="/dashboard/logout" class="btn btn-secondary">Logout</a>
<a href="{{ url_for(request, '/dashboard/logout') }}" class="btn btn-secondary">Logout</a>
</div>
{% endif %}
</div>
......@@ -104,12 +104,15 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
{% if session.logged_in %}
<div class="container">
<div class="nav">
<a href="/dashboard" {% if request.path == '/dashboard' %}class="active"{% endif %}>Overview</a>
<a href="/dashboard/providers" {% if '/providers' in request.path %}class="active"{% endif %}>Providers</a>
<a href="/dashboard/rotations" {% if '/rotations' in request.path %}class="active"{% endif %}>Rotations</a>
<a href="/dashboard/autoselect" {% if '/autoselect' in request.path %}class="active"{% endif %}>Autoselect</a>
<a href="/dashboard/condensation" {% if '/condensation' in request.path %}class="active"{% endif %}>Condensation</a>
<a href="/dashboard/settings" {% if '/settings' in request.path %}class="active"{% endif %}>Settings</a>
<a href="{{ url_for(request, '/dashboard') }}" {% if request.path == '/dashboard' %}class="active"{% endif %}>Overview</a>
<a href="{{ url_for(request, '/dashboard/providers') }}" {% if '/providers' in request.path %}class="active"{% endif %}>Providers</a>
<a href="{{ url_for(request, '/dashboard/rotations') }}" {% if '/rotations' in request.path %}class="active"{% endif %}>Rotations</a>
<a href="{{ url_for(request, '/dashboard/autoselect') }}" {% if '/autoselect' in request.path %}class="active"{% endif %}>Autoselect</a>
<a href="{{ url_for(request, '/dashboard/prompts') }}" {% if '/prompts' in request.path %}class="active"{% endif %}>Prompts</a>
<a href="{{ url_for(request, '/dashboard/settings') }}" {% if '/settings' in request.path %}class="active"{% endif %}>Settings</a>
<a href="{{ url_for(request, '/dashboard/docs') }}" {% if '/docs' in request.path %}class="active"{% endif %}>Docs</a>
<a href="{{ url_for(request, '/dashboard/about') }}" {% if '/about' in request.path %}class="active"{% endif %}>About</a>
<a href="{{ url_for(request, '/dashboard/license') }}" {% if '/license' in request.path %}class="active"{% endif %}>License</a>
</div>
</div>
{% endif %}
......
templates/dashboard/autoselect.html 0 → 100644
View file @ a3506741
This diff is collapsed.
templates/dashboard/docs.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}{{ title }} - AISBF Dashboard{% endblock %}
{% block content %}
<style>
.markdown-content {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
line-height: 1.6;
color: #e0e0e0;
max-width: 100%;
overflow-x: auto;
}
.markdown-content pre {
background: #0f3460;
padding: 16px;
border-radius: 6px;
overflow-x: auto;
border: 1px solid #1a4a7a;
font-family: 'Courier New', Consolas, Monaco, monospace;
font-size: 13px;
line-height: 1.45;
color: #e0e0e0;
}
.markdown-content code {
background: #0f3460;
padding: 2px 6px;
border-radius: 3px;
font-family: 'Courier New', Consolas, Monaco, monospace;
font-size: 13px;
color: #e0e0e0;
}
.markdown-content h1 {
font-size: 2em;
border-bottom: 2px solid #e94560;
padding-bottom: 0.3em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h2 {
font-size: 1.5em;
border-bottom: 1px solid #0f3460;
padding-bottom: 0.3em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h3 {
font-size: 1.25em;
margin-top: 24px;
margin-bottom: 16px;
color: #ffffff;
}
.markdown-content h4, .markdown-content h5, .markdown-content h6 {
color: #ffffff;
}
.markdown-content ul, .markdown-content ol {
padding-left: 2em;
margin-bottom: 16px;
}
.markdown-content li {
margin-bottom: 4px;
}
.markdown-content p {
margin-bottom: 16px;
}
.markdown-content a {
color: #e94560;
text-decoration: none;
}
.markdown-content a:hover {
text-decoration: underline;
}
.markdown-content blockquote {
padding: 0 1em;
color: #a0a0a0;
border-left: 4px solid #e94560;
margin-bottom: 16px;
}
.markdown-content table {
border-collapse: collapse;
width: 100%;
margin-bottom: 16px;
}
.markdown-content table th,
.markdown-content table td {
padding: 6px 13px;
border: 1px solid #0f3460;
}
.markdown-content table th {
background: #0f3460;
font-weight: 600;
}
.markdown-content table tr:nth-child(2n) {
background: #0f346055;
}
.markdown-content hr {
border: none;
border-top: 1px solid #0f3460;
margin: 24px 0;
}
.markdown-content img {
max-width: 100%;
border-radius: 4px;
}
/* Inline code that isn't in a pre block */
.markdown-content p > code,
.markdown-content li > code {
background: #0f3460;
padding: 2px 6px;
border-radius: 3px;
color: #e94560;
}
</style>
<h2>{{ title }}</h2>
<div class="markdown-content">
{{ content|safe }}
</div>
{% endblock %}
templates/dashboard/index.html
View file @ a3506741
......@@ -61,9 +61,11 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
</table>
<h3 style="margin-top: 30px; margin-bottom: 15px;">Quick Actions</h3>
<div style="display: flex; gap: 10px;">
<div style="display: flex; gap: 10px; flex-wrap: wrap;">
<a href="/dashboard/providers" class="btn">Manage Providers</a>
<a href="/dashboard/rotations" class="btn">Manage Rotations</a>
<a href="/dashboard/autoselect" class="btn">Manage Autoselect</a>
<a href="/dashboard/prompts" class="btn">Manage Prompts</a>
<a href="/dashboard/settings" class="btn btn-secondary">Server Settings</a>
</div>
{% endblock %}
templates/dashboard/prompts.html 0 → 100644
View file @ a3506741
<!--
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
{% extends "base.html" %}
{% block title %}Prompts - AISBF Dashboard{% endblock %}
{% block content %}
<h2 style="margin-bottom: 30px;">Prompts Configuration</h2>
{% if success %}
<div class="alert alert-success">{{ success }}</div>
{% endif %}
{% if error %}
<div class="alert alert-error">{{ error }}</div>
{% endif %}
<div style="margin-bottom: 20px;">
<label style="font-weight: 500; margin-bottom: 10px; display: block;">Select Prompt File:</label>
<select id="prompt-selector" onchange="switchPrompt()" style="width: 100%; padding: 10px; border: 1px solid #ddd; border-radius: 3px; font-size: 14px;">
{% for prompt in prompts %}
<option value="{{ prompt.key }}" {% if loop.first %}selected{% endif %}>{{ prompt.name }}</option>
{% endfor %}
</select>
</div>
<form method="POST" id="prompt-form">
<input type="hidden" name="prompt_key" id="prompt_key" value="">
<div class="form-group">
<label for="prompt_content" style="font-weight: 500; margin-bottom: 10px; display: block;">Prompt Content:</label>
<textarea id="prompt_content" name="prompt_content" style="width: 100%; min-height: 400px; padding: 10px; border: 1px solid #ddd; border-radius: 3px; font-family: monospace; font-size: 13px; line-height: 1.5;"></textarea>
<small style="color: #666; display: block; margin-top: 5px;">Edit the prompt template. Use markdown formatting as needed.</small>
</div>
<div style="display: flex; gap: 10px; margin-top: 20px;">
<button type="submit" class="btn">Save Prompt</button>
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
</form>
<script>
const prompts = {{ prompts_data | safe }};
let currentPromptKey = '';
function switchPrompt() {
const selector = document.getElementById('prompt-selector');
const key = selector.value;
currentPromptKey = key;
const prompt = prompts.find(p => p.key === key);
if (prompt) {
document.getElementById('prompt_content').value = prompt.content;
document.getElementById('prompt_key').value = key;
}
}
// Initialize with first prompt
if (prompts.length > 0) {
currentPromptKey = prompts[0].key;
document.getElementById('prompt_content').value = prompts[0].content;
document.getElementById('prompt_key').value = prompts[0].key;
}
</script>
<style>
.form-group {
margin-bottom: 20px;
}
textarea {
resize: vertical;
}
.btn {
padding: 10px 20px;
border: none;
border-radius: 3px;
cursor: pointer;
font-size: 14px;
text-decoration: none;
display: inline-block;
}
.btn-secondary {
background: #6c757d;
color: white;
}
.btn-secondary:hover {
background: #5a6268;
}
</style>
{% endblock %}
templates/dashboard/providers.html 0 → 100644
View file @ a3506741
This diff is collapsed.
templates/dashboard/rotations.html 0 → 100644
View file @ a3506741
This diff is collapsed.
templates/dashboard/settings.html
View file @ a3506741
......@@ -44,12 +44,32 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<div class="form-group">
<label for="protocol">Protocol</label>
<select id="protocol" name="protocol">
<select id="protocol" name="protocol" onchange="toggleSSLFields()">
<option value="http" {% if config.server.protocol == 'http' %}selected{% endif %}>HTTP</option>
<option value="https" {% if config.server.protocol == 'https' %}selected{% endif %}>HTTPS</option>
</select>
</div>
<div id="ssl-fields" style="display: {% if config.server.protocol == 'https' %}block{% else %}none{% endif %};">
<div class="form-group">
<label for="public_domain">Public Domain (for Let's Encrypt)</label>
<input type="text" id="public_domain" name="public_domain" value="{{ config.server.public_domain or '' }}" placeholder="example.com (optional)">
<small style="color: #666; display: block; margin-top: 5px;">If specified, Let's Encrypt will be used to generate valid SSL certificates. Leave empty to use self-signed certificates.</small>
</div>
<div class="form-group">
<label for="ssl_cert">SSL Certificate Path</label>
<input type="text" id="ssl_cert" name="ssl_cert" value="{{ config.server.ssl_cert or '' }}" placeholder="~/.aisbf/cert.pem (default)">
<small style="color: #666; display: block; margin-top: 5px;">Path to SSL certificate file. Leave empty to use default (~/.aisbf/cert.pem). Auto-generated using Let's Encrypt if public domain is set, otherwise self-signed.</small>
</div>
<div class="form-group">
<label for="ssl_key">SSL Key Path</label>
<input type="text" id="ssl_key" name="ssl_key" value="{{ config.server.ssl_key or '' }}" placeholder="~/.aisbf/key.pem (default)">
<small style="color: #666; display: block; margin-top: 5px;">Path to SSL private key file. Leave empty to use default (~/.aisbf/key.pem). Auto-generated with certificate.</small>
</div>
</div>
<h3 style="margin: 30px 0 20px;">Authentication</h3>
<div class="form-group">
......@@ -76,11 +96,18 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<input type="password" id="dashboard_password" name="dashboard_password" placeholder="Leave blank to keep current">
</div>
<h3 style="margin: 30px 0 20px;">Internal Model</h3>
<h3 style="margin: 30px 0 20px;">Internal Models</h3>
<div class="form-group">
<label for="internal_model_id">Model ID</label>
<input type="text" id="internal_model_id" name="internal_model_id" value="{{ config.internal_model.model_id }}" required>
<label for="condensation_model_id">Condensation Model ID</label>
<input type="text" id="condensation_model_id" name="condensation_model_id" value="{{ config.internal_model.condensation_model_id }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Used when condensation model is set to "internal"</small>
</div>
<div class="form-group">
<label for="autoselect_model_id">Autoselect Model ID</label>
<input type="text" id="autoselect_model_id" name="autoselect_model_id" value="{{ config.internal_model.autoselect_model_id }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Used when autoselect selection_model is set to "internal"</small>
</div>
<div style="display: flex; gap: 10px; margin-top: 30px;">
......@@ -88,4 +115,17 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<a href="/dashboard" class="btn btn-secondary">Cancel</a>
</div>
</form>
<script>
function toggleSSLFields() {
const protocol = document.getElementById('protocol').value;
const sslFields = document.getElementById('ssl-fields');
if (protocol === 'https') {
sslFields.style.display = 'block';
} else {
sslFields.style.display = 'none';
}
}
</script>
{% endblock %}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment