Add donation section to documentation and update .gitignore

parent 768fce37
Pipeline #216 canceled with stages
# Virtual environments
venv/
.venv/
env/
ENV/
# Python cache directories
__pycache__/
*.py[cod]
*$py.class
# Byte-compiled / optimized / DLL files
*.pyc
*.pyo
*.pyd
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
Pipfile.lock
.venv
# PEP 582
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# Temporary files
*.swp
*.swo
*~
# Setup.py artifacts
aisbf-venv/
aisbf.egg-info/
dist/
build/
*.egg
*.whl
# Local configuration files
.local/
.local/share/aisbf/
.aisbf/
# IDE files
.vscode/
.idea/
*.sublime-workspace
*.sublime-project
# OS files
.DS_Store
Thumbs.db
\ No newline at end of file
...@@ -334,3 +334,24 @@ For support and questions: ...@@ -334,3 +334,24 @@ For support and questions:
- Review the INSTALL.md file for installation details - Review the INSTALL.md file for installation details
- Check the README.md file for project overview - Check the README.md file for project overview
- Test with development scripts before production deployment - Test with development scripts before production deployment
## Donations
The extension includes multiple donation options to support its development:
### Web3/MetaMask Donation
Works on any website - The Web3 donation is completely independent of the current page
Click the "🦊 Donate with MetaMask" button in the extension popup (only appears if MetaMask is detected)
Supports both modern window.ethereum and legacy window.web3 providers
Default donation: 0.1 ETH to 0xdA6dAb526515b5cb556d20269207D43fcc760E51
Users can modify the amount in MetaMask before confirming
### PayPal Donation
Click the "💳 Donate with PayPal" button in the extension popup
Opens PayPal donation page for info@nexlab.net
Traditional payment method for users without cryptocurrency wallets
Always available regardless of browser setup
### Bitcoin Donation
Address: bc1qcpt2uutqkz4456j5r78rjm3gwq03h5fpwmcc5u
Traditional BTC donation method
# AISBF - AI Service Broker Framework
## Overview
AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Ollama) with support for provider rotation and error tracking.
## Project Structure
```
geminiproxy/
├── aisbf/ # Main Python module
│ ├── __init__.py # Module initialization with exports
│ ├── config.py # Configuration management
│ ├── models.py # Pydantic models
│ ├── providers.py # Provider handlers
│ ├── handlers.py # Request handlers
│ ├── providers.json # Default provider configs (moved to config/)
│ └── rotations.json # Default rotation configs (moved to config/)
├── config/ # Configuration files directory
│ ├── providers.json # Default provider configurations
│ └── rotations.json # Default rotation configurations
├── main.py # FastAPI application entry point
├── setup.py # Installation script
├── start_proxy.sh # Development start script
├── aisbf.sh # Alternative start script
├── requirements.txt # Python dependencies
├── INSTALL.md # Installation guide
└── README.md # Project documentation
```
## Installation
### User Installation (no root required)
```bash
python setup.py install
```
Installs to:
- `~/.local/lib/python*/site-packages/aisbf/` - Package
- `~/.local/aisbf-venv/` - Virtual environment
- `~/.local/share/aisbf/` - Config files
- `~/.local/bin/aisbf` - Executable script
### System-wide Installation (requires root)
```bash
sudo python setup.py install
```
Installs to:
- `/usr/local/lib/python*/dist-packages/aisbf/` - Package
- `/usr/local/aisbf-venv/` - Virtual environment
- `/usr/local/share/aisbf/` - Config files
- `/usr/local/bin/aisbf` - Executable script
## Configuration Management
### Configuration File Locations
**Installed Configuration Files (read-only defaults):**
- User: `~/.local/share/aisbf/providers.json`, `~/.local/share/aisbf/rotations.json`
- System: `/usr/local/share/aisbf/providers.json`, `/usr/local/share/aisbf/rotations.json`
**User Configuration Files (writable):**
- `~/.aisbf/providers.json` - Provider configurations
- `~/.aisbf/rotations.json` - Rotation configurations
**Development Mode:**
- `config/providers.json` and `config/rotations.json` in source tree
### First Run Behavior
1. Checks for config files in installed location
2. Creates `~/.aisbf/` directory if needed
3. Copies default configs from installed location to `~/.aisbf/`
4. Loads configuration from `~/.aisbf/` on subsequent runs
## API Endpoints
### Root Endpoint
- `GET /` - Returns server status and list of available providers
### Chat Completions
- `POST /api/{provider_id}/chat/completions` - Handle chat completion requests
- Supports both streaming and non-streaming responses
- Provider ID can be a specific provider or rotation name
### Model List
- `GET /api/{provider_id}/models` - List available models for a provider or rotation
## Provider Support
AISBF supports the following AI providers:
### Google
- Uses google-genai SDK
- Requires API key
- Supports streaming and non-streaming responses
### OpenAI
- Uses openai SDK
- Requires API key
- Supports streaming and non-streaming responses
### Anthropic
- Uses anthropic SDK
- Requires API key
- Static model list (no dynamic model discovery)
### Ollama
- Uses direct HTTP API
- No API key required
- Local model hosting support
## Rotation Support
AISBF supports provider rotation with weighted model selection:
### Rotation Configuration
```json
{
"rotations": {
"my_rotation": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
### Rotation Behavior
- Weighted random selection of models
- Automatic failover between providers
- Error tracking and rate limiting
## Error Tracking and Rate Limiting
### Error Tracking
- Tracks failures per provider
- Disables providers after 3 consecutive failures
- 5-minute cooldown period for disabled providers
### Rate Limiting
- Automatic provider disabling when rate limited
- Graceful error handling
- Configurable retry behavior
## Development vs Production
### Development
Use `start_proxy.sh`:
- Creates local venv in `./venv/`
- Installs dependencies from `requirements.txt`
- Starts server with auto-reload enabled
- Uses `config/` directory for configuration
### Production
Install with `python setup.py install`:
- Creates isolated venv
- Installs all dependencies
- Provides `aisbf` command with daemon support
- Uses installed config files
## AISBF Script Commands
### Starting in Foreground (Default)
```bash
aisbf
```
Starts server in foreground with visible output.
### Starting as Daemon
```bash
aisbf daemon
```
- Starts in background
- Saves PID to `/tmp/aisbf.pid`
- Redirects output to `/dev/null`
- Prints PID of started process
### Checking Status
```bash
aisbf status
```
Checks if AISBF is running and reports status/PID.
### Stopping the Daemon
```bash
aisbf stop
```
Stops running daemon and removes PID file.
## Key Classes and Functions
### aisbf/config.py
- `Config` - Configuration management class
- `ProviderConfig` - Provider configuration model
- `RotationConfig` - Rotation configuration model
- `AppConfig` - Application configuration model
### aisbf/models.py
- `Message` - Chat message structure
- `ChatCompletionRequest` - Request model
- `ChatCompletionResponse` - Response model
- `Model` - Model information
- `Provider` - Provider information
- `ErrorTracking` - Error tracking data
### aisbf/providers.py
- `BaseProviderHandler` - Base provider handler class
- `GoogleProviderHandler` - Google provider implementation
- `OpenAIProviderHandler` - OpenAI provider implementation
- `AnthropicProviderHandler` - Anthropic provider implementation
- `OllamaProviderHandler` - Ollama provider implementation
- `get_provider_handler()` - Factory function for provider handlers
### aisbf/handlers.py
- `RequestHandler` - Request handling logic
- `RotationHandler` - Rotation handling logic
## Dependencies
Key dependencies from requirements.txt:
- fastapi - Web framework
- uvicorn - ASGI server
- pydantic - Data validation
- httpx - HTTP client
- google-genai - Google AI SDK
- openai - OpenAI SDK
- anthropic - Anthropic SDK
## Adding New Providers
### Steps to Add a New Provider
1. Create handler class in `aisbf/providers.py` inheriting from `BaseProviderHandler`
2. Add to `PROVIDER_HANDLERS` dictionary
3. Add provider configuration to `config/providers.json`
### Provider Handler Requirements
- Implement `handle_request()` method
- Implement `get_models()` method
- Handle error tracking and rate limiting
## Configuration Examples
### Provider Configuration
```json
{
"providers": {
"openai": {
"id": "openai",
"name": "OpenAI",
"endpoint": "https://api.openai.com/v1",
"type": "openai",
"api_key_required": true
}
}
}
```
### Rotation Configuration
```json
{
"rotations": {
"balanced": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
## Testing and Development
### Development Workflow
1. Use `start_proxy.sh` for development
2. Test with `start_proxy.sh` for development
3. Install with `python setup.py install` for production testing
4. Test all `aisbf` script commands (default, daemon, status, stop)
5. Verify configuration file locations and behavior
6. Test both user and system installations
### Common Development Tasks
- Adding new providers
- Modifying configuration
- Updating installation
- Testing error handling
## License
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.
## Contributing
When making changes:
1. Update AI.PROMPT file with significant changes
2. Test all functionality
3. Update documentation as needed
4. Follow the project's coding conventions
5. Ensure all tests pass
## Support
For support and questions:
- Check the AI.PROMPT file for project-specific instructions
- Review the INSTALL.md file for installation details
- Check the README.md file for project overview
- Test with development scripts before production deployment
\ No newline at end of file
...@@ -88,4 +88,25 @@ The proxy includes robust error handling with: ...@@ -88,4 +88,25 @@ The proxy includes robust error handling with:
- Rate limiting for failed requests - Rate limiting for failed requests
- Automatic retry with provider rotation - Automatic retry with provider rotation
- Proper error tracking and logging - Proper error tracking and logging
\ No newline at end of file
## Donations
The extension includes multiple donation options to support its development:
### Web3/MetaMask Donation
Works on any website - The Web3 donation is completely independent of the current page
Click the "🦊 Donate with MetaMask" button in the extension popup (only appears if MetaMask is detected)
Supports both modern window.ethereum and legacy window.web3 providers
Default donation: 0.1 ETH to 0xdA6dAb526515b5cb556d20269207D43fcc760E51
Users can modify the amount in MetaMask before confirming
### PayPal Donation
Click the "💳 Donate with PayPal" button in the extension popup
Opens PayPal donation page for info@nexlab.net
Traditional payment method for users without cryptocurrency wallets
Always available regardless of browser setup
### Bitcoin Donation
Address: bc1qcpt2uutqkz4456j5r78rjm3gwq03h5fpwmcc5u
Traditional BTC donation method
\ No newline at end of file
#!/bin/bash
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
# AISBF - AI Service Broker Framework || AI Should Be Free
# This script starts the AISBF proxy server using the installed virtual environment
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Check if venv exists
if [ ! -d "venv" ]; then
echo "Error: Virtual environment not found. Please run: python setup.py install"
exit 1
fi
# Activate virtual environment
source venv/bin/activate
# Start the proxy server
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
A modular proxy server for managing multiple AI provider integrations.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
A modular proxy server for managing multiple AI provider integrations.
"""
from .config import config, Config, ProviderConfig, RotationConfig, AppConfig
from .models import (
Message,
ChatCompletionRequest,
ChatCompletionResponse,
Model,
Provider,
ErrorTracking
)
from .providers import (
BaseProviderHandler,
GoogleProviderHandler,
OpenAIProviderHandler,
AnthropicProviderHandler,
OllamaProviderHandler,
get_provider_handler,
PROVIDER_HANDLERS
)
from .handlers import RequestHandler, RotationHandler
__version__ = "0.1.0"
__all__ = [
# Config
"config",
"Config",
"ProviderConfig",
"RotationConfig",
"AppConfig",
# Models
"Message",
"ChatCompletionRequest",
"ChatCompletionResponse",
"Model",
"Provider",
"ErrorTracking",
# Providers
"BaseProviderHandler",
"GoogleProviderHandler",
"OpenAIProviderHandler",
"AnthropicProviderHandler",
"OllamaProviderHandler",
"get_provider_handler",
"PROVIDER_HANDLERS",
# Handlers
"RequestHandler",
"RotationHandler",
]
\ No newline at end of file
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Configuration management for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Configuration management for AISBF.
"""
from typing import Dict, List, Optional
from pydantic import BaseModel, Field
import json
import shutil
from pathlib import Path
class ProviderConfig(BaseModel):
id: str
name: str
endpoint: str
type: str
api_key_required: bool
class RotationConfig(BaseModel):
providers: List[Dict]
class AppConfig(BaseModel):
providers: Dict[str, ProviderConfig]
rotations: Dict[str, RotationConfig]
error_tracking: Dict[str, Dict]
class Config:
def __init__(self):
self._ensure_config_directory()
self._load_providers()
self._load_rotations()
self._initialize_error_tracking()
def _get_config_source_dir(self):
"""Get the directory containing default config files"""
# Try installed location first
installed_dirs = [
Path('/usr/local/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
if installed_dir.exists() and (installed_dir / 'providers.json').exists():
return installed_dir
# Fallback to source tree config directory
# This is for development mode
source_dir = Path(__file__).parent.parent.parent / 'config'
if source_dir.exists() and (source_dir / 'providers.json').exists():
return source_dir
# Last resort: try the old location in the package directory
package_dir = Path(__file__).parent
if (package_dir / 'providers.json').exists():
return package_dir
raise FileNotFoundError("Could not find configuration files")
def _ensure_config_directory(self):
"""Ensure ~/.aisbf/ directory exists and copy default config files if needed"""
config_dir = Path.home() / '.aisbf'
# Create config directory if it doesn't exist
config_dir.mkdir(exist_ok=True)
# Get the source directory for default config files
try:
source_dir = self._get_config_source_dir()
except FileNotFoundError:
print("Warning: Could not find default configuration files")
return
# Copy default config files if they don't exist
for config_file in ['providers.json', 'rotations.json']:
src = source_dir / config_file
dst = config_dir / config_file
if not dst.exists() and src.exists():
shutil.copy2(src, dst)
print(f"Created default config file: {dst}")
def _load_providers(self):
providers_path = Path.home() / '.aisbf' / 'providers.json'
if not providers_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
providers_path = source_dir / 'providers.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find providers.json configuration file")
with open(providers_path) as f:
data = json.load(f)
self.providers = {k: ProviderConfig(**v) for k, v in data['providers'].items()}
def _load_rotations(self):
rotations_path = Path.home() / '.aisbf' / 'rotations.json'
if not rotations_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
rotations_path = source_dir / 'rotations.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find rotations.json configuration file")
with open(rotations_path) as f:
data = json.load(f)
self.rotations = {k: RotationConfig(**v) for k, v in data['rotations'].items()}
def _initialize_error_tracking(self):
self.error_tracking = {}
for provider_id in self.providers:
self.error_tracking[provider_id] = {
'failures': 0,
'last_failure': None,
'disabled_until': None
}
def get_provider(self, provider_id: str) -> ProviderConfig:
return self.providers.get(provider_id)
def get_rotation(self, rotation_id: str) -> RotationConfig:
return self.rotations.get(rotation_id)
config = Config()
\ No newline at end of file
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Request handlers for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Request handlers for AISBF.
"""
import asyncio
from typing import Dict, List, Optional
from fastapi import HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
from .models import ChatCompletionRequest, ChatCompletionResponse
from .providers import get_provider_handler
from .config import config
class RequestHandler:
def __init__(self):
self.config = config
async def handle_chat_completion(self, request: Request, provider_id: str, request_data: Dict) -> Dict:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_streaming_chat_completion(self, request: Request, provider_id: str, request_data: Dict):
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
async def stream_generator():
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=True
)
for chunk in response:
yield f"data: {chunk}\n\n".encode('utf-8')
handler.record_success()
except Exception as e:
handler.record_failure()
yield f"data: {str(e)}\n\n".encode('utf-8')
return StreamingResponse(stream_generator(), media_type="text/event-stream")
async def handle_model_list(self, request: Request, provider_id: str) -> List[Dict]:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
try:
models = await handler.get_models()
return [model.dict() for model in models]
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
class RotationHandler:
def __init__(self):
self.config = config
async def handle_rotation_request(self, rotation_id: str, request_data: Dict) -> Dict:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
providers = rotation_config.providers
weighted_models = []
for provider in providers:
for model in provider['models']:
weighted_models.extend([model] * model['weight'])
if not weighted_models:
raise HTTPException(status_code=400, detail="No models available in rotation")
import random
selected_model = random.choice(weighted_models)
provider_id = selected_model['provider_id']
api_key = selected_model.get('api_key')
model_name = selected_model['name']
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="All providers temporarily unavailable")
try:
response = await handler.handle_request(
model=model_name,
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_rotation_model_list(self, rotation_id: str) -> List[Dict]:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
all_models = []
for provider in rotation_config.providers:
for model in provider['models']:
all_models.append({
"id": f"{provider['provider_id']}/{model['name']}",
"name": model['name'],
"provider_id": provider['provider_id'],
"weight": model['weight']
})
return all_models
from pydantic import BaseModel, Field
from typing import Dict, List, Optional, Union
class Message(BaseModel):
role: str
content: Union[str, List[Dict]]
class ChatCompletionRequest(BaseModel):
model: str
messages: List[Message]
max_tokens: Optional[int] = None
temperature: Optional[float] = 1.0
stream: Optional[bool] = False
class ChatCompletionResponse(BaseModel):
id: str
object: str
created: int
model: str
choices: List[Dict]
usage: Optional[Dict] = None
stream: Optional[bool] = False
class Model(BaseModel):
id: str
name: str
provider_id: str
weight: int = 1
class Provider(BaseModel):
id: str
name: str
type: str
endpoint: str
api_key_required: bool
models: List[Model] = []
class ErrorTracking(BaseModel):
failures: int
last_failure: Optional[int]
disabled_until: Optional[int]
\ No newline at end of file
This diff is collapsed.
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Main application for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Main application for AISBF.
"""
from fastapi import FastAPI, HTTPException, Request, status
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from aisbf.models import ChatCompletionRequest, ChatCompletionResponse
from aisbf.handlers import RequestHandler, RotationHandler
from aisbf.config import config
import time
import logging
from datetime import datetime, timedelta
from collections import defaultdict
# Configure logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
# Initialize handlers
request_handler = RequestHandler()
rotation_handler = RotationHandler()
app = FastAPI(title="AI Proxy Server")
# CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.get("/")
async def root():
return {"message": "AI Proxy Server is running", "providers": list(config.providers.keys())}
@app.post("/api/{provider_id}/chat/completions")
async def chat_completions(provider_id: str, request: Request, body: ChatCompletionRequest):
logger.debug(f"Received chat_completions request for provider: {provider_id}")
logger.debug(f"Request headers: {dict(request.headers)}")
logger.debug(f"Request body: {body}")
body_dict = body.model_dump()
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation request")
return await rotation_handler.handle_rotation_request(provider_id, body_dict)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
logger.debug(f"Provider config: {provider_config}")
try:
if body.stream:
logger.debug("Handling streaming chat completion")
return await request_handler.handle_streaming_chat_completion(request, provider_id, body_dict)
else:
logger.debug("Handling non-streaming chat completion")
result = await request_handler.handle_chat_completion(request, provider_id, body_dict)
logger.debug(f"Response result: {result}")
return result
except Exception as e:
logger.error(f"Error handling chat_completions: {str(e)}", exc_info=True)
raise
@app.get("/api/{provider_id}/models")
async def list_models(request: Request, provider_id: str):
logger.debug(f"Received list_models request for provider: {provider_id}")
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation model list request")
return await rotation_handler.handle_rotation_model_list(provider_id)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
try:
logger.debug("Handling model list request")
result = await request_handler.handle_model_list(request, provider_id)
logger.debug(f"Models result: {result}")
return result
except Exception as e:
logger.error(f"Error handling list_models: {str(e)}", exc_info=True)
raise
def main():
"""Main entry point for the AISBF server"""
import uvicorn
logger.info("Starting AI Proxy Server on http://localhost:8000")
uvicorn.run(app, host="0.0.0.0", port=8000)
if __name__ == "__main__":
main()
\ No newline at end of file
#!/bin/bash
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
# AISBF - AI Service Broker Framework || AI Should Be Free
# DEVELOPMENT START SCRIPT - For development use only
# For production use, install with: python setup.py install
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Create virtual environment if it doesn't exist
if [ ! -d "venv" ]; then
echo "Creating virtual environment for development..."
python3 -m venv venv
fi
# Activate virtual environment
source venv/bin/activate
# Install dependencies
echo "Installing dependencies..."
pip install -r requirements.txt
# Start the proxy server
echo "Starting AISBF development server..."
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment