Initial commit: AISBF - AI Service Broker Framework

parents
This diff is collapsed.
# AISBF - AI Service Broker Framework || AI Should Be Free
## Overview
AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Ollama) with support for provider rotation and error tracking.
## Project Structure
```
geminiproxy/
├── aisbf/ # Main Python module
│ ├── __init__.py # Module initialization with exports
│ ├── config.py # Configuration management
│ ├── models.py # Pydantic models
│ ├── providers.py # Provider handlers
│ ├── handlers.py # Request handlers
│ ├── providers.json # Default provider configs (moved to config/)
│ └── rotations.json # Default rotation configs (moved to config/)
├── config/ # Configuration files directory
│ ├── providers.json # Default provider configurations
│ └── rotations.json # Default rotation configurations
├── main.py # FastAPI application entry point
├── setup.py # Installation script
├── start_proxy.sh # Development start script
├── aisbf.sh # Alternative start script
├── requirements.txt # Python dependencies
├── INSTALL.md # Installation guide
└── README.md # Project documentation
```
## Installation
### User Installation (no root required)
```bash
python setup.py install
```
Installs to:
- `~/.local/lib/python*/site-packages/aisbf/` - Package
- `~/.local/aisbf-venv/` - Virtual environment
- `~/.local/share/aisbf/` - Config files
- `~/.local/bin/aisbf` - Executable script
### System-wide Installation (requires root)
```bash
sudo python setup.py install
```
Installs to:
- `/usr/local/lib/python*/dist-packages/aisbf/` - Package
- `/usr/local/aisbf-venv/` - Virtual environment
- `/usr/local/share/aisbf/` - Config files
- `/usr/local/bin/aisbf` - Executable script
## Configuration Management
### Configuration File Locations
**Installed Configuration Files (read-only defaults):**
- User: `~/.local/share/aisbf/providers.json`, `~/.local/share/aisbf/rotations.json`
- System: `/usr/local/share/aisbf/providers.json`, `/usr/local/share/aisbf/rotations.json`
**User Configuration Files (writable):**
- `~/.aisbf/providers.json` - Provider configurations
- `~/.aisbf/rotations.json` - Rotation configurations
**Development Mode:**
- `config/providers.json` and `config/rotations.json` in source tree
### First Run Behavior
1. Checks for config files in installed location
2. Creates `~/.aisbf/` directory if needed
3. Copies default configs from installed location to `~/.aisbf/`
4. Loads configuration from `~/.aisbf/` on subsequent runs
## API Endpoints
### Root Endpoint
- `GET /` - Returns server status and list of available providers
### Chat Completions
- `POST /api/{provider_id}/chat/completions` - Handle chat completion requests
- Supports both streaming and non-streaming responses
- Provider ID can be a specific provider or rotation name
### Model List
- `GET /api/{provider_id}/models` - List available models for a provider or rotation
## Provider Support
AISBF supports the following AI providers:
### Google
- Uses google-genai SDK
- Requires API key
- Supports streaming and non-streaming responses
### OpenAI
- Uses openai SDK
- Requires API key
- Supports streaming and non-streaming responses
### Anthropic
- Uses anthropic SDK
- Requires API key
- Static model list (no dynamic model discovery)
### Ollama
- Uses direct HTTP API
- No API key required
- Local model hosting support
## Rotation Support
AISBF supports provider rotation with weighted model selection:
### Rotation Configuration
```json
{
"rotations": {
"my_rotation": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
### Rotation Behavior
- Weighted random selection of models
- Automatic failover between providers
- Error tracking and rate limiting
## Error Tracking and Rate Limiting
### Error Tracking
- Tracks failures per provider
- Disables providers after 3 consecutive failures
- 5-minute cooldown period for disabled providers
### Rate Limiting
- Automatic provider disabling when rate limited
- Graceful error handling
- Configurable retry behavior
## Development vs Production
### Development
Use `start_proxy.sh`:
- Creates local venv in `./venv/`
- Installs dependencies from `requirements.txt`
- Starts server with auto-reload enabled
- Uses `config/` directory for configuration
### Production
Install with `python setup.py install`:
- Creates isolated venv
- Installs all dependencies
- Provides `aisbf` command with daemon support
- Uses installed config files
## AISBF Script Commands
### Starting in Foreground (Default)
```bash
aisbf
```
Starts server in foreground with visible output.
### Starting as Daemon
```bash
aisbf daemon
```
- Starts in background
- Saves PID to `/tmp/aisbf.pid`
- Redirects output to `/dev/null`
- Prints PID of started process
### Checking Status
```bash
aisbf status
```
Checks if AISBF is running and reports status/PID.
### Stopping the Daemon
```bash
aisbf stop
```
Stops running daemon and removes PID file.
## Key Classes and Functions
### aisbf/config.py
- `Config` - Configuration management class
- `ProviderConfig` - Provider configuration model
- `RotationConfig` - Rotation configuration model
- `AppConfig` - Application configuration model
### aisbf/models.py
- `Message` - Chat message structure
- `ChatCompletionRequest` - Request model
- `ChatCompletionResponse` - Response model
- `Model` - Model information
- `Provider` - Provider information
- `ErrorTracking` - Error tracking data
### aisbf/providers.py
- `BaseProviderHandler` - Base provider handler class
- `GoogleProviderHandler` - Google provider implementation
- `OpenAIProviderHandler` - OpenAI provider implementation
- `AnthropicProviderHandler` - Anthropic provider implementation
- `OllamaProviderHandler` - Ollama provider implementation
- `get_provider_handler()` - Factory function for provider handlers
### aisbf/handlers.py
- `RequestHandler` - Request handling logic
- `RotationHandler` - Rotation handling logic
## Dependencies
Key dependencies from requirements.txt:
- fastapi - Web framework
- uvicorn - ASGI server
- pydantic - Data validation
- httpx - HTTP client
- google-genai - Google AI SDK
- openai - OpenAI SDK
- anthropic - Anthropic SDK
## Adding New Providers
### Steps to Add a New Provider
1. Create handler class in `aisbf/providers.py` inheriting from `BaseProviderHandler`
2. Add to `PROVIDER_HANDLERS` dictionary
3. Add provider configuration to `config/providers.json`
### Provider Handler Requirements
- Implement `handle_request()` method
- Implement `get_models()` method
- Handle error tracking and rate limiting
## Configuration Examples
### Provider Configuration
```json
{
"providers": {
"openai": {
"id": "openai",
"name": "OpenAI",
"endpoint": "https://api.openai.com/v1",
"type": "openai",
"api_key_required": true
}
}
}
```
### Rotation Configuration
```json
{
"rotations": {
"balanced": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
## Testing and Development
### Development Workflow
1. Use `start_proxy.sh` for development
2. Test with `start_proxy.sh` for development
3. Install with `python setup.py install` for production testing
4. Test all `aisbf` script commands (default, daemon, status, stop)
5. Verify configuration file locations and behavior
6. Test both user and system installations
### Common Development Tasks
- Adding new providers
- Modifying configuration
- Updating installation
- Testing error handling
## License
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.
## Contributing
When making changes:
1. Update AI.PROMPT file with significant changes
2. Test all functionality
3. Update documentation as needed
4. Follow the project's coding conventions
5. Ensure all tests pass
## Support
For support and questions:
- Check the AI.PROMPT file for project-specific instructions
- Review the INSTALL.md file for installation details
- Check the README.md file for project overview
- Test with development scripts before production deployment
# AISBF - AI Service Broker Framework
## Overview
AISBF is a modular proxy server for managing multiple AI provider integrations. It provides a unified API interface for interacting with various AI services (Google, OpenAI, Anthropic, Ollama) with support for provider rotation and error tracking.
## Project Structure
```
geminiproxy/
├── aisbf/ # Main Python module
│ ├── __init__.py # Module initialization with exports
│ ├── config.py # Configuration management
│ ├── models.py # Pydantic models
│ ├── providers.py # Provider handlers
│ ├── handlers.py # Request handlers
│ ├── providers.json # Default provider configs (moved to config/)
│ └── rotations.json # Default rotation configs (moved to config/)
├── config/ # Configuration files directory
│ ├── providers.json # Default provider configurations
│ └── rotations.json # Default rotation configurations
├── main.py # FastAPI application entry point
├── setup.py # Installation script
├── start_proxy.sh # Development start script
├── aisbf.sh # Alternative start script
├── requirements.txt # Python dependencies
├── INSTALL.md # Installation guide
└── README.md # Project documentation
```
## Installation
### User Installation (no root required)
```bash
python setup.py install
```
Installs to:
- `~/.local/lib/python*/site-packages/aisbf/` - Package
- `~/.local/aisbf-venv/` - Virtual environment
- `~/.local/share/aisbf/` - Config files
- `~/.local/bin/aisbf` - Executable script
### System-wide Installation (requires root)
```bash
sudo python setup.py install
```
Installs to:
- `/usr/local/lib/python*/dist-packages/aisbf/` - Package
- `/usr/local/aisbf-venv/` - Virtual environment
- `/usr/local/share/aisbf/` - Config files
- `/usr/local/bin/aisbf` - Executable script
## Configuration Management
### Configuration File Locations
**Installed Configuration Files (read-only defaults):**
- User: `~/.local/share/aisbf/providers.json`, `~/.local/share/aisbf/rotations.json`
- System: `/usr/local/share/aisbf/providers.json`, `/usr/local/share/aisbf/rotations.json`
**User Configuration Files (writable):**
- `~/.aisbf/providers.json` - Provider configurations
- `~/.aisbf/rotations.json` - Rotation configurations
**Development Mode:**
- `config/providers.json` and `config/rotations.json` in source tree
### First Run Behavior
1. Checks for config files in installed location
2. Creates `~/.aisbf/` directory if needed
3. Copies default configs from installed location to `~/.aisbf/`
4. Loads configuration from `~/.aisbf/` on subsequent runs
## API Endpoints
### Root Endpoint
- `GET /` - Returns server status and list of available providers
### Chat Completions
- `POST /api/{provider_id}/chat/completions` - Handle chat completion requests
- Supports both streaming and non-streaming responses
- Provider ID can be a specific provider or rotation name
### Model List
- `GET /api/{provider_id}/models` - List available models for a provider or rotation
## Provider Support
AISBF supports the following AI providers:
### Google
- Uses google-genai SDK
- Requires API key
- Supports streaming and non-streaming responses
### OpenAI
- Uses openai SDK
- Requires API key
- Supports streaming and non-streaming responses
### Anthropic
- Uses anthropic SDK
- Requires API key
- Static model list (no dynamic model discovery)
### Ollama
- Uses direct HTTP API
- No API key required
- Local model hosting support
## Rotation Support
AISBF supports provider rotation with weighted model selection:
### Rotation Configuration
```json
{
"rotations": {
"my_rotation": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
### Rotation Behavior
- Weighted random selection of models
- Automatic failover between providers
- Error tracking and rate limiting
## Error Tracking and Rate Limiting
### Error Tracking
- Tracks failures per provider
- Disables providers after 3 consecutive failures
- 5-minute cooldown period for disabled providers
### Rate Limiting
- Automatic provider disabling when rate limited
- Graceful error handling
- Configurable retry behavior
## Development vs Production
### Development
Use `start_proxy.sh`:
- Creates local venv in `./venv/`
- Installs dependencies from `requirements.txt`
- Starts server with auto-reload enabled
- Uses `config/` directory for configuration
### Production
Install with `python setup.py install`:
- Creates isolated venv
- Installs all dependencies
- Provides `aisbf` command with daemon support
- Uses installed config files
## AISBF Script Commands
### Starting in Foreground (Default)
```bash
aisbf
```
Starts server in foreground with visible output.
### Starting as Daemon
```bash
aisbf daemon
```
- Starts in background
- Saves PID to `/tmp/aisbf.pid`
- Redirects output to `/dev/null`
- Prints PID of started process
### Checking Status
```bash
aisbf status
```
Checks if AISBF is running and reports status/PID.
### Stopping the Daemon
```bash
aisbf stop
```
Stops running daemon and removes PID file.
## Key Classes and Functions
### aisbf/config.py
- `Config` - Configuration management class
- `ProviderConfig` - Provider configuration model
- `RotationConfig` - Rotation configuration model
- `AppConfig` - Application configuration model
### aisbf/models.py
- `Message` - Chat message structure
- `ChatCompletionRequest` - Request model
- `ChatCompletionResponse` - Response model
- `Model` - Model information
- `Provider` - Provider information
- `ErrorTracking` - Error tracking data
### aisbf/providers.py
- `BaseProviderHandler` - Base provider handler class
- `GoogleProviderHandler` - Google provider implementation
- `OpenAIProviderHandler` - OpenAI provider implementation
- `AnthropicProviderHandler` - Anthropic provider implementation
- `OllamaProviderHandler` - Ollama provider implementation
- `get_provider_handler()` - Factory function for provider handlers
### aisbf/handlers.py
- `RequestHandler` - Request handling logic
- `RotationHandler` - Rotation handling logic
## Dependencies
Key dependencies from requirements.txt:
- fastapi - Web framework
- uvicorn - ASGI server
- pydantic - Data validation
- httpx - HTTP client
- google-genai - Google AI SDK
- openai - OpenAI SDK
- anthropic - Anthropic SDK
## Adding New Providers
### Steps to Add a New Provider
1. Create handler class in `aisbf/providers.py` inheriting from `BaseProviderHandler`
2. Add to `PROVIDER_HANDLERS` dictionary
3. Add provider configuration to `config/providers.json`
### Provider Handler Requirements
- Implement `handle_request()` method
- Implement `get_models()` method
- Handle error tracking and rate limiting
## Configuration Examples
### Provider Configuration
```json
{
"providers": {
"openai": {
"id": "openai",
"name": "OpenAI",
"endpoint": "https://api.openai.com/v1",
"type": "openai",
"api_key_required": true
}
}
}
```
### Rotation Configuration
```json
{
"rotations": {
"balanced": {
"providers": [
{
"provider_id": "openai",
"models": [
{"name": "gpt-4", "weight": 1},
{"name": "gpt-3.5-turbo", "weight": 3}
]
},
{
"provider_id": "anthropic",
"models": [
{"name": "claude-3-haiku-20240307", "weight": 2}
]
}
]
}
}
}
```
## Testing and Development
### Development Workflow
1. Use `start_proxy.sh` for development
2. Test with `start_proxy.sh` for development
3. Install with `python setup.py install` for production testing
4. Test all `aisbf` script commands (default, daemon, status, stop)
5. Verify configuration file locations and behavior
6. Test both user and system installations
### Common Development Tasks
- Adding new providers
- Modifying configuration
- Updating installation
- Testing error handling
## License
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>.
## Contributing
When making changes:
1. Update AI.PROMPT file with significant changes
2. Test all functionality
3. Update documentation as needed
4. Follow the project's coding conventions
5. Ensure all tests pass
## Support
For support and questions:
- Check the AI.PROMPT file for project-specific instructions
- Review the INSTALL.md file for installation details
- Check the README.md file for project overview
- Test with development scripts before production deployment
\ No newline at end of file
This diff is collapsed.
# AI Proxy Server
A unified proxy server for multiple AI providers with proper type handlers and library integration.
## Architecture
The proxy is organized into multiple modules for better maintainability:
- `main.py` - Entry point and FastAPI application setup
- `config.py` - Configuration management and provider loading
- `models.py` - Pydantic models for data structures
- `providers.py` - Provider type handlers with proper library integration
- `handlers.py` - Request/response handling logic
## Supported Providers
The proxy supports multiple AI providers with proper type handlers:
- **Google** - Uses `google-genai` library
- **OpenAI** - Uses `openai` library
- **Anthropic** - Uses `anthropic` library
- **Ollama** - Uses direct HTTP requests
## Installation
```bash
pip install -r requirements.txt
```
## Usage
Start the proxy server:
```bash
./start_proxy.sh
```
The server will start on `http://localhost:8000`
## API Endpoints
### Chat Completions
```http
POST /api/{provider_id}/chat/completions
```
**Request Body:**
```json
{
"model": "model_name",
"messages": [
{"role": "user", "content": "Hello!"}
],
"max_tokens": 100,
"temperature": 0.7,
"stream": false
}
```
### List Models
```http
GET /api/{provider_id}/models
```
## Configuration
Providers are configured in `providers.json` with proper type definitions:
```json
{
"providers": {
"openai": {
"id": "openai",
"name": "OpenAI",
"endpoint": "https://api.openai.com/v1",
"type": "openai",
"api_key_required": true
}
}
}
```
## Error Handling
The proxy includes robust error handling with:
- Rate limiting for failed requests
- Automatic retry with provider rotation
- Proper error tracking and logging
\ No newline at end of file
#!/bin/bash
########################################################
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
########################################################
# AISBF - AI Service Broker Framework || AI Should Be Free
# This script starts the AISBF proxy server using the installed virtual environment
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Check if venv exists
if [ ! -d "venv" ]; then
echo "Error: Virtual environment not found. Please run: python setup.py install"
exit 1
fi
# Activate virtual environment
source venv/bin/activate
# Start the proxy server
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
#!/bin/bash
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
# AISBF - AI Service Broker Framework || AI Should Be Free
# This script starts the AISBF proxy server using the installed virtual environment
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Check if venv exists
if [ ! -d "venv" ]; then
echo "Error: Virtual environment not found. Please run: python setup.py install"
exit 1
fi
# Activate virtual environment
source venv/bin/activate
# Start the proxy server
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
A modular proxy server for managing multiple AI provider integrations.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
A modular proxy server for managing multiple AI provider integrations.
"""
from .config import config, Config, ProviderConfig, RotationConfig, AppConfig
from .models import (
Message,
ChatCompletionRequest,
ChatCompletionResponse,
Model,
Provider,
ErrorTracking
)
from .providers import (
BaseProviderHandler,
GoogleProviderHandler,
OpenAIProviderHandler,
AnthropicProviderHandler,
OllamaProviderHandler,
get_provider_handler,
PROVIDER_HANDLERS
)
from .handlers import RequestHandler, RotationHandler
__version__ = "0.1.0"
__all__ = [
# Config
"config",
"Config",
"ProviderConfig",
"RotationConfig",
"AppConfig",
# Models
"Message",
"ChatCompletionRequest",
"ChatCompletionResponse",
"Model",
"Provider",
"ErrorTracking",
# Providers
"BaseProviderHandler",
"GoogleProviderHandler",
"OpenAIProviderHandler",
"AnthropicProviderHandler",
"OllamaProviderHandler",
"get_provider_handler",
"PROVIDER_HANDLERS",
# Handlers
"RequestHandler",
"RotationHandler",
]
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
A modular proxy server for managing multiple AI provider integrations.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
A modular proxy server for managing multiple AI provider integrations.
"""
from .config import config, Config, ProviderConfig, RotationConfig, AppConfig
from .models import (
Message,
ChatCompletionRequest,
ChatCompletionResponse,
Model,
Provider,
ErrorTracking
)
from .providers import (
BaseProviderHandler,
GoogleProviderHandler,
OpenAIProviderHandler,
AnthropicProviderHandler,
OllamaProviderHandler,
get_provider_handler,
PROVIDER_HANDLERS
)
from .handlers import RequestHandler, RotationHandler
__version__ = "0.1.0"
__all__ = [
# Config
"config",
"Config",
"ProviderConfig",
"RotationConfig",
"AppConfig",
# Models
"Message",
"ChatCompletionRequest",
"ChatCompletionResponse",
"Model",
"Provider",
"ErrorTracking",
# Providers
"BaseProviderHandler",
"GoogleProviderHandler",
"OpenAIProviderHandler",
"AnthropicProviderHandler",
"OllamaProviderHandler",
"get_provider_handler",
"PROVIDER_HANDLERS",
# Handlers
"RequestHandler",
"RotationHandler",
]
\ No newline at end of file
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Configuration management for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Configuration management for AISBF.
"""
from typing import Dict, List, Optional
from pydantic import BaseModel, Field
import json
import shutil
from pathlib import Path
class ProviderConfig(BaseModel):
id: str
name: str
endpoint: str
type: str
api_key_required: bool
class RotationConfig(BaseModel):
providers: List[Dict]
class AppConfig(BaseModel):
providers: Dict[str, ProviderConfig]
rotations: Dict[str, RotationConfig]
error_tracking: Dict[str, Dict]
class Config:
def __init__(self):
self._ensure_config_directory()
self._load_providers()
self._load_rotations()
self._initialize_error_tracking()
def _get_config_source_dir(self):
"""Get the directory containing default config files"""
# Try installed location first
installed_dirs = [
Path('/usr/local/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
if installed_dir.exists() and (installed_dir / 'providers.json').exists():
return installed_dir
# Fallback to source tree config directory
# This is for development mode
source_dir = Path(__file__).parent.parent.parent / 'config'
if source_dir.exists() and (source_dir / 'providers.json').exists():
return source_dir
# Last resort: try the old location in the package directory
package_dir = Path(__file__).parent
if (package_dir / 'providers.json').exists():
return package_dir
raise FileNotFoundError("Could not find configuration files")
def _ensure_config_directory(self):
"""Ensure ~/.aisbf/ directory exists and copy default config files if needed"""
config_dir = Path.home() / '.aisbf'
# Create config directory if it doesn't exist
config_dir.mkdir(exist_ok=True)
# Get the source directory for default config files
try:
source_dir = self._get_config_source_dir()
except FileNotFoundError:
print("Warning: Could not find default configuration files")
return
# Copy default config files if they don't exist
for config_file in ['providers.json', 'rotations.json']:
src = source_dir / config_file
dst = config_dir / config_file
if not dst.exists() and src.exists():
shutil.copy2(src, dst)
print(f"Created default config file: {dst}")
def _load_providers(self):
providers_path = Path.home() / '.aisbf' / 'providers.json'
if not providers_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
providers_path = source_dir / 'providers.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find providers.json configuration file")
with open(providers_path) as f:
data = json.load(f)
self.providers = {k: ProviderConfig(**v) for k, v in data['providers'].items()}
def _load_rotations(self):
rotations_path = Path.home() / '.aisbf' / 'rotations.json'
if not rotations_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
rotations_path = source_dir / 'rotations.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find rotations.json configuration file")
with open(rotations_path) as f:
data = json.load(f)
self.rotations = {k: RotationConfig(**v) for k, v in data['rotations'].items()}
def _initialize_error_tracking(self):
self.error_tracking = {}
for provider_id in self.providers:
self.error_tracking[provider_id] = {
'failures': 0,
'last_failure': None,
'disabled_until': None
}
def get_provider(self, provider_id: str) -> ProviderConfig:
return self.providers.get(provider_id)
def get_rotation(self, rotation_id: str) -> RotationConfig:
return self.rotations.get(rotation_id)
config = Config()
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Configuration management for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Configuration management for AISBF.
"""
from typing import Dict, List, Optional
from pydantic import BaseModel, Field
import json
import shutil
from pathlib import Path
class ProviderConfig(BaseModel):
id: str
name: str
endpoint: str
type: str
api_key_required: bool
class RotationConfig(BaseModel):
providers: List[Dict]
class AppConfig(BaseModel):
providers: Dict[str, ProviderConfig]
rotations: Dict[str, RotationConfig]
error_tracking: Dict[str, Dict]
class Config:
def __init__(self):
self._ensure_config_directory()
self._load_providers()
self._load_rotations()
self._initialize_error_tracking()
def _get_config_source_dir(self):
"""Get the directory containing default config files"""
# Try installed location first
installed_dirs = [
Path('/usr/local/share/aisbf'),
Path.home() / '.local' / 'share' / 'aisbf',
]
for installed_dir in installed_dirs:
if installed_dir.exists() and (installed_dir / 'providers.json').exists():
return installed_dir
# Fallback to source tree config directory
# This is for development mode
source_dir = Path(__file__).parent.parent.parent / 'config'
if source_dir.exists() and (source_dir / 'providers.json').exists():
return source_dir
# Last resort: try the old location in the package directory
package_dir = Path(__file__).parent
if (package_dir / 'providers.json').exists():
return package_dir
raise FileNotFoundError("Could not find configuration files")
def _ensure_config_directory(self):
"""Ensure ~/.aisbf/ directory exists and copy default config files if needed"""
config_dir = Path.home() / '.aisbf'
# Create config directory if it doesn't exist
config_dir.mkdir(exist_ok=True)
# Get the source directory for default config files
try:
source_dir = self._get_config_source_dir()
except FileNotFoundError:
print("Warning: Could not find default configuration files")
return
# Copy default config files if they don't exist
for config_file in ['providers.json', 'rotations.json']:
src = source_dir / config_file
dst = config_dir / config_file
if not dst.exists() and src.exists():
shutil.copy2(src, dst)
print(f"Created default config file: {dst}")
def _load_providers(self):
providers_path = Path.home() / '.aisbf' / 'providers.json'
if not providers_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
providers_path = source_dir / 'providers.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find providers.json configuration file")
with open(providers_path) as f:
data = json.load(f)
self.providers = {k: ProviderConfig(**v) for k, v in data['providers'].items()}
def _load_rotations(self):
rotations_path = Path.home() / '.aisbf' / 'rotations.json'
if not rotations_path.exists():
# Fallback to source config if user config doesn't exist
try:
source_dir = self._get_config_source_dir()
rotations_path = source_dir / 'rotations.json'
except FileNotFoundError:
raise FileNotFoundError("Could not find rotations.json configuration file")
with open(rotations_path) as f:
data = json.load(f)
self.rotations = {k: RotationConfig(**v) for k, v in data['rotations'].items()}
def _initialize_error_tracking(self):
self.error_tracking = {}
for provider_id in self.providers:
self.error_tracking[provider_id] = {
'failures': 0,
'last_failure': None,
'disabled_until': None
}
def get_provider(self, provider_id: str) -> ProviderConfig:
return self.providers.get(provider_id)
def get_rotation(self, rotation_id: str) -> RotationConfig:
return self.rotations.get(rotation_id)
config = Config()
\ No newline at end of file
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Request handlers for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Request handlers for AISBF.
"""
import asyncio
from typing import Dict, List, Optional
from fastapi import HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
from .models import ChatCompletionRequest, ChatCompletionResponse
from .providers import get_provider_handler
from .config import config
class RequestHandler:
def __init__(self):
self.config = config
async def handle_chat_completion(self, request: Request, provider_id: str, request_data: Dict) -> Dict:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_streaming_chat_completion(self, request: Request, provider_id: str, request_data: Dict):
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
async def stream_generator():
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=True
)
for chunk in response:
yield f"data: {chunk}\n\n".encode('utf-8')
handler.record_success()
except Exception as e:
handler.record_failure()
yield f"data: {str(e)}\n\n".encode('utf-8')
return StreamingResponse(stream_generator(), media_type="text/event-stream")
async def handle_model_list(self, request: Request, provider_id: str) -> List[Dict]:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
try:
models = await handler.get_models()
return [model.dict() for model in models]
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
class RotationHandler:
def __init__(self):
self.config = config
async def handle_rotation_request(self, rotation_id: str, request_data: Dict) -> Dict:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
providers = rotation_config.providers
weighted_models = []
for provider in providers:
for model in provider['models']:
weighted_models.extend([model] * model['weight'])
if not weighted_models:
raise HTTPException(status_code=400, detail="No models available in rotation")
import random
selected_model = random.choice(weighted_models)
provider_id = selected_model['provider_id']
api_key = selected_model.get('api_key')
model_name = selected_model['name']
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="All providers temporarily unavailable")
try:
response = await handler.handle_request(
model=model_name,
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_rotation_model_list(self, rotation_id: str) -> List[Dict]:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
all_models = []
for provider in rotation_config.providers:
for model in provider['models']:
all_models.append({
"id": f"{provider['provider_id']}/{model['name']}",
"name": model['name'],
"provider_id": provider['provider_id'],
"weight": model['weight']
})
return all_models
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Request handlers for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Request handlers for AISBF.
"""
import asyncio
from typing import Dict, List, Optional
from fastapi import HTTPException, Request
from fastapi.responses import JSONResponse, StreamingResponse
from .models import ChatCompletionRequest, ChatCompletionResponse
from .providers import get_provider_handler
from .config import config
class RequestHandler:
def __init__(self):
self.config = config
async def handle_chat_completion(self, request: Request, provider_id: str, request_data: Dict) -> Dict:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_streaming_chat_completion(self, request: Request, provider_id: str, request_data: Dict):
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request_data.get('api_key') or request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="Provider temporarily unavailable")
async def stream_generator():
try:
response = await handler.handle_request(
model=request_data['model'],
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=True
)
for chunk in response:
yield f"data: {chunk}\n\n".encode('utf-8')
handler.record_success()
except Exception as e:
handler.record_failure()
yield f"data: {str(e)}\n\n".encode('utf-8')
return StreamingResponse(stream_generator(), media_type="text/event-stream")
async def handle_model_list(self, request: Request, provider_id: str) -> List[Dict]:
provider_config = self.config.get_provider(provider_id)
if provider_config.api_key_required:
api_key = request.headers.get('Authorization', '').replace('Bearer ', '')
if not api_key:
raise HTTPException(status_code=401, detail="API key required")
else:
api_key = None
handler = get_provider_handler(provider_id, api_key)
try:
models = await handler.get_models()
return [model.dict() for model in models]
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
class RotationHandler:
def __init__(self):
self.config = config
async def handle_rotation_request(self, rotation_id: str, request_data: Dict) -> Dict:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
providers = rotation_config.providers
weighted_models = []
for provider in providers:
for model in provider['models']:
weighted_models.extend([model] * model['weight'])
if not weighted_models:
raise HTTPException(status_code=400, detail="No models available in rotation")
import random
selected_model = random.choice(weighted_models)
provider_id = selected_model['provider_id']
api_key = selected_model.get('api_key')
model_name = selected_model['name']
handler = get_provider_handler(provider_id, api_key)
if handler.is_rate_limited():
raise HTTPException(status_code=503, detail="All providers temporarily unavailable")
try:
response = await handler.handle_request(
model=model_name,
messages=request_data['messages'],
max_tokens=request_data.get('max_tokens'),
temperature=request_data.get('temperature', 1.0),
stream=request_data.get('stream', False)
)
handler.record_success()
return response
except Exception as e:
handler.record_failure()
raise HTTPException(status_code=500, detail=str(e))
async def handle_rotation_model_list(self, rotation_id: str) -> List[Dict]:
rotation_config = self.config.get_rotation(rotation_id)
if not rotation_config:
raise HTTPException(status_code=400, detail=f"Rotation {rotation_id} not found")
all_models = []
for provider in rotation_config.providers:
for model in provider['models']:
all_models.append({
"id": f"{provider['provider_id']}/{model['name']}",
"name": model['name'],
"provider_id": provider['provider_id'],
"weight": model['weight']
})
return all_models
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
A modular proxy server for managing multiple AI provider integrations.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
A modular proxy server for managing multiple AI provider integrations.
"""
from pydantic import BaseModel, Field
from typing import Dict, List, Optional, Union
class Message(BaseModel):
role: str
content: Union[str, List[Dict]]
class ChatCompletionRequest(BaseModel):
model: str
messages: List[Message]
max_tokens: Optional[int] = None
temperature: Optional[float] = 1.0
stream: Optional[bool] = False
class ChatCompletionResponse(BaseModel):
id: str
object: str
created: int
model: str
choices: List[Dict]
usage: Optional[Dict] = None
stream: Optional[bool] = False
class Model(BaseModel):
id: str
name: str
provider_id: str
weight: int = 1
class Provider(BaseModel):
id: str
name: str
type: str
endpoint: str
api_key_required: bool
models: List[Model] = []
class ErrorTracking(BaseModel):
failures: int
last_failure: Optional[int]
disabled_until: Optional[int]
from pydantic import BaseModel, Field
from typing import Dict, List, Optional, Union
class Message(BaseModel):
role: str
content: Union[str, List[Dict]]
class ChatCompletionRequest(BaseModel):
model: str
messages: List[Message]
max_tokens: Optional[int] = None
temperature: Optional[float] = 1.0
stream: Optional[bool] = False
class ChatCompletionResponse(BaseModel):
id: str
object: str
created: int
model: str
choices: List[Dict]
usage: Optional[Dict] = None
stream: Optional[bool] = False
class Model(BaseModel):
id: str
name: str
provider_id: str
weight: int = 1
class Provider(BaseModel):
id: str
name: str
type: str
endpoint: str
api_key_required: bool
models: List[Model] = []
class ErrorTracking(BaseModel):
failures: int
last_failure: Optional[int]
disabled_until: Optional[int]
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
{
"rotations": {
"coding": {
"model_name": "coding",
"providers": [
{
"provider_id": "gemini",
"api_key": "YOUR_GEMINI_API_KEY",
"models": [
{
"name": "gemini-2.0-flash",
"weight": 3
},
{
"name": "gemini-1.5-pro",
"weight": 1
}
]
},
{
"provider_id": "openai",
"api_key": "YOUR_OPENAI_API_KEY",
"models": [
{
"name": "gpt-4",
"weight": 2
},
{
"name": "gpt-3.5-turbo",
"weight": 1
}
]
},
{
"provider_id": "anthropic",
"api_key": "YOUR_ANTHROPIC_API_KEY",
"models": [
{
"name": "claude-3-5-sonnet-20241022",
"weight": 2
},
{
"name": "claude-3-haiku-20240307",
"weight": 1
}
]
}
]
},
"general": {
"model_name": "general",
"providers": [
{
"provider_id": "gemini",
"api_key": "YOUR_GEMINI_API_KEY",
"models": [
{
"name": "gemini-1.5-pro",
"weight": 2
},
{
"name": "gemini-2.0-flash",
"weight": 1
}
]
},
{
"provider_id": "openai",
"api_key": "YOUR_OPENAI_API_KEY",
"models": [
{
"name": "gpt-4",
"weight": 2
},
{
"name": "gpt-3.5-turbo",
"weight": 1
}
]
}
]
}
}
}
\ No newline at end of file
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Main application for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Main application for AISBF.
"""
from fastapi import FastAPI, HTTPException, Request, status
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from aisbf.models import ChatCompletionRequest, ChatCompletionResponse
from aisbf.handlers import RequestHandler, RotationHandler
from aisbf.config import config
import time
import logging
from datetime import datetime, timedelta
from collections import defaultdict
# Configure logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
# Initialize handlers
request_handler = RequestHandler()
rotation_handler = RotationHandler()
app = FastAPI(title="AI Proxy Server")
# CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.get("/")
async def root():
return {"message": "AI Proxy Server is running", "providers": list(config.providers.keys())}
@app.post("/api/{provider_id}/chat/completions")
async def chat_completions(provider_id: str, request: Request, body: ChatCompletionRequest):
logger.debug(f"Received chat_completions request for provider: {provider_id}")
logger.debug(f"Request headers: {dict(request.headers)}")
logger.debug(f"Request body: {body}")
body_dict = body.model_dump()
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation request")
return await rotation_handler.handle_rotation_request(provider_id, body_dict)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
logger.debug(f"Provider config: {provider_config}")
try:
if body.stream:
logger.debug("Handling streaming chat completion")
return await request_handler.handle_streaming_chat_completion(request, provider_id, body_dict)
else:
logger.debug("Handling non-streaming chat completion")
result = await request_handler.handle_chat_completion(request, provider_id, body_dict)
logger.debug(f"Response result: {result}")
return result
except Exception as e:
logger.error(f"Error handling chat_completions: {str(e)}", exc_info=True)
raise
@app.get("/api/{provider_id}/models")
async def list_models(request: Request, provider_id: str):
logger.debug(f"Received list_models request for provider: {provider_id}")
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation model list request")
return await rotation_handler.handle_rotation_model_list(provider_id)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
try:
logger.debug("Handling model list request")
result = await request_handler.handle_model_list(request, provider_id)
logger.debug(f"Models result: {result}")
return result
except Exception as e:
logger.error(f"Error handling list_models: {str(e)}", exc_info=True)
raise
def main():
"""Main entry point for the AISBF server"""
import uvicorn
logger.info("Starting AI Proxy Server on http://localhost:8000")
uvicorn.run(app, host="0.0.0.0", port=8000)
if __name__ == "__main__":
main()
"""
Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Main application for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
Main application for AISBF.
"""
from fastapi import FastAPI, HTTPException, Request, status
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from aisbf.models import ChatCompletionRequest, ChatCompletionResponse
from aisbf.handlers import RequestHandler, RotationHandler
from aisbf.config import config
import time
import logging
from datetime import datetime, timedelta
from collections import defaultdict
# Configure logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
# Initialize handlers
request_handler = RequestHandler()
rotation_handler = RotationHandler()
app = FastAPI(title="AI Proxy Server")
# CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.get("/")
async def root():
return {"message": "AI Proxy Server is running", "providers": list(config.providers.keys())}
@app.post("/api/{provider_id}/chat/completions")
async def chat_completions(provider_id: str, request: Request, body: ChatCompletionRequest):
logger.debug(f"Received chat_completions request for provider: {provider_id}")
logger.debug(f"Request headers: {dict(request.headers)}")
logger.debug(f"Request body: {body}")
body_dict = body.model_dump()
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation request")
return await rotation_handler.handle_rotation_request(provider_id, body_dict)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
logger.debug(f"Provider config: {provider_config}")
try:
if body.stream:
logger.debug("Handling streaming chat completion")
return await request_handler.handle_streaming_chat_completion(request, provider_id, body_dict)
else:
logger.debug("Handling non-streaming chat completion")
result = await request_handler.handle_chat_completion(request, provider_id, body_dict)
logger.debug(f"Response result: {result}")
return result
except Exception as e:
logger.error(f"Error handling chat_completions: {str(e)}", exc_info=True)
raise
@app.get("/api/{provider_id}/models")
async def list_models(request: Request, provider_id: str):
logger.debug(f"Received list_models request for provider: {provider_id}")
# Check if it's a rotation
if provider_id in config.rotations:
logger.debug("Handling rotation model list request")
return await rotation_handler.handle_rotation_model_list(provider_id)
# Check if it's a provider
if provider_id not in config.providers:
logger.error(f"Provider {provider_id} not found")
raise HTTPException(status_code=400, detail=f"Provider {provider_id} not found")
provider_config = config.get_provider(provider_id)
try:
logger.debug("Handling model list request")
result = await request_handler.handle_model_list(request, provider_id)
logger.debug(f"Models result: {result}")
return result
except Exception as e:
logger.error(f"Error handling list_models: {str(e)}", exc_info=True)
raise
def main():
"""Main entry point for the AISBF server"""
import uvicorn
logger.info("Starting AI Proxy Server on http://localhost:8000")
uvicorn.run(app, host="0.0.0.0", port=8000)
if __name__ == "__main__":
main()
\ No newline at end of file
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Setup configuration for AISBF.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Why did the programmer quit his job? Because he didn't get arrays!
"""
from setuptools import setup, find_packages, Command
from setuptools.command.install import install as _install
from pathlib import Path
import os
import shutil
import sys
import subprocess
# Read the contents of README file
this_directory = Path(__file__).parent
long_description = (this_directory / "README.md").read_text() if (this_directory / "README.md").exists() else ""
# Read requirements
requirements = []
if (this_directory / "requirements.txt").exists():
with open(this_directory / "requirements.txt") as f:
requirements = [line.strip() for line in f if line.strip() and not line.startswith("#")]
class InstallCommand(_install):
"""Custom install command that also installs the aisbf script and creates venv"""
def initialize_options(self):
_install.initialize_options(self)
# Check if running as non-root without --user flag
if os.geteuid() != 0 and '--user' not in sys.argv:
print("Installing as non-root user. Adding --user flag for user-local installation.")
self.user = True
def run(self):
# Run the standard install
_install.run(self)
# Install config files
self._install_config_files()
# Create venv and install requirements
self._create_venv_and_install_requirements()
# Install the aisbf script
self._install_aisbf_script()
def _create_venv_and_install_requirements(self):
"""Create a virtual environment and install requirements"""
# Determine installation directory
if '--user' in sys.argv or os.geteuid() != 0:
# User installation - use ~/.local
install_dir = Path.home() / '.local'
else:
# System installation - use /usr/local
install_dir = Path('/usr/local')
venv_dir = install_dir / 'aisbf-venv'
# Create venv if it doesn't exist
if not venv_dir.exists():
print(f"Creating virtual environment at {venv_dir}")
subprocess.run([sys.executable, '-m', 'venv', str(venv_dir)], check=True)
else:
print(f"Virtual environment already exists at {venv_dir}")
# Install requirements in the venv
pip_path = venv_dir / 'bin' / 'pip'
requirements_path = this_directory / 'requirements.txt'
if requirements_path.exists():
print(f"Installing requirements from {requirements_path}")
subprocess.run([str(pip_path), 'install', '-r', str(requirements_path)], check=True)
else:
print("No requirements.txt found, skipping dependency installation")
def _install_config_files(self):
"""Install config files to the appropriate share directory"""
# Determine the share directory
if '--user' in sys.argv or os.geteuid() != 0:
# User installation - use ~/.local/share
share_dir = Path.home() / '.local' / 'share' / 'aisbf'
else:
# System installation - use /usr/local/share
share_dir = Path('/usr/local/share/aisbf')
# Create the share directory if it doesn't exist
share_dir.mkdir(parents=True, exist_ok=True)
# Copy config files
config_dir = this_directory / 'config'
if config_dir.exists():
for config_file in config_dir.glob('*.json'):
dst = share_dir / config_file.name
shutil.copy2(config_file, dst)
print(f"Installed config file {config_file.name} to {dst}")
else:
print(f"Warning: Config directory {config_dir} not found")
def _install_aisbf_script(self):
"""Install the aisbf script that uses the venv"""
# Determine the installation directory
if '--user' in sys.argv or os.geteuid() != 0:
# User installation - use ~/.local/bin
bin_dir = Path.home() / '.local' / 'bin'
venv_dir = Path.home() / '.local' / 'aisbf-venv'
else:
# System installation - use /usr/local/bin
bin_dir = Path('/usr/local/bin')
venv_dir = Path('/usr/local') / 'aisbf-venv'
# Create the bin directory if it doesn't exist
bin_dir.mkdir(parents=True, exist_ok=True)
# Create the aisbf script that uses the venv
script_content = f"""#!/bin/bash
# AISBF - AI Service Broker Framework || AI Should Be Free
# This script manages the AISBF server using the installed virtual environment
PIDFILE="/tmp/aisbf.pid"
VENV_DIR="{venv_dir}"
# Function to start the server
start_server() {{
# Activate the virtual environment
source $VENV_DIR/bin/activate
# Start the proxy server
uvicorn main:app --host 0.0.0.0 --port 8000
}}
# Function to start as daemon
start_daemon() {{
# Check if already running
if [ -f "$PIDFILE" ]; then
PID=$(cat "$PIDFILE")
if ps -p "$PID" > /dev/null 2>&1; then
echo "AISBF is already running (PID: $PID)"
exit 1
else
# Stale PID file, remove it
rm -f "$PIDFILE"
fi
fi
# Start in background with nohup
nohup bash -c "source $VENV_DIR/bin/activate && uvicorn main:app --host 0.0.0.0 --port 8000" > /dev/null 2>&1 &
PID=$!
echo $PID > "$PIDFILE"
echo "AISBF started in background (PID: $PID)"
}}
# Function to check status
check_status() {{
if [ -f "$PIDFILE" ]; then
PID=$(cat "$PIDFILE")
if ps -p "$PID" > /dev/null 2>&1; then
echo "AISBF is running (PID: $PID)"
exit 0
else
echo "AISBF is not running (stale PID file)"
rm -f "$PIDFILE"
exit 1
fi
else
echo "AISBF is not running"
exit 1
fi
}}
# Function to stop the daemon
stop_daemon() {{
if [ -f "$PIDFILE" ]; then
PID=$(cat "$PIDFILE")
if ps -p "$PID" > /dev/null 2>&1; then
kill "$PID"
rm -f "$PIDFILE"
echo "AISBF stopped (PID: $PID)"
else
echo "AISBF is not running (stale PID file)"
rm -f "$PIDFILE"
fi
else
echo "AISBF is not running"
fi
}}
# Main command handling
case "$1" in
daemon)
start_daemon
;;
status)
check_status
;;
stop)
stop_daemon
;;
*)
# Default: start in foreground
start_server
;;
esac
"""
dst = bin_dir / 'aisbf'
with open(dst, 'w') as f:
f.write(script_content)
os.chmod(dst, 0o755)
print(f"Installed 'aisbf' script to {dst}")
setup(
name="aisbf",
version="0.1.0",
author="AISBF Contributors",
author_email="",
description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/yourusername/aisbf",
packages=find_packages(),
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Topic :: Software Development :: Libraries :: Python Modules",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
],
python_requires=">=3.8",
install_requires=requirements,
include_package_data=True,
package_data={
"aisbf": ["*.json"],
},
entry_points={
"console_scripts": [
"aisbf=main:main",
],
},
cmdclass={
'install': InstallCommand,
},
)
\ No newline at end of file
#!/bin/bash
###########################################################
# Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
##########################################################
# AISBF - AI Service Broker Framework || AI Should Be Free
# DEVELOPMENT START SCRIPT - For development use only
# For production use, install with: python setup.py install
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Create virtual environment if it doesn't exist
if [ ! -d "venv" ]; then
echo "Creating virtual environment for development..."
python3 -m venv venv
fi
# Activate virtual environment
source venv/bin/activate
# Install dependencies
echo "Installing dependencies..."
pip install -r requirements.txt
# Start the proxy server
echo "Starting AISBF development server..."
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
#!/bin/bash
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# Why did the programmer quit his job? Because he didn't get arrays!
# AISBF - AI Service Broker Framework || AI Should Be Free
# DEVELOPMENT START SCRIPT - For development use only
# For production use, install with: python setup.py install
# Get the directory where this script is located
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Change to the script directory
cd "$SCRIPT_DIR"
# Create virtual environment if it doesn't exist
if [ ! -d "venv" ]; then
echo "Creating virtual environment for development..."
python3 -m venv venv
fi
# Activate virtual environment
source venv/bin/activate
# Install dependencies
echo "Installing dependencies..."
pip install -r requirements.txt
# Start the proxy server
echo "Starting AISBF development server..."
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment