Commit e02ed7fc authored by Your Name's avatar Your Name

feat: Add NSFW/privacy content filtering and semantic classification

- Add NSFW/privacy boolean fields to models (providers.json, rotations.json, autoselect.json)
- Implement content classification using last 3 messages for performance
- Add semantic classification with hybrid BM25 + sentence-transformer re-ranking
- Update autoselect handler to support classify_semantic flag
- Add new semantic_classifier.py module with hybrid search capabilities
- Update dashboard templates to manage new configuration fields
- Update documentation (README.md, DOCUMENTATION.md) with new features
- Bump version to 0.6.0 in pyproject.toml and setup.py
- Add new dependencies: sentence-transformers, rank-bm25
- Update package configuration for PyPI distribution
parent b2151583
...@@ -415,6 +415,83 @@ Autoselect models use AI to analyze the user's request and select the best model ...@@ -415,6 +415,83 @@ Autoselect models use AI to analyze the user's request and select the best model
3. **Automatic Routing**: Routes the request to the most suitable model 3. **Automatic Routing**: Routes the request to the most suitable model
4. **Fallback**: Uses a default model if selection fails or is uncertain 4. **Fallback**: Uses a default model if selection fails or is uncertain
### Semantic Classification (New Feature)
Autoselect configurations can now use semantic classification instead of AI model selection for faster and potentially more accurate model selection:
- **Hybrid BM25 + Semantic Search**: Combines fast keyword matching with semantic similarity using sentence transformers
- **Automatic Model Library Indexing**: Builds and caches model embeddings for efficient similarity matching
- **Performance Optimized**: No API calls required, significantly faster than AI-based selection
- **Fallback Support**: Falls back to AI model selection if semantic classification fails
**Enable Semantic Classification:**
```json
{
"autoselect": {
"smart": {
"classify_semantic": true,
"selection_model": "openai/gpt-4",
"available_models": [...]
}
}
}
```
**Benefits:**
- Faster model selection (no API costs)
- Deterministic results based on semantic similarity
- Automatic model library caching in `~/.aisbf/`
- Lower latency for model selection
### Content Classification and Filtering (New Feature)
AISBF provides advanced content classification for NSFW and privacy-sensitive content:
#### NSFW/Privacy Content Filtering
Models can be configured with boolean flags to indicate their suitability for sensitive content:
- **`nsfw`**: Model supports NSFW (Not Safe For Work) content
- **`privacy`**: Model supports privacy-sensitive content (medical, financial, legal data)
When global `classify_nsfw` or `classify_privacy` is enabled, AISBF analyzes the last 3 user messages and routes requests only to appropriate models. If no suitable models are available, requests return a 404 error.
**Configuration Levels (cascading priority):**
1. Rotation model config (highest priority)
2. Provider model config
3. Global AISBF config (enables/disables classification)
**Example Model Configuration:**
```json
{
"models": [
{
"name": "gpt-4",
"nsfw": true,
"privacy": true
},
{
"name": "gpt-3.5-turbo",
"nsfw": false,
"privacy": true
}
]
}
```
**Global Classification Settings:**
```json
{
"classify_nsfw": true,
"classify_privacy": true
}
```
**Content Classification Window:**
- Only analyzes the last 3 user messages for performance
- Avoids processing full conversation history
- Uses HuggingFace transformers for content analysis
- Automatic fallback if classification models are unavailable
### Autoselect Configuration ### Autoselect Configuration
```json ```json
......
...@@ -21,6 +21,8 @@ Access the dashboard at `http://localhost:17765/dashboard` (default credentials: ...@@ -21,6 +21,8 @@ Access the dashboard at `http://localhost:17765/dashboard` (default credentials:
- **Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Ollama, and Kiro (Amazon Q Developer) - **Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Ollama, and Kiro (Amazon Q Developer)
- **Rotation Models**: Weighted load balancing across multiple providers with automatic failover - **Rotation Models**: Weighted load balancing across multiple providers with automatic failover
- **Autoselect Models**: AI-powered model selection based on content analysis and request characteristics - **Autoselect Models**: AI-powered model selection based on content analysis and request characteristics
- **Semantic Classification**: Fast hybrid BM25 + semantic model selection using sentence transformers (optional)
- **Content Classification**: NSFW/privacy content filtering with configurable classification windows
- **Streaming Support**: Full support for streaming responses from all providers - **Streaming Support**: Full support for streaming responses from all providers
- **Error Tracking**: Automatic provider disabling after consecutive failures with cooldown periods - **Error Tracking**: Automatic provider disabling after consecutive failures with cooldown periods
- **Rate Limiting**: Built-in rate limiting and graceful error handling - **Rate Limiting**: Built-in rate limiting and graceful error handling
...@@ -287,6 +289,53 @@ When context exceeds the configured percentage of `context_size`, the system aut ...@@ -287,6 +289,53 @@ When context exceeds the configured percentage of `context_size`, the system aut
See `config/providers.json` and `config/rotations.json` for configuration examples. See `config/providers.json` and `config/rotations.json` for configuration examples.
### Content Classification and Semantic Selection
AISBF provides advanced content filtering and intelligent model selection based on content analysis:
#### NSFW/Privacy Content Filtering
Models can be configured with `nsfw` and `privacy` boolean flags to indicate their suitability for sensitive content:
- **`nsfw`**: Model supports NSFW (Not Safe For Work) content
- **`privacy`**: Model supports privacy-sensitive content (e.g., medical, financial, legal data)
When global `classify_nsfw` or `classify_privacy` is enabled, AISBF automatically analyzes the last 3 user messages to classify content and routes requests only to appropriate models. If no suitable models are available, the request returns a 404 error.
**Configuration:**
- Provider models: Set in `config/providers.json`
- Rotation models: Override in `config/rotations.json`
- Global settings: Enable/disable in `config/aisbf.json` or dashboard
#### Semantic Model Selection
For enhanced performance, autoselect configurations can use semantic classification instead of AI model selection:
- **Hybrid BM25 + Semantic Search**: Combines fast keyword matching with semantic similarity
- **Sentence Transformers**: Uses pre-trained embeddings for content understanding
- **Automatic Fallback**: Falls back to AI model selection if semantic classification fails
**Enable Semantic Classification:**
```json
{
"autoselect": {
"autoselect": {
"classify_semantic": true,
"selection_model": "openai/gpt-4",
"available_models": [...]
}
}
}
```
**Benefits:**
- Faster model selection (no API calls required)
- Lower costs (no tokens consumed for selection)
- Deterministic results based on content similarity
- Automatic model library indexing and caching
See `config/autoselect.json` for configuration examples.
## API Endpoints ## API Endpoints
### Three Proxy Paths ### Three Proxy Paths
......
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Content classification for NSFW and privacy detection.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Content classifier for NSFW and privacy detection.
"""
import logging
import threading
from typing import Optional, Tuple
class ContentClassifier:
"""
Content classifier for NSFW and privacy detection.
Uses HuggingFace transformers for classification.
"""
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
if self._initialized:
return
self._initialized = True
self._nsfw_classifier = None
self._privacy_classifier = None
self._classifier_lock = threading.Lock()
self._nsfw_model_name = None
self._privacy_model_name = None
self.logger = logging.getLogger(__name__)
def initialize(self, nsfw_model_name: Optional[str] = None, privacy_model_name: Optional[str] = None):
"""
Initialize the classifiers with the specified model names.
Args:
nsfw_model_name: HuggingFace model name for NSFW classification
privacy_model_name: HuggingFace model name for privacy classification
"""
self._nsfw_model_name = nsfw_model_name
self._privacy_model_name = privacy_model_name
if nsfw_model_name:
self._load_nsfw_classifier(nsfw_model_name)
if privacy_model_name:
self._load_privacy_classifier(privacy_model_name)
def _load_nsfw_classifier(self, model_name: str):
"""Load the NSFW classifier model"""
try:
from transformers import pipeline
self.logger.info(f"Loading NSFW classifier model: {model_name}")
self._nsfw_classifier = pipeline("text-classification", model=model_name)
self.logger.info("NSFW classifier loaded successfully")
except Exception as e:
self.logger.error(f"Failed to load NSFW classifier: {e}")
self._nsfw_classifier = None
def _load_privacy_classifier(self, model_name: str):
"""Load the privacy classifier model"""
try:
from transformers import pipeline
self.logger.info(f"Loading privacy classifier model: {model_name}")
self._privacy_classifier = pipeline("text-classification", model=model_name)
self.logger.info("Privacy classifier loaded successfully")
except Exception as e:
self.logger.error(f"Failed to load privacy classifier: {e}")
self._privacy_classifier = None
def check_nsfw(self, text: str, threshold: float = 0.8) -> Tuple[bool, str]:
"""
Check if the given text contains NSFW content.
Args:
text: The text to check
threshold: Confidence threshold for NSFW detection (default: 0.8)
Returns:
Tuple of (is_safe, message)
- is_safe: True if content is safe, False if NSFW
- message: Description of the result
"""
if self._nsfw_classifier is None:
self.logger.warning("NSFW classifier not initialized, allowing content")
return True, "NSFW classifier not available"
try:
# Truncate to first 512 characters for classification (avoid huge inputs)
# This is enough to detect the intent/content type
text_to_check = text[:512] if len(text) > 512 else text
result = self._nsfw_classifier(text_to_check)[0]
self.logger.debug(f"NSFW classification result: {result}")
if result['label'] == 'NSFW' and result['score'] > threshold:
return False, f"Content classified as NSFW (confidence: {result['score']:.2f})"
return True, "Content is safe"
except Exception as e:
self.logger.error(f"Error during NSFW classification: {e}")
# Default to safe on error
return True, f"Error during classification: {str(e)}"
def check_privacy(self, text: str, threshold: float = 0.8) -> Tuple[bool, str]:
"""
Check if the given text contains privacy-sensitive information.
Args:
text: The text to check
threshold: Confidence threshold for privacy detection (default: 0.8)
Returns:
Tuple of (is_safe, message)
- is_safe: True if content is safe, False if contains sensitive info
- message: Description of the result
"""
if self._privacy_classifier is None:
self.logger.warning("Privacy classifier not initialized, allowing content")
return True, "Privacy classifier not available"
try:
# Truncate to first 512 characters for classification (avoid huge inputs)
# This is enough to detect personal/sensitive information
text_to_check = text[:512] if len(text) > 512 else text
result = self._privacy_classifier(text_to_check)[0]
self.logger.debug(f"Privacy classification result: {result}")
# Common labels for sensitive/personal information
sensitive_labels = ['personal', 'pii', 'sensitive', 'private', 'nlp/privacy']
if result['label'].lower() in [l.lower() for l in sensitive_labels] and result['score'] > threshold:
return False, f"Content contains privacy-sensitive information (confidence: {result['score']:.2f})"
return True, "Content is safe"
except Exception as e:
self.logger.error(f"Error during privacy classification: {e}")
# Default to safe on error
return True, f"Error during classification: {str(e)}"
def check_content(self, text: str, check_nsfw: bool = True, check_privacy: bool = True,
threshold: float = 0.8) -> Tuple[bool, str]:
"""
Check content for both NSFW and privacy concerns.
Args:
text: The text to check
check_nsfw: Whether to check for NSFW content
check_privacy: Whether to check for privacy-sensitive content
threshold: Confidence threshold for detection
Returns:
Tuple of (is_safe, message)
"""
if check_nsfw:
is_safe, message = self.check_nsfw(text, threshold)
if not is_safe:
return False, f"NSFW: {message}"
if check_privacy:
is_safe, message = self.check_privacy(text, threshold)
if not is_safe:
return False, f"Privacy: {message}"
return True, "All content is safe"
# Global classifier instance
content_classifier = ContentClassifier()
\ No newline at end of file
This diff is collapsed.
...@@ -40,6 +40,8 @@ from .utils import ( ...@@ -40,6 +40,8 @@ from .utils import (
get_max_request_tokens_for_model get_max_request_tokens_for_model
) )
from .context import ContextManager, get_context_config_for_model from .context import ContextManager, get_context_config_for_model
from .classifier import content_classifier
from .semantic_classifier import SemanticClassifier
def generate_system_fingerprint(provider_id: str, seed: Optional[int] = None) -> str: def generate_system_fingerprint(provider_id: str, seed: Optional[int] = None) -> str:
...@@ -775,10 +777,16 @@ class RequestHandler: ...@@ -775,10 +777,16 @@ class RequestHandler:
# Enhance model information with context window and capabilities # Enhance model information with context window and capabilities
enhanced_models = [] enhanced_models = []
current_time = int(time_module.time())
for model in models: for model in models:
model_dict = model.dict() model_dict = model.dict()
model_name = model_dict.get('id', '') model_name = model_dict.get('id', '')
# Add OpenAI-compatible required fields
model_dict['object'] = 'model'
model_dict['created'] = current_time
model_dict['owned_by'] = provider_config.name
# Try to find model config in provider config # Try to find model config in provider config
model_config = None model_config = None
if provider_config.models: if provider_config.models:
...@@ -1511,7 +1519,9 @@ class RotationHandler: ...@@ -1511,7 +1519,9 @@ class RotationHandler:
'name': provider_model.name, 'name': provider_model.name,
'weight': provider_weight, # Use provider-level weight 'weight': provider_weight, # Use provider-level weight
'rate_limit': provider_model.rate_limit, 'rate_limit': provider_model.rate_limit,
'max_request_tokens': provider_model.max_request_tokens 'max_request_tokens': provider_model.max_request_tokens,
'nsfw': getattr(provider_model, 'nsfw', False),
'privacy': getattr(provider_model, 'privacy', False)
} }
rotation_models.append(model_dict) rotation_models.append(model_dict)
logger.info(f" Loaded {len(rotation_models)} model(s) from provider config with weight {provider_weight}") logger.info(f" Loaded {len(rotation_models)} model(s) from provider config with weight {provider_weight}")
...@@ -1553,6 +1563,94 @@ class RotationHandler: ...@@ -1553,6 +1563,94 @@ class RotationHandler:
logger.info(f"Total models considered: {total_models_considered}") logger.info(f"Total models considered: {total_models_considered}")
logger.info(f"Total models available: {len(available_models)}") logger.info(f"Total models available: {len(available_models)}")
# Apply NSFW/Privacy content classification filtering
# Only classify the immediate intent (last 3 messages + current query) to avoid
# huge context issues and long classification times
aisbf_config = self.config.get_aisbf_config()
if aisbf_config and (aisbf_config.classify_nsfw or aisbf_config.classify_privacy):
logger.info(f"=== CONTENT CLASSIFICATION FILTERING ===")
# Get messages for classification - only last 3 user messages + current query
messages = request_data.get('messages', [])
# Extract last 3 user messages for classification window
recent_user_messages = []
for msg in reversed(messages):
if msg.get('role') == 'user':
content = msg.get('content', '')
if isinstance(content, str):
recent_user_messages.append(content)
if len(recent_user_messages) >= 3:
break
# Build the classification prompt from recent messages only
# Reverse to get correct order (oldest first)
recent_user_messages.reverse()
user_prompt = " ".join(recent_user_messages)
logger.info(f"Classifying only recent context ({len(recent_user_messages)} messages))")
logger.info(f"Recent context preview: {user_prompt[:200]}..." if len(user_prompt) > 200 else f"Recent context: {user_prompt}")
logger.info(f"Classify NSFW: {aisbf_config.classify_nsfw}")
logger.info(f"Classify privacy: {aisbf_config.classify_privacy}")
# Check if content classification is needed
check_nsfw = aisbf_config.classify_nsfw
check_privacy = aisbf_config.classify_privacy
if check_nsfw or check_privacy:
# Initialize classifier with models from config
internal_model_config = aisbf_config.internal_model or {}
nsfw_model = internal_model_config.get('nsfw_classifier')
privacy_model = internal_model_config.get('privacy_classifier')
content_classifier.initialize(nsfw_model, privacy_model)
# Check user prompt for NSFW/privacy content
is_safe, message = content_classifier.check_content(
user_prompt,
check_nsfw=check_nsfw,
check_privacy=check_privacy
)
logger.info(f"Content classification result: {message}")
if not is_safe:
# Content is flagged - filter to only nsfw=True or privacy=True models
logger.info(f"Content flagged - filtering available models")
if check_nsfw and not check_privacy:
# Only NSFW filtering needed
original_count = len(available_models)
available_models = [m for m in available_models if m.get('nsfw', False)]
logger.info(f"NSFW filtering: {original_count} -> {len(available_models)} models")
elif check_privacy and not check_nsfw:
# Only privacy filtering needed
original_count = len(available_models)
available_models = [m for m in available_models if m.get('privacy', False)]
logger.info(f"Privacy filtering: {original_count} -> {len(available_models)} models")
elif check_nsfw and check_privacy:
# Both filtering - need models with EITHER flag
original_count = len(available_models)
available_models = [m for m in available_models if m.get('nsfw', False) or m.get('privacy', False)]
logger.info(f"NSFW+Privacy filtering: {original_count} -> {len(available_models)} models")
logger.info(f"=== CONTENT CLASSIFICATION FILTERING END ===")
# Check if rotation-level nsfw/privacy flags also apply
rotation_nsfw = getattr(rotation_config, 'nsfw', False)
rotation_privacy = getattr(rotation_config, 'privacy', False)
if rotation_nsfw or rotation_privacy:
logger.info(f"=== ROTATION-LEVEL CONTENT FLAGS ===")
logger.info(f"Rotation nsfw: {rotation_nsfw}, privacy: {rotation_privacy}")
# If rotation explicitly allows NSFW content, keep models that can handle it
# If rotation explicitly allows Privacy content, keep models that can handle it
if rotation_nsfw:
logger.info(f"Rotation allows NSFW content - keeping models that support it")
if rotation_privacy:
logger.info(f"Rotation allows Privacy content - keeping models that support it")
if not available_models: if not available_models:
logger.error("No models available in rotation (all providers may be rate limited)") logger.error("No models available in rotation (all providers may be rate limited)")
logger.error("All providers in this rotation are currently deactivated") logger.error("All providers in this rotation are currently deactivated")
...@@ -2865,8 +2963,51 @@ class AutoselectHandler: ...@@ -2865,8 +2963,51 @@ class AutoselectHandler:
import logging import logging
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.info(f"=== AUTOSELECT MODEL SELECTION START ===") logger.info(f"=== AUTOSELECT MODEL SELECTION START ===")
# Check if semantic classification is enabled
if autoselect_config.classify_semantic:
logger.info("=== SEMANTIC CLASSIFICATION ENABLED ===")
logger.info(f"Using semantic classification for model selection")
try:
# Initialize semantic classifier
semantic_classifier = SemanticClassifier()
semantic_classifier.initialize()
# Build model library for semantic search (model_id -> description)
model_library = {}
for model_info in autoselect_config.available_models:
model_library[model_info.model_id] = model_info.description
# Extract recent chat history (last 3 messages)
# Split user_prompt into messages (it's formatted as "role: content\nrole: content\n...")
chat_history = []
if user_prompt:
lines = user_prompt.strip().split('\n')
for line in lines[-3:]: # Last 3 messages
if ': ' in line:
role, content = line.split(': ', 1)
chat_history.append(content)
# Perform hybrid BM25 + semantic re-ranking
results = semantic_classifier.hybrid_model_search(user_prompt, chat_history, model_library, top_k=1)
if results:
selected_model_id, score = results[0]
logger.info(f"=== SEMANTIC CLASSIFICATION SUCCESS ===")
logger.info(f"Selected model ID: {selected_model_id} (score: {score:.4f})")
return selected_model_id
else:
logger.warning(f"=== SEMANTIC CLASSIFICATION FAILED ===")
logger.warning("No models returned from semantic search, falling back to AI model selection")
except Exception as e:
logger.error(f"=== SEMANTIC CLASSIFICATION ERROR ===")
logger.error(f"Error during semantic classification: {str(e)}")
logger.warning("Falling back to AI model selection")
logger.info(f"Using '{autoselect_config.selection_model}' for model selection") logger.info(f"Using '{autoselect_config.selection_model}' for model selection")
# Build messages (system + user) # Build messages (system + user)
messages = self._build_autoselect_messages(user_prompt, autoselect_config) messages = self._build_autoselect_messages(user_prompt, autoselect_config)
......
...@@ -22,6 +22,7 @@ Why did the programmer quit his job? Because he didn't get arrays! ...@@ -22,6 +22,7 @@ Why did the programmer quit his job? Because he didn't get arrays!
MCP Server for AISBF - Provides remote agent configuration capabilities. MCP Server for AISBF - Provides remote agent configuration capabilities.
""" """
import time
import json import json
import logging import logging
from typing import Dict, List, Optional, Any, Union from typing import Dict, List, Optional, Any, Union
...@@ -494,6 +495,13 @@ class MCPServer: ...@@ -494,6 +495,13 @@ class MCPServer:
provider_models = await request_handler.handle_model_list(dummy_request, provider_id) provider_models = await request_handler.handle_model_list(dummy_request, provider_id)
for model in provider_models: for model in provider_models:
model['id'] = f"{provider_id}/{model.get('id', '')}" model['id'] = f"{provider_id}/{model.get('id', '')}"
# Ensure OpenAI-compatible required fields are present
if 'object' not in model:
model['object'] = 'model'
if 'created' not in model:
model['created'] = int(time.time())
if 'owned_by' not in model:
model['owned_by'] = provider_config.name
model['type'] = 'provider' model['type'] = 'provider'
all_models.append(model) all_models.append(model)
except Exception as e: except Exception as e:
...@@ -504,11 +512,12 @@ class MCPServer: ...@@ -504,11 +512,12 @@ class MCPServer:
all_models.append({ all_models.append({
'id': f"rotation/{rotation_id}", 'id': f"rotation/{rotation_id}",
'object': 'model', 'object': 'model',
'created': 0, 'created': int(time.time()),
'owned_by': 'aisbf-rotation', 'owned_by': 'aisbf-rotation',
'type': 'rotation', 'type': 'rotation',
'rotation_id': rotation_id, 'rotation_id': rotation_id,
'model_name': rotation_config.model_name 'model_name': rotation_config.model_name,
'capabilities': getattr(rotation_config, 'capabilities', [])
}) })
# Add autoselect # Add autoselect
...@@ -516,12 +525,13 @@ class MCPServer: ...@@ -516,12 +525,13 @@ class MCPServer:
all_models.append({ all_models.append({
'id': f"autoselect/{autoselect_id}", 'id': f"autoselect/{autoselect_id}",
'object': 'model', 'object': 'model',
'created': 0, 'created': int(time.time()),
'owned_by': 'aisbf-autoselect', 'owned_by': 'aisbf-autoselect',
'type': 'autoselect', 'type': 'autoselect',
'autoselect_id': autoselect_id, 'autoselect_id': autoselect_id,
'model_name': autoselect_config.model_name, 'model_name': autoselect_config.model_name,
'description': autoselect_config.description 'description': autoselect_config.description,
'capabilities': getattr(autoselect_config, 'capabilities', [])
}) })
return {"models": all_models} return {"models": all_models}
......
"""
Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
AISBF - AI Service Broker Framework || AI Should Be Free
Semantic classification for model selection using hybrid BM25 + semantic re-ranking.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Semantic classifier for model selection.
"""
import logging
import threading
from typing import List, Dict, Optional, Tuple
class SemanticClassifier:
"""
Semantic classifier for model selection using hybrid BM25 + semantic re-ranking.
Uses BM25 for fast keyword search and semantic embeddings for re-ranking.
"""
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
if self._initialized:
return
self._initialized = True
self._embedder = None
self._embedder_lock = threading.Lock()
self._model_name = None
self.logger = logging.getLogger(__name__)
def initialize(self, model_name: Optional[str] = None):
"""
Initialize the semantic embedder.
Args:
model_name: HuggingFace model name for semantic embeddings
"""
self._model_name = model_name or "sentence-transformers/all-MiniLM-L6-v2"
self._load_embedder()
def _load_embedder(self):
"""Load the semantic embedder model"""
try:
from sentence_transformers import SentenceTransformer
self.logger.info(f"Loading semantic embedder model: {self._model_name}")
self._embedder = SentenceTransformer(self._model_name)
self.logger.info("Semantic embedder loaded successfully")
except Exception as e:
self.logger.error(f"Failed to load semantic embedder: {e}")
self._embedder = None
def hybrid_model_search(
self,
query: str,
chat_history: List[str],
model_library: Dict[str, str],
top_k: int = 3
) -> List[Tuple[str, float]]:
"""
Perform hybrid BM25 + semantic search to find the best matching models.
Args:
query: The current user query
chat_history: Recent chat history (last 3 messages)
model_library: Dict of {model_id: description}
top_k: Number of top candidates to return
Returns:
List of (model_id, score) tuples sorted by relevance
"""
if self._embedder is None:
self.logger.warning("Semantic embedder not initialized, falling back to simple matching")
return [(list(model_library.keys())[0], 1.0)] if model_library else []
try:
from rank_bm25 import BM25Okapi
from sentence_transformers import util
import numpy as np
# STEP 1: Build active window (last 3 messages + current query)
active_window = " ".join(chat_history[-3:] + [query])
self.logger.debug(f"Active window: {len(active_window.split())} words")
# STEP 2: BM25 keyword search on model descriptions
model_ids = list(model_library.keys())
descriptions = list(model_library.values())
# Tokenize corpus for BM25
tokenized_corpus = [desc.lower().split() for desc in descriptions]
bm25 = BM25Okapi(tokenized_corpus)
# Get BM25 scores for all models
tokenized_query = active_window.lower().split()
bm25_scores = bm25.get_scores(tokenized_query)
# Get top candidates based on BM25 (limit to top_k * 2 for re-ranking)
num_candidates = min(len(model_ids), top_k * 2)
top_bm25_indices = np.argsort(bm25_scores)[::-1][:num_candidates]
self.logger.debug(f"BM25 selected {len(top_bm25_indices)} candidates for re-ranking")
# STEP 3: Semantic re-ranking of BM25 candidates
# Vectorize active window (intent)
intent_vector = self._embedder.encode([active_window], convert_to_tensor=True)
# Vectorize only the candidate descriptions
candidate_descriptions = [descriptions[i] for i in top_bm25_indices]
candidate_vectors = self._embedder.encode(candidate_descriptions, convert_to_tensor=True)
# Compute cosine similarity
cosine_scores = util.cos_sim(intent_vector, candidate_vectors)[0]
# Get top_k from re-ranked candidates
top_semantic_indices = np.argsort(cosine_scores.cpu().numpy())[::-1][:top_k]
# Build results with scores
results = []
for idx in top_semantic_indices:
original_idx = top_bm25_indices[idx]
model_id = model_ids[original_idx]
score = float(cosine_scores[idx])
results.append((model_id, score))
self.logger.debug(f"Model: {model_id}, Score: {score:.4f}")
self.logger.info(f"Hybrid search completed: {len(results)} models ranked")
return results
except ImportError as e:
self.logger.error(f"Missing dependencies for hybrid search: {e}")
self.logger.error("Please install: pip install rank-bm25 sentence-transformers")
# Fallback to first model
return [(list(model_library.keys())[0], 1.0)] if model_library else []
except Exception as e:
self.logger.error(f"Error during hybrid model search: {e}")
# Fallback to first model
return [(list(model_library.keys())[0], 1.0)] if model_library else []
def select_best_model(
self,
query: str,
chat_history: List[str],
model_library: Dict[str, str]
) -> Optional[str]:
"""
Select the best model based on semantic similarity.
Args:
query: The current user query
chat_history: Recent chat history
model_library: Dict of {model_id: description}
Returns:
The best matching model_id or None
"""
results = self.hybrid_model_search(query, chat_history, model_library, top_k=1)
if results:
best_model, score = results[0]
self.logger.info(f"Selected model: {best_model} (score: {score:.4f})")
return best_model
return None
# Global semantic classifier instance
semantic_classifier = SemanticClassifier()
{ {
"classify_nsfw": false,
"classify_privacy": false,
"classify_semantic": false,
"server": { "server": {
"host": "0.0.0.0", "host": "0.0.0.0",
"port": 17765, "port": 17765,
...@@ -31,7 +34,10 @@ ...@@ -31,7 +34,10 @@
}, },
"internal_model": { "internal_model": {
"condensation_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3", "condensation_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
"autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3" "autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
"nsfw_classifier": "michelleli99/NSFW_text_classifier",
"privacy_classifier": "iiiorg/piiranha-v1-detect-personal-information",
"semantic_vectorization": "sentence-transformers/all-MiniLM-L6-v2"
}, },
"tor": { "tor": {
"enabled": false, "enabled": false,
......
{ {
"autoselect": { "autoselect": {
"nsfw": false,
"privacy": false,
"classify_nsfw": false,
"classify_privacy": false,
"classify_semantic": false,
"model_name": "autoselect", "model_name": "autoselect",
"description": "Auto-selects the best rotating model based on user prompt analysis", "description": "Auto-selects the best rotating model based on user prompt analysis",
"selection_model": "general", "selection_model": "general",
"fallback": "general", "fallback": "general",
"capabilities": ["t2t", "reasoning", "multimodal"],
"available_models": [ "available_models": [
{ {
"model_id": "coding", "model_id": "coding",
"nsfw": false,
"privacy": false,
"description": "Best for programming, code generation, debugging, and technical tasks. Optimized for software development, code reviews, and algorithm design." "description": "Best for programming, code generation, debugging, and technical tasks. Optimized for software development, code reviews, and algorithm design."
}, },
{ {
"model_id": "general", "model_id": "general",
"nsfw": false,
"privacy": false,
"description": "General purpose model for everyday tasks, conversations, and general knowledge queries. Good for a wide range of topics including writing, analysis, and explanations." "description": "General purpose model for everyday tasks, conversations, and general knowledge queries. Good for a wide range of topics including writing, analysis, and explanations."
} }
] ]
......
...@@ -17,6 +17,8 @@ ...@@ -17,6 +17,8 @@
"models": [ "models": [
{ {
"name": "gemini-2.0-flash", "name": "gemini-2.0-flash",
"nsfw": false,
"privacy": false,
"rate_limit": 0, "rate_limit": 0,
"max_request_tokens": 1000000, "max_request_tokens": 1000000,
"rate_limit_TPM": 15000, "rate_limit_TPM": 15000,
...@@ -29,6 +31,8 @@ ...@@ -29,6 +31,8 @@
}, },
{ {
"name": "gemini-1.5-pro", "name": "gemini-1.5-pro",
"nsfw": false,
"privacy": false,
"rate_limit": 0, "rate_limit": 0,
"max_request_tokens": 2000000, "max_request_tokens": 2000000,
"rate_limit_TPM": 15000, "rate_limit_TPM": 15000,
...@@ -48,6 +52,8 @@ ...@@ -48,6 +52,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_OPENAI_API_KEY", "api_key": "YOUR_OPENAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"anthropic": { "anthropic": {
...@@ -57,6 +63,8 @@ ...@@ -57,6 +63,8 @@
"type": "anthropic", "type": "anthropic",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_ANTHROPIC_API_KEY", "api_key": "YOUR_ANTHROPIC_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"ollama": { "ollama": {
...@@ -65,6 +73,8 @@ ...@@ -65,6 +73,8 @@
"endpoint": "http://localhost:11434", "endpoint": "http://localhost:11434",
"type": "ollama", "type": "ollama",
"api_key_required": false, "api_key_required": false,
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"azure_openai": { "azure_openai": {
...@@ -74,6 +84,8 @@ ...@@ -74,6 +84,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_AZURE_OPENAI_API_KEY", "api_key": "YOUR_AZURE_OPENAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"cohere": { "cohere": {
...@@ -83,6 +95,8 @@ ...@@ -83,6 +95,8 @@
"type": "cohere", "type": "cohere",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_COHERE_API_KEY", "api_key": "YOUR_COHERE_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"huggingface": { "huggingface": {
...@@ -92,6 +106,8 @@ ...@@ -92,6 +106,8 @@
"type": "huggingface", "type": "huggingface",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_HUGGINGFACE_API_KEY", "api_key": "YOUR_HUGGINGFACE_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"replicate": { "replicate": {
...@@ -101,6 +117,8 @@ ...@@ -101,6 +117,8 @@
"type": "replicate", "type": "replicate",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_REPLICATE_API_KEY", "api_key": "YOUR_REPLICATE_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"togetherai": { "togetherai": {
...@@ -110,6 +128,8 @@ ...@@ -110,6 +128,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_TOGETHERAI_API_KEY", "api_key": "YOUR_TOGETHERAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"groq": { "groq": {
...@@ -119,6 +139,8 @@ ...@@ -119,6 +139,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_GROQ_API_KEY", "api_key": "YOUR_GROQ_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"mistralai": { "mistralai": {
...@@ -128,6 +150,8 @@ ...@@ -128,6 +150,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_MISTRALAI_API_KEY", "api_key": "YOUR_MISTRALAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"stabilityai": { "stabilityai": {
...@@ -137,6 +161,8 @@ ...@@ -137,6 +161,8 @@
"type": "stabilityai", "type": "stabilityai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_STABILITYAI_API_KEY", "api_key": "YOUR_STABILITYAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"kilo": { "kilo": {
...@@ -146,6 +172,8 @@ ...@@ -146,6 +172,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_KILO_API_KEY", "api_key": "YOUR_KILO_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"perplexity": { "perplexity": {
...@@ -155,6 +183,8 @@ ...@@ -155,6 +183,8 @@
"type": "openai", "type": "openai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_PERPLEXITY_API_KEY", "api_key": "YOUR_PERPLEXITY_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"poe": { "poe": {
...@@ -164,6 +194,8 @@ ...@@ -164,6 +194,8 @@
"type": "poe", "type": "poe",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_POE_API_KEY", "api_key": "YOUR_POE_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"lanai": { "lanai": {
...@@ -173,6 +205,8 @@ ...@@ -173,6 +205,8 @@
"type": "lanai", "type": "lanai",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_LANAI_API_KEY", "api_key": "YOUR_LANAI_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"amazon": { "amazon": {
...@@ -182,6 +216,8 @@ ...@@ -182,6 +216,8 @@
"type": "amazon", "type": "amazon",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_AMAZON_API_KEY", "api_key": "YOUR_AMAZON_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"ibm": { "ibm": {
...@@ -191,6 +227,8 @@ ...@@ -191,6 +227,8 @@
"type": "ibm", "type": "ibm",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_IBM_API_KEY", "api_key": "YOUR_IBM_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"microsoft": { "microsoft": {
...@@ -200,6 +238,8 @@ ...@@ -200,6 +238,8 @@
"type": "microsoft", "type": "microsoft",
"api_key_required": true, "api_key_required": true,
"api_key": "YOUR_MICROSOFT_API_KEY", "api_key": "YOUR_MICROSOFT_API_KEY",
"nsfw": false,
"privacy": false,
"rate_limit": 0 "rate_limit": 0
}, },
"kiro": { "kiro": {
...@@ -208,6 +248,8 @@ ...@@ -208,6 +248,8 @@
"endpoint": "https://q.us-east-1.amazonaws.com", "endpoint": "https://q.us-east-1.amazonaws.com",
"type": "kiro", "type": "kiro",
"api_key_required": false, "api_key_required": false,
"nsfw": false,
"privacy": false,
"rate_limit": 0, "rate_limit": 0,
"kiro_config": { "kiro_config": {
"_comment": "Uses Kiro IDE credentials (VS Code extension)", "_comment": "Uses Kiro IDE credentials (VS Code extension)",
...@@ -221,6 +263,8 @@ ...@@ -221,6 +263,8 @@
"endpoint": "https://q.us-east-1.amazonaws.com", "endpoint": "https://q.us-east-1.amazonaws.com",
"type": "kiro", "type": "kiro",
"api_key_required": false, "api_key_required": false,
"nsfw": false,
"privacy": false,
"rate_limit": 0, "rate_limit": 0,
"kiro_config": { "kiro_config": {
"_comment": "Uses kiro-cli credentials (SQLite database)", "_comment": "Uses kiro-cli credentials (SQLite database)",
......
...@@ -3,7 +3,10 @@ ...@@ -3,7 +3,10 @@
"rotations": { "rotations": {
"coding": { "coding": {
"model_name": "coding", "model_name": "coding",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["code_generation", "code_completion", "reasoning"],
"providers": [ "providers": [
{ {
"provider_id": "gemini", "provider_id": "gemini",
...@@ -84,7 +87,10 @@ ...@@ -84,7 +87,10 @@
}, },
"general": { "general": {
"model_name": "general", "model_name": "general",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["t2t", "translation", "summarization", "question_answering"],
"providers": [ "providers": [
{ {
"provider_id": "gemini", "provider_id": "gemini",
...@@ -120,7 +126,10 @@ ...@@ -120,7 +126,10 @@
}, },
"googletest": { "googletest": {
"model_name": "googletest", "model_name": "googletest",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["code_generation", "testing", "code_completion"],
"providers": [ "providers": [
{ {
"provider_id": "gemini", "provider_id": "gemini",
...@@ -141,7 +150,10 @@ ...@@ -141,7 +150,10 @@
}, },
"kiro-claude": { "kiro-claude": {
"model_name": "kiro-claude", "model_name": "kiro-claude",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["t2t", "reasoning", "function_calling"],
"providers": [ "providers": [
{ {
"provider_id": "kiro", "provider_id": "kiro",
...@@ -180,7 +192,10 @@ ...@@ -180,7 +192,10 @@
"simple-gemini": { "simple-gemini": {
"_comment": "SIMPLE FORMAT: Specify provider_id only, models will be loaded from providers.json automatically", "_comment": "SIMPLE FORMAT: Specify provider_id only, models will be loaded from providers.json automatically",
"model_name": "simple-gemini", "model_name": "simple-gemini",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["t2t"],
"providers": [ "providers": [
{ {
"provider_id": "gemini" "provider_id": "gemini"
...@@ -190,7 +205,10 @@ ...@@ -190,7 +205,10 @@
"simple-openai": { "simple-openai": {
"_comment": "SIMPLE FORMAT: Provider with custom weight, models loaded from providers.json", "_comment": "SIMPLE FORMAT: Provider with custom weight, models loaded from providers.json",
"model_name": "simple-openai", "model_name": "simple-openai",
"nsfw": false,
"privacy": false,
"notifyerrors": false, "notifyerrors": false,
"capabilities": ["t2t"],
"providers": [ "providers": [
{ {
"provider_id": "openai", "provider_id": "openai",
...@@ -199,4 +217,4 @@ ...@@ -199,4 +217,4 @@
] ]
} }
} }
} }
\ No newline at end of file
...@@ -713,6 +713,8 @@ async def get_provider_models(provider_id: str, provider_config) -> list: ...@@ -713,6 +713,8 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
logger.debug(f"Skipping provider {provider_id}: Kiro credentials not available or invalid") logger.debug(f"Skipping provider {provider_id}: Kiro credentials not available or invalid")
return [] return []
current_time = int(time.time())
# If provider has local model config, use it # If provider has local model config, use it
if hasattr(provider_config, 'models') and provider_config.models: if hasattr(provider_config, 'models') and provider_config.models:
models = [] models = []
...@@ -721,13 +723,19 @@ async def get_provider_models(provider_id: str, provider_config) -> list: ...@@ -721,13 +723,19 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
models.append({ models.append({
'id': model_id, 'id': model_id,
'object': 'model', 'object': 'model',
'created': int(time.time()), 'created': current_time,
'owned_by': provider_config.name, 'owned_by': provider_config.name,
'provider': provider_id, 'provider': provider_id,
'type': 'provider', 'type': 'provider',
'model_name': model.name, 'model_name': model.name,
'context_size': getattr(model, 'context_size', None), 'context_size': getattr(model, 'context_size', None),
'capabilities': getattr(model, 'capabilities', []), 'capabilities': getattr(model, 'capabilities', []),
'description': getattr(model, 'description', None),
'architecture': getattr(model, 'architecture', None),
'pricing': getattr(model, 'pricing', None),
'top_provider': getattr(model, 'top_provider', None),
'supported_parameters': getattr(model, 'supported_parameters', None),
'default_parameters': getattr(model, 'default_parameters', None),
'source': 'local_config' 'source': 'local_config'
}) })
return models return models
...@@ -739,11 +747,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list: ...@@ -739,11 +747,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
# Cache is still fresh, use it # Cache is still fresh, use it
cached_models = _model_cache[provider_id] cached_models = _model_cache[provider_id]
if cached_models: # Only return if we have actual models if cached_models: # Only return if we have actual models
# Add provider prefix to model IDs # Add provider prefix to model IDs and ensure all required fields
models = [] models = []
for model in cached_models: for model in cached_models:
model_copy = model.copy() model_copy = model.copy()
model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}" model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
# Ensure OpenAI-compatible required fields are present
if 'object' not in model_copy:
model_copy['object'] = 'model'
if 'created' not in model_copy:
model_copy['created'] = current_time
if 'owned_by' not in model_copy:
model_copy['owned_by'] = provider_config.name
model_copy['provider'] = provider_id model_copy['provider'] = provider_id
model_copy['type'] = 'provider' model_copy['type'] = 'provider'
model_copy['source'] = 'api_cache' model_copy['source'] = 'api_cache'
...@@ -755,11 +770,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list: ...@@ -755,11 +770,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
try: try:
fetched_models = await fetch_provider_models(provider_id) fetched_models = await fetch_provider_models(provider_id)
if fetched_models: if fetched_models:
# Add provider prefix to model IDs # Add provider prefix to model IDs and ensure all required fields
models = [] models = []
for model in fetched_models: for model in fetched_models:
model_copy = model.copy() model_copy = model.copy()
model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}" model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
# Ensure OpenAI-compatible required fields are present
if 'object' not in model_copy:
model_copy['object'] = 'model'
if 'created' not in model_copy:
model_copy['created'] = current_time
if 'owned_by' not in model_copy:
model_copy['owned_by'] = provider_config.name
model_copy['provider'] = provider_id model_copy['provider'] = provider_id
model_copy['type'] = 'provider' model_copy['type'] = 'provider'
model_copy['source'] = 'api_cache' model_copy['source'] = 'api_cache'
...@@ -1887,7 +1909,8 @@ async def list_all_models(request: Request): ...@@ -1887,7 +1909,8 @@ async def list_all_models(request: Request):
'owned_by': 'aisbf-rotation', 'owned_by': 'aisbf-rotation',
'type': 'rotation', 'type': 'rotation',
'rotation_id': rotation_id, 'rotation_id': rotation_id,
'model_name': rotation_config.model_name 'model_name': rotation_config.model_name,
'capabilities': getattr(rotation_config, 'capabilities', [])
}) })
except Exception as e: except Exception as e:
logger.warning(f"Error listing rotation {rotation_id}: {e}") logger.warning(f"Error listing rotation {rotation_id}: {e}")
...@@ -1903,7 +1926,8 @@ async def list_all_models(request: Request): ...@@ -1903,7 +1926,8 @@ async def list_all_models(request: Request):
'type': 'autoselect', 'type': 'autoselect',
'autoselect_id': autoselect_id, 'autoselect_id': autoselect_id,
'model_name': autoselect_config.model_name, 'model_name': autoselect_config.model_name,
'description': autoselect_config.description 'description': autoselect_config.description,
'capabilities': getattr(autoselect_config, 'capabilities', [])
}) })
except Exception as e: except Exception as e:
logger.warning(f"Error listing autoselect {autoselect_id}: {e}") logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
...@@ -1936,7 +1960,8 @@ async def v1_list_all_models(request: Request): ...@@ -1936,7 +1960,8 @@ async def v1_list_all_models(request: Request):
'owned_by': 'aisbf-rotation', 'owned_by': 'aisbf-rotation',
'type': 'rotation', 'type': 'rotation',
'rotation_id': rotation_id, 'rotation_id': rotation_id,
'model_name': rotation_config.model_name 'model_name': rotation_config.model_name,
'capabilities': getattr(rotation_config, 'capabilities', [])
}) })
except Exception as e: except Exception as e:
logger.warning(f"Error listing rotation {rotation_id}: {e}") logger.warning(f"Error listing rotation {rotation_id}: {e}")
...@@ -1952,7 +1977,8 @@ async def v1_list_all_models(request: Request): ...@@ -1952,7 +1977,8 @@ async def v1_list_all_models(request: Request):
'type': 'autoselect', 'type': 'autoselect',
'autoselect_id': autoselect_id, 'autoselect_id': autoselect_id,
'model_name': autoselect_config.model_name, 'model_name': autoselect_config.model_name,
'description': autoselect_config.description 'description': autoselect_config.description,
'capabilities': getattr(autoselect_config, 'capabilities', [])
}) })
except Exception as e: except Exception as e:
logger.warning(f"Error listing autoselect {autoselect_id}: {e}") logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
...@@ -2297,8 +2323,9 @@ async def list_rotation_models(): ...@@ -2297,8 +2323,9 @@ async def list_rotation_models():
for model in provider['models']: for model in provider['models']:
all_models.append({ all_models.append({
"id": f"{rotation_id}/{model['name']}", "id": f"{rotation_id}/{model['name']}",
"name": rotation_id, # Use rotation name as the model name for selection "name": rotation_id,
"object": "model", "object": "model",
"created": int(time.time()),
"owned_by": provider['provider_id'], "owned_by": provider['provider_id'],
"rotation_id": rotation_id, "rotation_id": rotation_id,
"actual_model": model['name'], "actual_model": model['name'],
...@@ -2384,8 +2411,9 @@ async def list_autoselect_models(): ...@@ -2384,8 +2411,9 @@ async def list_autoselect_models():
for model_info in autoselect_config.available_models: for model_info in autoselect_config.available_models:
all_models.append({ all_models.append({
"id": model_info.model_id, "id": model_info.model_id,
"name": autoselect_id, # Use autoselect name as the model name for selection "name": autoselect_id,
"object": "model", "object": "model",
"created": int(time.time()),
"owned_by": "autoselect", "owned_by": "autoselect",
"autoselect_id": autoselect_id, "autoselect_id": autoselect_id,
"description": model_info.description, "description": model_info.description,
......
...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" ...@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "aisbf" name = "aisbf"
version = "0.5.0" version = "0.6.0"
description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations" description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations"
readme = "README.md" readme = "README.md"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"
......
...@@ -13,6 +13,8 @@ langchain-text-splitters ...@@ -13,6 +13,8 @@ langchain-text-splitters
tiktoken tiktoken
torch torch
transformers transformers
sentence-transformers
rank-bm25
jinja2 jinja2
itsdangerous itsdangerous
bs4 bs4
......
...@@ -49,7 +49,7 @@ class InstallCommand(_install): ...@@ -49,7 +49,7 @@ class InstallCommand(_install):
setup( setup(
name="aisbf", name="aisbf",
version="0.5.0", version="0.6.0",
author="AISBF Contributors", author="AISBF Contributors",
author_email="stefy@nexlab.net", author_email="stefy@nexlab.net",
description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations", description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",
...@@ -111,6 +111,7 @@ setup( ...@@ -111,6 +111,7 @@ setup(
'aisbf/kiro_models.py', 'aisbf/kiro_models.py',
'aisbf/kiro_parsers.py', 'aisbf/kiro_parsers.py',
'aisbf/kiro_utils.py', 'aisbf/kiro_utils.py',
'aisbf/semantic_classifier.py',
]), ]),
# Install dashboard templates # Install dashboard templates
('share/aisbf/templates', [ ('share/aisbf/templates', [
......
...@@ -131,6 +131,49 @@ function renderAutoselectDetails(autoselectKey) { ...@@ -131,6 +131,49 @@ function renderAutoselectDetails(autoselectKey) {
<input type="text" value="${autoselect.model_name}" onchange="updateAutoselect('${autoselectKey}', 'model_name', this.value)" required> <input type="text" value="${autoselect.model_name}" onchange="updateAutoselect('${autoselectKey}', 'model_name', this.value)" required>
</div> </div>
<div class="form-group">
<label>Capabilities (comma-separated)</label>
<input type="text" value="${autoselect.capabilities ? autoselect.capabilities.join(', ') : ''}" onchange="updateAutoselectCapabilities('${autoselectKey}', this.value)" placeholder="e.g., t2t, reasoning, multimodal">
</div>
<div class="form-group">
<label>
<input type="checkbox" ${autoselect.nsfw ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'nsfw', this.checked)">
NSFW
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${autoselect.privacy ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'privacy', this.checked)">
Privacy
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${autoselect.classify_nsfw ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_nsfw', this.checked)">
Classify NSFW
</label>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_nsfw setting for this autoselection</small>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${autoselect.classify_privacy ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_privacy', this.checked)">
Classify Privacy
</label>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_privacy setting for this autoselection</small>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${autoselect.classify_semantic ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_semantic', this.checked)">
Classify Semantic
</label>
<small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_semantic setting for this autoselection</small>
</div>
<div class="form-group"> <div class="form-group">
<label>Description</label> <label>Description</label>
<textarea onchange="updateAutoselect('${autoselectKey}', 'description', this.value)" style="min-height: 60px;">${autoselect.description || ''}</textarea> <textarea onchange="updateAutoselect('${autoselectKey}', 'description', this.value)" style="min-height: 60px;">${autoselect.description || ''}</textarea>
...@@ -201,6 +244,20 @@ function renderAutoselectModels(autoselectKey) { ...@@ -201,6 +244,20 @@ function renderAutoselectModels(autoselectKey) {
<textarea onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'description', this.value)" style="min-height: 80px;" required>${model.description || ''}</textarea> <textarea onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'description', this.value)" style="min-height: 80px;" required>${model.description || ''}</textarea>
<small style="color: #666; font-size: 12px;">Be specific about when this model should be used (e.g., "Best for programming, code generation, debugging")</small> <small style="color: #666; font-size: 12px;">Be specific about when this model should be used (e.g., "Best for programming, code generation, debugging")</small>
</div> </div>
<div class="form-group">
<label>
<input type="checkbox" ${model.nsfw ? 'checked' : ''} onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'nsfw', this.checked)">
NSFW
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${model.privacy ? 'checked' : ''} onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'privacy', this.checked)">
Privacy
</label>
</div>
`; `;
container.appendChild(modelDiv); container.appendChild(modelDiv);
...@@ -242,6 +299,18 @@ function updateAutoselect(key, field, value) { ...@@ -242,6 +299,18 @@ function updateAutoselect(key, field, value) {
autoselectConfig[key][field] = value; autoselectConfig[key][field] = value;
} }
function updateAutoselectCapabilities(autoselectKey, value) {
const trimmed = value.trim();
if (!trimmed) {
autoselectConfig[autoselectKey].capabilities = null;
return;
}
// Split by comma and clean up
autoselectConfig[autoselectKey].capabilities =
trimmed.split(',').map(s => s.trim()).filter(s => s);
}
function addAutoselectModel(autoselectKey) { function addAutoselectModel(autoselectKey) {
if (!autoselectConfig[autoselectKey].available_models) { if (!autoselectConfig[autoselectKey].available_models) {
autoselectConfig[autoselectKey].available_models = []; autoselectConfig[autoselectKey].available_models = [];
......
...@@ -287,6 +287,20 @@ function renderProviderDetails(key) { ...@@ -287,6 +287,20 @@ function renderProviderDetails(key) {
<input type="text" value="${Array.isArray(provider.default_condense_method) ? provider.default_condense_method.join(', ') : (provider.default_condense_method || '')}" onchange="updateProviderCondenseMethod('${key}', this.value)" placeholder="e.g., semantic, conversational"> <input type="text" value="${Array.isArray(provider.default_condense_method) ? provider.default_condense_method.join(', ') : (provider.default_condense_method || '')}" onchange="updateProviderCondenseMethod('${key}', this.value)" placeholder="e.g., semantic, conversational">
</div> </div>
<div class="form-group">
<label>
<input type="checkbox" ${provider.nsfw ? 'checked' : ''} onchange="updateProvider('${key}', 'nsfw', this.checked)">
NSFW
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${provider.privacy ? 'checked' : ''} onchange="updateProvider('${key}', 'privacy', this.checked)">
Privacy
</label>
</div>
<h4 style="margin-top: 20px; margin-bottom: 10px;">Models</h4> <h4 style="margin-top: 20px; margin-bottom: 10px;">Models</h4>
<div id="models-${key}"></div> <div id="models-${key}"></div>
<button type="button" class="btn btn-secondary" onclick="addModel('${key}')" style="margin-top: 10px;">Add Model</button> <button type="button" class="btn btn-secondary" onclick="addModel('${key}')" style="margin-top: 10px;">Add Model</button>
...@@ -359,6 +373,20 @@ function renderModels(providerKey) { ...@@ -359,6 +373,20 @@ function renderModels(providerKey) {
<label>Condense Method (conversational, semantic, hierarchical, algorithmic)</label> <label>Condense Method (conversational, semantic, hierarchical, algorithmic)</label>
<input type="text" value="${Array.isArray(model.condense_method) ? model.condense_method.join(', ') : (model.condense_method || '')}" onchange="updateModelCondenseMethod('${providerKey}', ${index}, this.value)" placeholder="e.g., semantic, conversational"> <input type="text" value="${Array.isArray(model.condense_method) ? model.condense_method.join(', ') : (model.condense_method || '')}" onchange="updateModelCondenseMethod('${providerKey}', ${index}, this.value)" placeholder="e.g., semantic, conversational">
</div> </div>
<div class="form-group">
<label>
<input type="checkbox" ${model.nsfw ? 'checked' : ''} onchange="updateModel('${providerKey}', ${index}, 'nsfw', this.checked)">
NSFW
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${model.privacy ? 'checked' : ''} onchange="updateModel('${providerKey}', ${index}, 'privacy', this.checked)">
Privacy
</label>
</div>
`; `;
container.appendChild(modelDiv); container.appendChild(modelDiv);
......
...@@ -113,6 +113,11 @@ function renderRotationDetails(rotationKey) { ...@@ -113,6 +113,11 @@ function renderRotationDetails(rotationKey) {
</label> </label>
</div> </div>
<div class="form-group">
<label>Capabilities (comma-separated)</label>
<input type="text" value="${rotation.capabilities ? rotation.capabilities.join(', ') : ''}" onchange="updateRotationCapabilities('${rotationKey}', this.value)" placeholder="e.g., code_generation, t2t, reasoning">
</div>
<div class="form-group"> <div class="form-group">
<label>Default Rate Limit (seconds)</label> <label>Default Rate Limit (seconds)</label>
<input type="number" value="${rotation.default_rate_limit || ''}" onchange="updateRotation('${rotationKey}', 'default_rate_limit', this.value ? parseFloat(this.value) : null)" step="0.1" placeholder="Optional"> <input type="number" value="${rotation.default_rate_limit || ''}" onchange="updateRotation('${rotationKey}', 'default_rate_limit', this.value ? parseFloat(this.value) : null)" step="0.1" placeholder="Optional">
...@@ -123,6 +128,20 @@ function renderRotationDetails(rotationKey) { ...@@ -123,6 +128,20 @@ function renderRotationDetails(rotationKey) {
<input type="number" value="${rotation.default_context_size || ''}" onchange="updateRotation('${rotationKey}', 'default_context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional"> <input type="number" value="${rotation.default_context_size || ''}" onchange="updateRotation('${rotationKey}', 'default_context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional">
</div> </div>
<div class="form-group">
<label>
<input type="checkbox" ${rotation.nsfw ? 'checked' : ''} onchange="updateRotation('${rotationKey}', 'nsfw', this.checked)">
NSFW
</label>
</div>
<div class="form-group">
<label>
<input type="checkbox" ${rotation.privacy ? 'checked' : ''} onchange="updateRotation('${rotationKey}', 'privacy', this.checked)">
Privacy
</label>
</div>
<h4 style="margin-top: 20px; margin-bottom: 10px;">Providers</h4> <h4 style="margin-top: 20px; margin-bottom: 10px;">Providers</h4>
<div id="providers-${rotationKey}"></div> <div id="providers-${rotationKey}"></div>
<button type="button" class="btn btn-secondary" onclick="addRotationProvider('${rotationKey}')" style="margin-top: 10px;">Add Provider</button> <button type="button" class="btn btn-secondary" onclick="addRotationProvider('${rotationKey}')" style="margin-top: 10px;">Add Provider</button>
...@@ -257,6 +276,7 @@ function addRotation() { ...@@ -257,6 +276,7 @@ function addRotation() {
rotationsConfig.rotations[key] = { rotationsConfig.rotations[key] = {
model_name: key, model_name: key,
notifyerrors: false, notifyerrors: false,
capabilities: [],
providers: [] providers: []
}; };
...@@ -276,6 +296,18 @@ function updateRotation(key, field, value) { ...@@ -276,6 +296,18 @@ function updateRotation(key, field, value) {
rotationsConfig.rotations[key][field] = value; rotationsConfig.rotations[key][field] = value;
} }
function updateRotationCapabilities(rotationKey, value) {
const trimmed = value.trim();
if (!trimmed) {
rotationsConfig.rotations[rotationKey].capabilities = null;
return;
}
// Split by comma and clean up
rotationsConfig.rotations[rotationKey].capabilities =
trimmed.split(',').map(s => s.trim()).filter(s => s);
}
function addRotationProvider(rotationKey) { function addRotationProvider(rotationKey) {
if (!rotationsConfig.rotations[rotationKey].providers) { if (!rotationsConfig.rotations[rotationKey].providers) {
rotationsConfig.rotations[rotationKey].providers = []; rotationsConfig.rotations[rotationKey].providers = [];
......
...@@ -132,6 +132,50 @@ along with this program. If not, see <https://www.gnu.org/licenses/>. ...@@ -132,6 +132,50 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
<small style="color: #666; display: block; margin-top: 5px;">Used when autoselect selection_model is set to "internal"</small> <small style="color: #666; display: block; margin-top: 5px;">Used when autoselect selection_model is set to "internal"</small>
</div> </div>
<div class="form-group">
<label for="nsfw_classifier">NSFW Classifier Model ID</label>
<input type="text" id="nsfw_classifier" name="nsfw_classifier" value="{{ config.internal_model.nsfw_classifier or 'michelleli99/NSFW_text_classifier' }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Model used for NSFW content detection</small>
</div>
<div class="form-group">
<label for="privacy_classifier">Privacy Classifier Model ID</label>
<input type="text" id="privacy_classifier" name="privacy_classifier" value="{{ config.internal_model.privacy_classifier or 'iiiorg/piiranha-v1-detect-personal-information' }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Model used for privacy-sensitive information detection</small>
</div>
<div class="form-group">
<label for="semantic_vectorization">Semantic Vectorization Model ID</label>
<input type="text" id="semantic_vectorization" name="semantic_vectorization" value="{{ config.internal_model.semantic_vectorization or 'sentence-transformers/all-MiniLM-L6-v2' }}" required>
<small style="color: #666; display: block; margin-top: 5px;">Model used for semantic embedding and vectorization</small>
</div>
<h3 style="margin: 30px 0 20px;">Content Classification</h3>
<div class="form-group">
<label>
<input type="checkbox" name="classify_nsfw" {% if config.classify_nsfw %}checked{% endif %}>
Enable NSFW Classification
</label>
<small style="color: #666; display: block; margin-top: 5px;">Enable automatic NSFW content detection for model selection</small>
</div>
<div class="form-group">
<label>
<input type="checkbox" name="classify_privacy" {% if config.classify_privacy %}checked{% endif %}>
Enable Privacy Classification
</label>
<small style="color: #666; display: block; margin-top: 5px;">Enable automatic privacy-sensitive content detection for model selection</small>
</div>
<div class="form-group">
<label>
<input type="checkbox" name="classify_semantic" {% if config.classify_semantic %}checked{% endif %}>
Enable Semantic Classification
</label>
<small style="color: #666; display: block; margin-top: 5px;">Enable semantic content classification using the configured model</small>
</div>
<h3 style="margin: 30px 0 20px;">TOR Hidden Service</h3> <h3 style="margin: 30px 0 20px;">TOR Hidden Service</h3>
<div id="tor-status" style="margin-bottom: 20px; padding: 15px; background: #0f3460; border-radius: 6px; border-left: 4px solid #16213e;"> <div id="tor-status" style="margin-bottom: 20px; padding: 15px; background: #0f3460; border-radius: 6px; border-left: 4px solid #16213e;">
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment