feat: Add NSFW/privacy content filtering and semantic classification

- Add NSFW/privacy boolean fields to models (providers.json, rotations.json, autoselect.json)
- Implement content classification using last 3 messages for performance
- Add semantic classification with hybrid BM25 + sentence-transformer re-ranking
- Update autoselect handler to support classify_semantic flag
- Add new semantic_classifier.py module with hybrid search capabilities
- Update dashboard templates to manage new configuration fields
- Update documentation (README.md, DOCUMENTATION.md) with new features
- Bump version to 0.6.0 in pyproject.toml and setup.py
- Add new dependencies: sentence-transformers, rank-bm25
- Update package configuration for PyPI distribution

DOCUMENTATION.md

@@ -415,6 +415,83 @@ Autoselect models use AI to analyze the user's request and select the best model
 3. **Automatic Routing**: Routes the request to the most suitable model
 4. **Fallback**: Uses a default model if selection fails or is uncertain
 
+### Semantic Classification (New Feature)
+
+Autoselect configurations can now use semantic classification instead of AI model selection for faster and potentially more accurate model selection:
+
+- **Hybrid BM25 + Semantic Search**: Combines fast keyword matching with semantic similarity using sentence transformers
+- **Automatic Model Library Indexing**: Builds and caches model embeddings for efficient similarity matching
+- **Performance Optimized**: No API calls required, significantly faster than AI-based selection
+- **Fallback Support**: Falls back to AI model selection if semantic classification fails
+
+**Enable Semantic Classification:**
+```json
+{
+  "autoselect": {
+    "smart": {
+      "classify_semantic": true,
+      "selection_model": "openai/gpt-4",
+      "available_models": [...]
+    }
+  }
+}
+```
+
+**Benefits:**
+- Faster model selection (no API costs)
+- Deterministic results based on semantic similarity
+- Automatic model library caching in `~/.aisbf/`
+- Lower latency for model selection
+
+### Content Classification and Filtering (New Feature)
+
+AISBF provides advanced content classification for NSFW and privacy-sensitive content:
+
+#### NSFW/Privacy Content Filtering
+
+Models can be configured with boolean flags to indicate their suitability for sensitive content:
+- **`nsfw`**: Model supports NSFW (Not Safe For Work) content
+- **`privacy`**: Model supports privacy-sensitive content (medical, financial, legal data)
+
+When global `classify_nsfw` or `classify_privacy` is enabled, AISBF analyzes the last 3 user messages and routes requests only to appropriate models. If no suitable models are available, requests return a 404 error.
+
+**Configuration Levels (cascading priority):**
+1. Rotation model config (highest priority)
+2. Provider model config
+3. Global AISBF config (enables/disables classification)
+
+**Example Model Configuration:**
+```json
+{
+  "models": [
+    {
+      "name": "gpt-4",
+      "nsfw": true,
+      "privacy": true
+    },
+    {
+      "name": "gpt-3.5-turbo",
+      "nsfw": false,
+      "privacy": true
+    }
+  ]
+}
+```
+
+**Global Classification Settings:**
+```json
+{
+  "classify_nsfw": true,
+  "classify_privacy": true
+}
+```
+
+**Content Classification Window:**
+- Only analyzes the last 3 user messages for performance
+- Avoids processing full conversation history
+- Uses HuggingFace transformers for content analysis
+- Automatic fallback if classification models are unavailable
+
 ### Autoselect Configuration
 
 ```json

README.md

@@ -21,6 +21,8 @@ Access the dashboard at `http://localhost:17765/dashboard` (default credentials:
 - **Multi-Provider Support**: Unified interface for Google, OpenAI, Anthropic, Ollama, and Kiro (Amazon Q Developer)
 - **Rotation Models**: Weighted load balancing across multiple providers with automatic failover
 - **Autoselect Models**: AI-powered model selection based on content analysis and request characteristics
+- **Semantic Classification**: Fast hybrid BM25 + semantic model selection using sentence transformers (optional)
+- **Content Classification**: NSFW/privacy content filtering with configurable classification windows
 - **Streaming Support**: Full support for streaming responses from all providers
 - **Error Tracking**: Automatic provider disabling after consecutive failures with cooldown periods
 - **Rate Limiting**: Built-in rate limiting and graceful error handling
@@ -287,6 +289,53 @@ When context exceeds the configured percentage of `context_size`, the system aut
 
 See `config/providers.json` and `config/rotations.json` for configuration examples.
 
+### Content Classification and Semantic Selection
+
+AISBF provides advanced content filtering and intelligent model selection based on content analysis:
+
+#### NSFW/Privacy Content Filtering
+
+Models can be configured with `nsfw` and `privacy` boolean flags to indicate their suitability for sensitive content:
+
+- **`nsfw`**: Model supports NSFW (Not Safe For Work) content
+- **`privacy`**: Model supports privacy-sensitive content (e.g., medical, financial, legal data)
+
+When global `classify_nsfw` or `classify_privacy` is enabled, AISBF automatically analyzes the last 3 user messages to classify content and routes requests only to appropriate models. If no suitable models are available, the request returns a 404 error.
+
+**Configuration:**
+- Provider models: Set in `config/providers.json`
+- Rotation models: Override in `config/rotations.json`
+- Global settings: Enable/disable in `config/aisbf.json` or dashboard
+
+#### Semantic Model Selection
+
+For enhanced performance, autoselect configurations can use semantic classification instead of AI model selection:
+
+- **Hybrid BM25 + Semantic Search**: Combines fast keyword matching with semantic similarity
+- **Sentence Transformers**: Uses pre-trained embeddings for content understanding
+- **Automatic Fallback**: Falls back to AI model selection if semantic classification fails
+
+**Enable Semantic Classification:**
+```json
+{
+  "autoselect": {
+    "autoselect": {
+      "classify_semantic": true,
+      "selection_model": "openai/gpt-4",
+      "available_models": [...]
+    }
+  }
+}
+```
+
+**Benefits:**
+- Faster model selection (no API calls required)
+- Lower costs (no tokens consumed for selection)
+- Deterministic results based on content similarity
+- Automatic model library indexing and caching
+
+See `config/autoselect.json` for configuration examples.
+
 ## API Endpoints
 
 ### Three Proxy Paths

aisbf/classifier.py

+"""
+Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
+
+AISBF - AI Service Broker Framework || AI Should Be Free
+
+Content classification for NSFW and privacy detection.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Content classifier for NSFW and privacy detection.
+"""
+import logging
+import threading
+from typing import Optional, Tuple
+
+class ContentClassifier:
+    """
+    Content classifier for NSFW and privacy detection.
+    Uses HuggingFace transformers for classification.
+    """
+    
+    _instance = None
+    _lock = threading.Lock()
+    
+    def __new__(cls):
+        if cls._instance is None:
+            with cls._lock:
+                if cls._instance is None:
+                    cls._instance = super().__new__(cls)
+                    cls._instance._initialized = False
+        return cls._instance
+    
+    def __init__(self):
+        if self._initialized:
+            return
+        
+        self._initialized = True
+        self._nsfw_classifier = None
+        self._privacy_classifier = None
+        self._classifier_lock = threading.Lock()
+        self._nsfw_model_name = None
+        self._privacy_model_name = None
+        self.logger = logging.getLogger(__name__)
+    
+    def initialize(self, nsfw_model_name: Optional[str] = None, privacy_model_name: Optional[str] = None):
+        """
+        Initialize the classifiers with the specified model names.
+        
+        Args:
+            nsfw_model_name: HuggingFace model name for NSFW classification
+            privacy_model_name: HuggingFace model name for privacy classification
+        """
+        self._nsfw_model_name = nsfw_model_name
+        self._privacy_model_name = privacy_model_name
+        
+        if nsfw_model_name:
+            self._load_nsfw_classifier(nsfw_model_name)
+        
+        if privacy_model_name:
+            self._load_privacy_classifier(privacy_model_name)
+    
+    def _load_nsfw_classifier(self, model_name: str):
+        """Load the NSFW classifier model"""
+        try:
+            from transformers import pipeline
+            self.logger.info(f"Loading NSFW classifier model: {model_name}")
+            self._nsfw_classifier = pipeline("text-classification", model=model_name)
+            self.logger.info("NSFW classifier loaded successfully")
+        except Exception as e:
+            self.logger.error(f"Failed to load NSFW classifier: {e}")
+            self._nsfw_classifier = None
+    
+    def _load_privacy_classifier(self, model_name: str):
+        """Load the privacy classifier model"""
+        try:
+            from transformers import pipeline
+            self.logger.info(f"Loading privacy classifier model: {model_name}")
+            self._privacy_classifier = pipeline("text-classification", model=model_name)
+            self.logger.info("Privacy classifier loaded successfully")
+        except Exception as e:
+            self.logger.error(f"Failed to load privacy classifier: {e}")
+            self._privacy_classifier = None
+    
+    def check_nsfw(self, text: str, threshold: float = 0.8) -> Tuple[bool, str]:
+        """
+        Check if the given text contains NSFW content.
+        
+        Args:
+            text: The text to check
+            threshold: Confidence threshold for NSFW detection (default: 0.8)
+        
+        Returns:
+            Tuple of (is_safe, message)
+            - is_safe: True if content is safe, False if NSFW
+            - message: Description of the result
+        """
+        if self._nsfw_classifier is None:
+            self.logger.warning("NSFW classifier not initialized, allowing content")
+            return True, "NSFW classifier not available"
+        
+        try:
+            # Truncate to first 512 characters for classification (avoid huge inputs)
+            # This is enough to detect the intent/content type
+            text_to_check = text[:512] if len(text) > 512 else text
+            result = self._nsfw_classifier(text_to_check)[0]
+            
+            self.logger.debug(f"NSFW classification result: {result}")
+            
+            if result['label'] == 'NSFW' and result['score'] > threshold:
+                return False, f"Content classified as NSFW (confidence: {result['score']:.2f})"
+            
+            return True, "Content is safe"
+        except Exception as e:
+            self.logger.error(f"Error during NSFW classification: {e}")
+            # Default to safe on error
+            return True, f"Error during classification: {str(e)}"
+    
+    def check_privacy(self, text: str, threshold: float = 0.8) -> Tuple[bool, str]:
+        """
+        Check if the given text contains privacy-sensitive information.
+        
+        Args:
+            text: The text to check
+            threshold: Confidence threshold for privacy detection (default: 0.8)
+        
+        Returns:
+            Tuple of (is_safe, message)
+            - is_safe: True if content is safe, False if contains sensitive info
+            - message: Description of the result
+        """
+        if self._privacy_classifier is None:
+            self.logger.warning("Privacy classifier not initialized, allowing content")
+            return True, "Privacy classifier not available"
+        
+        try:
+            # Truncate to first 512 characters for classification (avoid huge inputs)
+            # This is enough to detect personal/sensitive information
+            text_to_check = text[:512] if len(text) > 512 else text
+            result = self._privacy_classifier(text_to_check)[0]
+            
+            self.logger.debug(f"Privacy classification result: {result}")
+            
+            # Common labels for sensitive/personal information
+            sensitive_labels = ['personal', 'pii', 'sensitive', 'private', 'nlp/privacy']
+            
+            if result['label'].lower() in [l.lower() for l in sensitive_labels] and result['score'] > threshold:
+                return False, f"Content contains privacy-sensitive information (confidence: {result['score']:.2f})"
+            
+            return True, "Content is safe"
+        except Exception as e:
+            self.logger.error(f"Error during privacy classification: {e}")
+            # Default to safe on error
+            return True, f"Error during classification: {str(e)}"
+    
+    def check_content(self, text: str, check_nsfw: bool = True, check_privacy: bool = True, 
+                      threshold: float = 0.8) -> Tuple[bool, str]:
+        """
+        Check content for both NSFW and privacy concerns.
+        
+        Args:
+            text: The text to check
+            check_nsfw: Whether to check for NSFW content
+            check_privacy: Whether to check for privacy-sensitive content
+            threshold: Confidence threshold for detection
+        
+        Returns:
+            Tuple of (is_safe, message)
+        """
+        if check_nsfw:
+            is_safe, message = self.check_nsfw(text, threshold)
+            if not is_safe:
+                return False, f"NSFW: {message}"
+        
+        if check_privacy:
+            is_safe, message = self.check_privacy(text, threshold)
+            if not is_safe:
+                return False, f"Privacy: {message}"
+        
+        return True, "All content is safe"
+
+
+# Global classifier instance
+content_classifier = ContentClassifier()
\ No newline at end of file

aisbf/config.py

@@ -35,6 +35,17 @@ class ProviderModelConfig(BaseModel):
     rate_limit: Optional[float] = None
     max_request_tokens: Optional[int] = None
     error_cooldown: Optional[int] = None  # Cooldown period in seconds after 3 consecutive failures
+    # OpenRouter-style extended fields
+    description: Optional[str] = None
+    context_length: Optional[int] = None
+    architecture: Optional[Dict] = None  # modality, input_modalities, output_modalities, tokenizer, instruct_type
+    pricing: Optional[Dict] = None  # prompt, completion, input_cache_read
+    top_provider: Optional[Dict] = None  # context_length, max_completion_tokens, is_moderated
+    supported_parameters: Optional[List[str]] = None
+    default_parameters: Optional[Dict] = None
+    # Content classification flags
+    nsfw: bool = False  # Model can handle NSFW content
+    privacy: bool = False  # Model can handle privacy-sensitive content
 
 
 class CondensationConfig(BaseModel):
@@ -71,6 +82,17 @@ class RotationConfig(BaseModel):
     model_name: str
     providers: List[Dict]
     notifyerrors: bool = False
+    capabilities: Optional[List[str]] = None  # Capabilities for this rotation
+    # Content classification flags
+    nsfw: bool = False  # Model can handle NSFW content
+    privacy: bool = False  # Model can handle privacy-sensitive content
+    # OpenRouter-style extended fields
+    description: Optional[str] = None
+    context_length: Optional[int] = None
+    architecture: Optional[Dict] = None
+    pricing: Optional[Dict] = None
+    supported_parameters: Optional[List[str]] = None
+    default_parameters: Optional[Dict] = None
     # Default settings for models in this rotation
     default_rate_limit: Optional[float] = None
     default_max_request_tokens: Optional[int] = None
@@ -85,6 +107,8 @@ class RotationConfig(BaseModel):
 class AutoselectModelInfo(BaseModel):
     model_id: str
     description: str
+    nsfw: bool = False  # Model can handle NSFW content
+    privacy: bool = False  # Model can handle privacy-sensitive content
 
 class AutoselectConfig(BaseModel):
     model_name: str
@@ -92,6 +116,19 @@ class AutoselectConfig(BaseModel):
     selection_model: str = "general"
     fallback: str
     available_models: List[AutoselectModelInfo]
+    capabilities: Optional[List[str]] = None  # Capabilities for this autoselect configuration
+    # Content classification flags
+    nsfw: bool = False  # Model can handle NSFW content
+    privacy: bool = False  # Model can handle privacy-sensitive content
+    classify_nsfw: bool = False  # Enable NSFW classification for this autoselect
+    classify_privacy: bool = False  # Enable privacy classification for this autoselect
+    classify_semantic: bool = False  # Enable semantic classification for this autoselect
+    # OpenRouter-style extended fields
+    context_length: Optional[int] = None
+    architecture: Optional[Dict] = None
+    pricing: Optional[Dict] = None
+    supported_parameters: Optional[List[str]] = None
+    default_parameters: Optional[Dict] = None
 
 class TorConfig(BaseModel):
     """Configuration for TOR hidden service"""
@@ -104,6 +141,19 @@ class TorConfig(BaseModel):
     socks_port: int = 9050
     socks_host: str = "127.0.0.1"
 
+class AISBFConfig(BaseModel):
+    """Global AISBF configuration from aisbf.json"""
+    classify_nsfw: bool = False
+    classify_privacy: bool = False
+    classify_semantic: bool = False
+    server: Optional[Dict] = None
+    auth: Optional[Dict] = None
+    mcp: Optional[Dict] = None
+    dashboard: Optional[Dict] = None
+    internal_model: Optional[Dict] = None
+    tor: Optional[Dict] = None
+
+
 class AppConfig(BaseModel):
     providers: Dict[str, ProviderConfig]
     rotations: Dict[str, RotationConfig]
@@ -111,6 +161,7 @@ class AppConfig(BaseModel):
     condensation: Optional[CondensationConfig] = None
     error_tracking: Dict[str, Dict]
     tor: Optional[TorConfig] = None
+    aisbf: Optional[AISBFConfig] = None  # Global AISBF config
 
 class Config:
     def __init__(self):
@@ -129,6 +180,7 @@ class Config:
         self._load_autoselect()
         self._load_condensation()
         self._load_tor()
+        self._load_aisbf_config()
         self._initialize_error_tracking()
         self._log_configuration_summary()
 
@@ -277,6 +329,7 @@ class Config:
             logger.info(f"=== Config._load_rotations END ===")
 
     def _load_autoselect(self):
+        """Load autoselect configuration and build model embeddings for semantic matching."""
         import logging
         logger = logging.getLogger(__name__)
         logger.info(f"=== Config._load_autoselect START ===")
@@ -301,7 +354,151 @@ class Config:
             self.autoselect = {k: AutoselectConfig(**v) for k, v in data.items()}
             self._loaded_files['autoselect'] = str(autoselect_path.absolute())
             logger.info(f"Loaded {len(self.autoselect)} autoselect configurations: {list(self.autoselect.keys())}")
-            logger.info(f"=== Config._load_autoselect END ===")
+            
+            # Build and cache model embeddings for semantic matching
+            self._build_model_embeddings()
+            
+        logger.info(f"=== Config._load_autoselect END ===")
+    
+    def _build_model_embeddings(self):
+        """
+        Build and cache vectorized versions of model descriptions for semantic matching.
+        Saves embeddings to ~/.aisbf/ for persistent storage.
+        """
+        import logging
+        import numpy as np
+        logger = logging.getLogger(__name__)
+        
+        config_dir = Path.home() / '.aisbf'
+        vector_file = config_dir / 'model_embeddings.npy'
+        meta_file = config_dir / 'model_embeddings_meta.json'
+        
+        # Collect all model descriptions from all autoselect configs
+        model_library = {}
+        for autoselect_id, autoselect_config in self.autoselect.items():
+            for model_info in autoselect_config.available_models:
+                model_library[model_info.model_id] = model_info.description
+        
+        if not model_library:
+            logger.info("No models to vectorize")
+            self._model_embeddings = None
+            self._model_embeddings_meta = []
+            return
+        
+        # Check if embeddings file exists and is up-to-date
+        rebuild_needed = True
+        if vector_file.exists() and meta_file.exists():
+            try:
+                with open(meta_file) as f:
+                    saved_models = json.load(f)
+                if saved_models == list(model_library.keys()):
+                    logger.info(f"Loading cached model embeddings from {vector_file}")
+                    self._model_embeddings = np.load(vector_file)
+                    self._model_embeddings_meta = saved_models
+                    rebuild_needed = False
+                    logger.info(f"Loaded {len(self._model_embeddings)} model embeddings")
+            except Exception as e:
+                logger.warning(f"Could not load cached embeddings: {e}")
+        
+        if rebuild_needed:
+            logger.info(f"Building model embeddings for {len(model_library)} models...")
+            
+            try:
+                from sentence_transformers import SentenceTransformer
+                
+                # Use CPU-friendly model from config
+                model_id = "sentence-transformers/all-MiniLM-L6-v2"
+                
+                # Check if custom model is configured in aisbf.json
+                if self.aisbf and self.aisbf.internal_model:
+                    custom_model = self.aisbf.internal_model.get('semantic_vectorization')
+                    if custom_model:
+                        model_id = custom_model
+                
+                logger.info(f"Using embedding model: {model_id}")
+                embedder = SentenceTransformer(model_id)
+                
+                names = list(model_library.keys())
+                descriptions = list(model_library.values())
+                
+                logger.info(f"Vectorizing {len(names)} model descriptions on CPU...")
+                embeddings = embedder.encode(descriptions, show_progress_bar=True)
+                
+                # Save the vectors as binary file
+                np.save(vector_file, embeddings)
+                
+                # Save the names as JSON
+                with open(meta_file, 'w') as f:
+                    json.dump(names, f)
+                
+                self._model_embeddings = embeddings
+                self._model_embeddings_meta = names
+                
+                logger.info(f"Saved embeddings to {vector_file} and {meta_file}")
+                logger.info(f"Embedding shape: {embeddings.shape}")
+                
+            except ImportError as e:
+                logger.warning(f"sentence-transformers not installed, skipping embeddings: {e}")
+                self._model_embeddings = None
+                self._model_embeddings_meta = []
+            except Exception as e:
+                logger.warning(f"Failed to build model embeddings: {e}")
+                self._model_embeddings = None
+                self._model_embeddings_meta = []
+    
+    def find_similar_models(self, query: str, top_k: int = 3) -> List[str]:
+        """
+        Find the most similar models to a query based on embeddings.
+        
+        Args:
+            query: The user query/description to match against
+            top_k: Number of top matches to return
+            
+        Returns:
+            List of model_ids sorted by similarity (best match first)
+        """
+        import logging
+        logger = logging.getLogger(__name__)
+        
+        if self._model_embeddings is None or self._model_embeddings_meta is None:
+            logger.debug("No embeddings available, returning empty list")
+            return []
+        
+        try:
+            from sentence_transformers import SentenceTransformer
+            import numpy as np
+            
+            # Load embedder
+            model_id = "sentence-transformers/all-MiniLM-L6-v2"
+            if self.aisbf and self.aisbf.internal_model:
+                custom_model = self.aisbf.internal_model.get('semantic_vectorization')
+                if custom_model:
+                    model_id = custom_model
+            
+            embedder = SentenceTransformer(model_id)
+            
+            # Encode query
+            query_embedding = embedder.encode([query])
+            
+            # Calculate cosine similarity
+            query_norm = query_embedding / np.linalg.norm(query_embedding, axis=1, keepdims=True)
+            embeddings_norm = self._model_embeddings / np.linalg.norm(self._model_embeddings, axis=1, keepdims=True)
+            
+            # Compute similarities
+            similarities = np.dot(query_norm, embeddings_norm.T)[0]
+            
+            # Get top_k indices
+            top_indices = np.argsort(similarities)[::-1][:top_k]
+            
+            # Return model_ids in order of similarity
+            results = [self._model_embeddings_meta[i] for i in top_indices]
+            logger.debug(f"Found similar models: {results}")
+            
+            return results
+            
+        except Exception as e:
+            logger.warning(f"Error finding similar models: {e}")
+            return []
     
     def _load_condensation(self):
         """Load condensation configuration from providers.json"""
@@ -362,6 +559,34 @@ class Config:
             self._loaded_files['tor'] = str(aisbf_path.absolute())
             logger.info(f"Loaded TOR config: enabled={self.tor.enabled}, control_port={self.tor.control_port}, hidden_service_port={self.tor.hidden_service_port}")
             logger.info(f"=== Config._load_tor END ===")
+    
+    def _load_aisbf_config(self):
+        """Load global AISBF configuration from aisbf.json"""
+        import logging
+        logger = logging.getLogger(__name__)
+        logger.info(f"=== Config._load_aisbf_config START ===")
+        
+        aisbf_path = Path.home() / '.aisbf' / 'aisbf.json'
+        logger.info(f"Looking for AISBF config in: {aisbf_path}")
+        
+        if not aisbf_path.exists():
+            logger.info(f"User config not found, falling back to source config")
+            try:
+                source_dir = self._get_config_source_dir()
+                aisbf_path = source_dir / 'aisbf.json'
+                logger.info(f"Using source config at: {aisbf_path}")
+            except FileNotFoundError:
+                logger.warning("Could not find aisbf.json for AISBF config")
+                self.aisbf = AISBFConfig()
+                return
+        
+        logger.info(f"Loading AISBF config from: {aisbf_path}")
+        with open(aisbf_path) as f:
+            data = json.load(f)
+            self.aisbf = AISBFConfig(**data)
+            self._loaded_files['aisbf'] = str(aisbf_path.absolute())
+            logger.info(f"Loaded AISBF config: classify_nsfw={self.aisbf.classify_nsfw}, classify_privacy={self.aisbf.classify_privacy}")
+            logger.info(f"=== Config._load_aisbf_config END ===")
 
     def _initialize_error_tracking(self):
         self.error_tracking = {}
@@ -420,5 +645,8 @@ class Config:
     
     def get_tor(self) -> TorConfig:
         return self.tor
+    
+    def get_aisbf_config(self) -> AISBFConfig:
+        return self.aisbf
 
 config = Config()

aisbf/handlers.py

@@ -40,6 +40,8 @@ from .utils import (
     get_max_request_tokens_for_model
 )
 from .context import ContextManager, get_context_config_for_model
+from .classifier import content_classifier
+from .semantic_classifier import SemanticClassifier
 
 
 def generate_system_fingerprint(provider_id: str, seed: Optional[int] = None) -> str:
@@ -775,10 +777,16 @@ class RequestHandler:
             
             # Enhance model information with context window and capabilities
             enhanced_models = []
+            current_time = int(time_module.time())
             for model in models:
                 model_dict = model.dict()
                 model_name = model_dict.get('id', '')
                 
+                # Add OpenAI-compatible required fields
+                model_dict['object'] = 'model'
+                model_dict['created'] = current_time
+                model_dict['owned_by'] = provider_config.name
+                
                 # Try to find model config in provider config
                 model_config = None
                 if provider_config.models:
@@ -1511,7 +1519,9 @@ class RotationHandler:
                             'name': provider_model.name,
                             'weight': provider_weight,  # Use provider-level weight
                             'rate_limit': provider_model.rate_limit,
-                            'max_request_tokens': provider_model.max_request_tokens
+                            'max_request_tokens': provider_model.max_request_tokens,
+                            'nsfw': getattr(provider_model, 'nsfw', False),
+                            'privacy': getattr(provider_model, 'privacy', False)
                         }
                         rotation_models.append(model_dict)
                     logger.info(f"  Loaded {len(rotation_models)} model(s) from provider config with weight {provider_weight}")
@@ -1553,6 +1563,94 @@ class RotationHandler:
         logger.info(f"Total models considered: {total_models_considered}")
         logger.info(f"Total models available: {len(available_models)}")
         
+        # Apply NSFW/Privacy content classification filtering
+        # Only classify the immediate intent (last 3 messages + current query) to avoid
+        # huge context issues and long classification times
+        aisbf_config = self.config.get_aisbf_config()
+        if aisbf_config and (aisbf_config.classify_nsfw or aisbf_config.classify_privacy):
+            logger.info(f"=== CONTENT CLASSIFICATION FILTERING ===")
+            
+            # Get messages for classification - only last 3 user messages + current query
+            messages = request_data.get('messages', [])
+            
+            # Extract last 3 user messages for classification window
+            recent_user_messages = []
+            for msg in reversed(messages):
+                if msg.get('role') == 'user':
+                    content = msg.get('content', '')
+                    if isinstance(content, str):
+                        recent_user_messages.append(content)
+                        if len(recent_user_messages) >= 3:
+                            break
+            
+            # Build the classification prompt from recent messages only
+            # Reverse to get correct order (oldest first)
+            recent_user_messages.reverse()
+            user_prompt = " ".join(recent_user_messages)
+            
+            logger.info(f"Classifying only recent context ({len(recent_user_messages)} messages))")
+            logger.info(f"Recent context preview: {user_prompt[:200]}..." if len(user_prompt) > 200 else f"Recent context: {user_prompt}")
+            logger.info(f"Classify NSFW: {aisbf_config.classify_nsfw}")
+            logger.info(f"Classify privacy: {aisbf_config.classify_privacy}")
+            
+            # Check if content classification is needed
+            check_nsfw = aisbf_config.classify_nsfw
+            check_privacy = aisbf_config.classify_privacy
+            
+            if check_nsfw or check_privacy:
+                # Initialize classifier with models from config
+                internal_model_config = aisbf_config.internal_model or {}
+                nsfw_model = internal_model_config.get('nsfw_classifier')
+                privacy_model = internal_model_config.get('privacy_classifier')
+                
+                content_classifier.initialize(nsfw_model, privacy_model)
+                
+                # Check user prompt for NSFW/privacy content
+                is_safe, message = content_classifier.check_content(
+                    user_prompt, 
+                    check_nsfw=check_nsfw, 
+                    check_privacy=check_privacy
+                )
+                
+                logger.info(f"Content classification result: {message}")
+                
+                if not is_safe:
+                    # Content is flagged - filter to only nsfw=True or privacy=True models
+                    logger.info(f"Content flagged - filtering available models")
+                    
+                    if check_nsfw and not check_privacy:
+                        # Only NSFW filtering needed
+                        original_count = len(available_models)
+                        available_models = [m for m in available_models if m.get('nsfw', False)]
+                        logger.info(f"NSFW filtering: {original_count} -> {len(available_models)} models")
+                    elif check_privacy and not check_nsfw:
+                        # Only privacy filtering needed
+                        original_count = len(available_models)
+                        available_models = [m for m in available_models if m.get('privacy', False)]
+                        logger.info(f"Privacy filtering: {original_count} -> {len(available_models)} models")
+                    elif check_nsfw and check_privacy:
+                        # Both filtering - need models with EITHER flag
+                        original_count = len(available_models)
+                        available_models = [m for m in available_models if m.get('nsfw', False) or m.get('privacy', False)]
+                        logger.info(f"NSFW+Privacy filtering: {original_count} -> {len(available_models)} models")
+            
+            logger.info(f"=== CONTENT CLASSIFICATION FILTERING END ===")
+        
+        # Check if rotation-level nsfw/privacy flags also apply
+        rotation_nsfw = getattr(rotation_config, 'nsfw', False)
+        rotation_privacy = getattr(rotation_config, 'privacy', False)
+        
+        if rotation_nsfw or rotation_privacy:
+            logger.info(f"=== ROTATION-LEVEL CONTENT FLAGS ===")
+            logger.info(f"Rotation nsfw: {rotation_nsfw}, privacy: {rotation_privacy}")
+            
+            # If rotation explicitly allows NSFW content, keep models that can handle it
+            # If rotation explicitly allows Privacy content, keep models that can handle it
+            if rotation_nsfw:
+                logger.info(f"Rotation allows NSFW content - keeping models that support it")
+            if rotation_privacy:
+                logger.info(f"Rotation allows Privacy content - keeping models that support it")
+        
         if not available_models:
             logger.error("No models available in rotation (all providers may be rate limited)")
             logger.error("All providers in this rotation are currently deactivated")
@@ -2865,8 +2963,51 @@ class AutoselectHandler:
         import logging
         logger = logging.getLogger(__name__)
         logger.info(f"=== AUTOSELECT MODEL SELECTION START ===")
+
+        # Check if semantic classification is enabled
+        if autoselect_config.classify_semantic:
+            logger.info("=== SEMANTIC CLASSIFICATION ENABLED ===")
+            logger.info(f"Using semantic classification for model selection")
+
+            try:
+                # Initialize semantic classifier
+                semantic_classifier = SemanticClassifier()
+                semantic_classifier.initialize()
+
+                # Build model library for semantic search (model_id -> description)
+                model_library = {}
+                for model_info in autoselect_config.available_models:
+                    model_library[model_info.model_id] = model_info.description
+
+                # Extract recent chat history (last 3 messages)
+                # Split user_prompt into messages (it's formatted as "role: content\nrole: content\n...")
+                chat_history = []
+                if user_prompt:
+                    lines = user_prompt.strip().split('\n')
+                    for line in lines[-3:]:  # Last 3 messages
+                        if ': ' in line:
+                            role, content = line.split(': ', 1)
+                            chat_history.append(content)
+
+                # Perform hybrid BM25 + semantic re-ranking
+                results = semantic_classifier.hybrid_model_search(user_prompt, chat_history, model_library, top_k=1)
+
+                if results:
+                    selected_model_id, score = results[0]
+                    logger.info(f"=== SEMANTIC CLASSIFICATION SUCCESS ===")
+                    logger.info(f"Selected model ID: {selected_model_id} (score: {score:.4f})")
+                    return selected_model_id
+                else:
+                    logger.warning(f"=== SEMANTIC CLASSIFICATION FAILED ===")
+                    logger.warning("No models returned from semantic search, falling back to AI model selection")
+
+            except Exception as e:
+                logger.error(f"=== SEMANTIC CLASSIFICATION ERROR ===")
+                logger.error(f"Error during semantic classification: {str(e)}")
+                logger.warning("Falling back to AI model selection")
+
         logger.info(f"Using '{autoselect_config.selection_model}' for model selection")
-        
+
         # Build messages (system + user)
         messages = self._build_autoselect_messages(user_prompt, autoselect_config)
         

aisbf/mcp.py

@@ -22,6 +22,7 @@ Why did the programmer quit his job? Because he didn't get arrays!
 
 MCP Server for AISBF - Provides remote agent configuration capabilities.
 """
+import time
 import json
 import logging
 from typing import Dict, List, Optional, Any, Union
@@ -494,6 +495,13 @@ class MCPServer:
                 provider_models = await request_handler.handle_model_list(dummy_request, provider_id)
                 for model in provider_models:
                     model['id'] = f"{provider_id}/{model.get('id', '')}"
+                    # Ensure OpenAI-compatible required fields are present
+                    if 'object' not in model:
+                        model['object'] = 'model'
+                    if 'created' not in model:
+                        model['created'] = int(time.time())
+                    if 'owned_by' not in model:
+                        model['owned_by'] = provider_config.name
                     model['type'] = 'provider'
                     all_models.append(model)
             except Exception as e:
@@ -504,11 +512,12 @@ class MCPServer:
             all_models.append({
                 'id': f"rotation/{rotation_id}",
                 'object': 'model',
-                'created': 0,
+                'created': int(time.time()),
                 'owned_by': 'aisbf-rotation',
                 'type': 'rotation',
                 'rotation_id': rotation_id,
-                'model_name': rotation_config.model_name
+                'model_name': rotation_config.model_name,
+                'capabilities': getattr(rotation_config, 'capabilities', [])
             })
         
         # Add autoselect
@@ -516,12 +525,13 @@ class MCPServer:
             all_models.append({
                 'id': f"autoselect/{autoselect_id}",
                 'object': 'model',
-                'created': 0,
+                'created': int(time.time()),
                 'owned_by': 'aisbf-autoselect',
                 'type': 'autoselect',
                 'autoselect_id': autoselect_id,
                 'model_name': autoselect_config.model_name,
-                'description': autoselect_config.description
+                'description': autoselect_config.description,
+                'capabilities': getattr(autoselect_config, 'capabilities', [])
             })
         
         return {"models": all_models}

aisbf/semantic_classifier.py

+"""
+Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
+
+AISBF - AI Service Broker Framework || AI Should Be Free
+
+Semantic classification for model selection using hybrid BM25 + semantic re-ranking.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Semantic classifier for model selection.
+"""
+import logging
+import threading
+from typing import List, Dict, Optional, Tuple
+
+class SemanticClassifier:
+    """
+    Semantic classifier for model selection using hybrid BM25 + semantic re-ranking.
+    Uses BM25 for fast keyword search and semantic embeddings for re-ranking.
+    """
+    
+    _instance = None
+    _lock = threading.Lock()
+    
+    def __new__(cls):
+        if cls._instance is None:
+            with cls._lock:
+                if cls._instance is None:
+                    cls._instance = super().__new__(cls)
+                    cls._instance._initialized = False
+        return cls._instance
+    
+    def __init__(self):
+        if self._initialized:
+            return
+        
+        self._initialized = True
+        self._embedder = None
+        self._embedder_lock = threading.Lock()
+        self._model_name = None
+        self.logger = logging.getLogger(__name__)
+    
+    def initialize(self, model_name: Optional[str] = None):
+        """
+        Initialize the semantic embedder.
+        
+        Args:
+            model_name: HuggingFace model name for semantic embeddings
+        """
+        self._model_name = model_name or "sentence-transformers/all-MiniLM-L6-v2"
+        self._load_embedder()
+    
+    def _load_embedder(self):
+        """Load the semantic embedder model"""
+        try:
+            from sentence_transformers import SentenceTransformer
+            self.logger.info(f"Loading semantic embedder model: {self._model_name}")
+            self._embedder = SentenceTransformer(self._model_name)
+            self.logger.info("Semantic embedder loaded successfully")
+        except Exception as e:
+            self.logger.error(f"Failed to load semantic embedder: {e}")
+            self._embedder = None
+    
+    def hybrid_model_search(
+        self, 
+        query: str, 
+        chat_history: List[str],
+        model_library: Dict[str, str],
+        top_k: int = 3
+    ) -> List[Tuple[str, float]]:
+        """
+        Perform hybrid BM25 + semantic search to find the best matching models.
+        
+        Args:
+            query: The current user query
+            chat_history: Recent chat history (last 3 messages)
+            model_library: Dict of {model_id: description}
+            top_k: Number of top candidates to return
+        
+        Returns:
+            List of (model_id, score) tuples sorted by relevance
+        """
+        if self._embedder is None:
+            self.logger.warning("Semantic embedder not initialized, falling back to simple matching")
+            return [(list(model_library.keys())[0], 1.0)] if model_library else []
+        
+        try:
+            from rank_bm25 import BM25Okapi
+            from sentence_transformers import util
+            import numpy as np
+            
+            # STEP 1: Build active window (last 3 messages + current query)
+            active_window = " ".join(chat_history[-3:] + [query])
+            self.logger.debug(f"Active window: {len(active_window.split())} words")
+            
+            # STEP 2: BM25 keyword search on model descriptions
+            model_ids = list(model_library.keys())
+            descriptions = list(model_library.values())
+            
+            # Tokenize corpus for BM25
+            tokenized_corpus = [desc.lower().split() for desc in descriptions]
+            bm25 = BM25Okapi(tokenized_corpus)
+            
+            # Get BM25 scores for all models
+            tokenized_query = active_window.lower().split()
+            bm25_scores = bm25.get_scores(tokenized_query)
+            
+            # Get top candidates based on BM25 (limit to top_k * 2 for re-ranking)
+            num_candidates = min(len(model_ids), top_k * 2)
+            top_bm25_indices = np.argsort(bm25_scores)[::-1][:num_candidates]
+            
+            self.logger.debug(f"BM25 selected {len(top_bm25_indices)} candidates for re-ranking")
+            
+            # STEP 3: Semantic re-ranking of BM25 candidates
+            # Vectorize active window (intent)
+            intent_vector = self._embedder.encode([active_window], convert_to_tensor=True)
+            
+            # Vectorize only the candidate descriptions
+            candidate_descriptions = [descriptions[i] for i in top_bm25_indices]
+            candidate_vectors = self._embedder.encode(candidate_descriptions, convert_to_tensor=True)
+            
+            # Compute cosine similarity
+            cosine_scores = util.cos_sim(intent_vector, candidate_vectors)[0]
+            
+            # Get top_k from re-ranked candidates
+            top_semantic_indices = np.argsort(cosine_scores.cpu().numpy())[::-1][:top_k]
+            
+            # Build results with scores
+            results = []
+            for idx in top_semantic_indices:
+                original_idx = top_bm25_indices[idx]
+                model_id = model_ids[original_idx]
+                score = float(cosine_scores[idx])
+                results.append((model_id, score))
+                self.logger.debug(f"Model: {model_id}, Score: {score:.4f}")
+            
+            self.logger.info(f"Hybrid search completed: {len(results)} models ranked")
+            return results
+            
+        except ImportError as e:
+            self.logger.error(f"Missing dependencies for hybrid search: {e}")
+            self.logger.error("Please install: pip install rank-bm25 sentence-transformers")
+            # Fallback to first model
+            return [(list(model_library.keys())[0], 1.0)] if model_library else []
+        except Exception as e:
+            self.logger.error(f"Error during hybrid model search: {e}")
+            # Fallback to first model
+            return [(list(model_library.keys())[0], 1.0)] if model_library else []
+    
+    def select_best_model(
+        self,
+        query: str,
+        chat_history: List[str],
+        model_library: Dict[str, str]
+    ) -> Optional[str]:
+        """
+        Select the best model based on semantic similarity.
+        
+        Args:
+            query: The current user query
+            chat_history: Recent chat history
+            model_library: Dict of {model_id: description}
+        
+        Returns:
+            The best matching model_id or None
+        """
+        results = self.hybrid_model_search(query, chat_history, model_library, top_k=1)
+        if results:
+            best_model, score = results[0]
+            self.logger.info(f"Selected model: {best_model} (score: {score:.4f})")
+            return best_model
+        return None
+
+
+# Global semantic classifier instance
+semantic_classifier = SemanticClassifier()

config/aisbf.json

 {
+  "classify_nsfw": false,
+  "classify_privacy": false,
+  "classify_semantic": false,
   "server": {
     "host": "0.0.0.0",
     "port": 17765,
@@ -31,7 +34,10 @@
   },
   "internal_model": {
     "condensation_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
-    "autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3"
+    "autoselect_model_id": "huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3",
+    "nsfw_classifier": "michelleli99/NSFW_text_classifier",
+    "privacy_classifier": "iiiorg/piiranha-v1-detect-personal-information",
+    "semantic_vectorization": "sentence-transformers/all-MiniLM-L6-v2"
   },
   "tor": {
     "enabled": false,

config/autoselect.json

 {
   "autoselect": {
+    "nsfw": false,
+    "privacy": false,
+    "classify_nsfw": false,
+    "classify_privacy": false,
+    "classify_semantic": false,
     "model_name": "autoselect",
     "description": "Auto-selects the best rotating model based on user prompt analysis",
     "selection_model": "general",
     "fallback": "general",
+    "capabilities": ["t2t", "reasoning", "multimodal"],
     "available_models": [
       {
         "model_id": "coding",
+        "nsfw": false,
+        "privacy": false,
         "description": "Best for programming, code generation, debugging, and technical tasks. Optimized for software development, code reviews, and algorithm design."
       },
       {
         "model_id": "general",
+        "nsfw": false,
+        "privacy": false,
         "description": "General purpose model for everyday tasks, conversations, and general knowledge queries. Good for a wide range of topics including writing, analysis, and explanations."
       }
     ]

config/providers.json

@@ -17,6 +17,8 @@
       "models": [
         {
           "name": "gemini-2.0-flash",
+          "nsfw": false,
+          "privacy": false,
           "rate_limit": 0,
           "max_request_tokens": 1000000,
           "rate_limit_TPM": 15000,
@@ -29,6 +31,8 @@
         },
         {
           "name": "gemini-1.5-pro",
+          "nsfw": false,
+          "privacy": false,
           "rate_limit": 0,
           "max_request_tokens": 2000000,
           "rate_limit_TPM": 15000,
@@ -48,6 +52,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_OPENAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "anthropic": {
@@ -57,6 +63,8 @@
       "type": "anthropic",
       "api_key_required": true,
       "api_key": "YOUR_ANTHROPIC_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "ollama": {
@@ -65,6 +73,8 @@
       "endpoint": "http://localhost:11434",
       "type": "ollama",
       "api_key_required": false,
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "azure_openai": {
@@ -74,6 +84,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_AZURE_OPENAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "cohere": {
@@ -83,6 +95,8 @@
       "type": "cohere",
       "api_key_required": true,
       "api_key": "YOUR_COHERE_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "huggingface": {
@@ -92,6 +106,8 @@
       "type": "huggingface",
       "api_key_required": true,
       "api_key": "YOUR_HUGGINGFACE_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "replicate": {
@@ -101,6 +117,8 @@
       "type": "replicate",
       "api_key_required": true,
       "api_key": "YOUR_REPLICATE_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "togetherai": {
@@ -110,6 +128,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_TOGETHERAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "groq": {
@@ -119,6 +139,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_GROQ_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "mistralai": {
@@ -128,6 +150,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_MISTRALAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "stabilityai": {
@@ -137,6 +161,8 @@
       "type": "stabilityai",
       "api_key_required": true,
       "api_key": "YOUR_STABILITYAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "kilo": {
@@ -146,6 +172,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_KILO_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "perplexity": {
@@ -155,6 +183,8 @@
       "type": "openai",
       "api_key_required": true,
       "api_key": "YOUR_PERPLEXITY_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "poe": {
@@ -164,6 +194,8 @@
       "type": "poe",
       "api_key_required": true,
       "api_key": "YOUR_POE_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "lanai": {
@@ -173,6 +205,8 @@
       "type": "lanai",
       "api_key_required": true,
       "api_key": "YOUR_LANAI_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "amazon": {
@@ -182,6 +216,8 @@
       "type": "amazon",
       "api_key_required": true,
       "api_key": "YOUR_AMAZON_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "ibm": {
@@ -191,6 +227,8 @@
       "type": "ibm",
       "api_key_required": true,
       "api_key": "YOUR_IBM_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "microsoft": {
@@ -200,6 +238,8 @@
       "type": "microsoft",
       "api_key_required": true,
       "api_key": "YOUR_MICROSOFT_API_KEY",
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0
     },
     "kiro": {
@@ -208,6 +248,8 @@
       "endpoint": "https://q.us-east-1.amazonaws.com",
       "type": "kiro",
       "api_key_required": false,
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0,
       "kiro_config": {
         "_comment": "Uses Kiro IDE credentials (VS Code extension)",
@@ -221,6 +263,8 @@
       "endpoint": "https://q.us-east-1.amazonaws.com",
       "type": "kiro",
       "api_key_required": false,
+      "nsfw": false,
+      "privacy": false,
       "rate_limit": 0,
       "kiro_config": {
         "_comment": "Uses kiro-cli credentials (SQLite database)",

config/rotations.json

@@ -3,7 +3,10 @@
   "rotations": {
     "coding": {
       "model_name": "coding",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["code_generation", "code_completion", "reasoning"],
       "providers": [
         {
           "provider_id": "gemini",
@@ -84,7 +87,10 @@
     },
     "general": {
       "model_name": "general",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["t2t", "translation", "summarization", "question_answering"],
       "providers": [
         {
           "provider_id": "gemini",
@@ -120,7 +126,10 @@
     },
     "googletest": {
       "model_name": "googletest",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["code_generation", "testing", "code_completion"],
       "providers": [
         {
           "provider_id": "gemini",
@@ -141,7 +150,10 @@
     },
     "kiro-claude": {
       "model_name": "kiro-claude",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["t2t", "reasoning", "function_calling"],
       "providers": [
         {
           "provider_id": "kiro",
@@ -180,7 +192,10 @@
     "simple-gemini": {
       "_comment": "SIMPLE FORMAT: Specify provider_id only, models will be loaded from providers.json automatically",
       "model_name": "simple-gemini",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["t2t"],
       "providers": [
         {
           "provider_id": "gemini"
@@ -190,7 +205,10 @@
     "simple-openai": {
       "_comment": "SIMPLE FORMAT: Provider with custom weight, models loaded from providers.json",
       "model_name": "simple-openai",
+      "nsfw": false,
+      "privacy": false,
       "notifyerrors": false,
+      "capabilities": ["t2t"],
       "providers": [
         {
           "provider_id": "openai",
@@ -199,4 +217,4 @@
       ]
     }
   }
-}
\ No newline at end of file
+}

main.py

@@ -713,6 +713,8 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
         logger.debug(f"Skipping provider {provider_id}: Kiro credentials not available or invalid")
         return []
     
+    current_time = int(time.time())
+    
     # If provider has local model config, use it
     if hasattr(provider_config, 'models') and provider_config.models:
         models = []
@@ -721,13 +723,19 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
             models.append({
                 'id': model_id,
                 'object': 'model',
-                'created': int(time.time()),
+                'created': current_time,
                 'owned_by': provider_config.name,
                 'provider': provider_id,
                 'type': 'provider',
                 'model_name': model.name,
                 'context_size': getattr(model, 'context_size', None),
                 'capabilities': getattr(model, 'capabilities', []),
+                'description': getattr(model, 'description', None),
+                'architecture': getattr(model, 'architecture', None),
+                'pricing': getattr(model, 'pricing', None),
+                'top_provider': getattr(model, 'top_provider', None),
+                'supported_parameters': getattr(model, 'supported_parameters', None),
+                'default_parameters': getattr(model, 'default_parameters', None),
                 'source': 'local_config'
             })
         return models
@@ -739,11 +747,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
             # Cache is still fresh, use it
             cached_models = _model_cache[provider_id]
             if cached_models:  # Only return if we have actual models
-                # Add provider prefix to model IDs
+                # Add provider prefix to model IDs and ensure all required fields
                 models = []
                 for model in cached_models:
                     model_copy = model.copy()
                     model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
+                    # Ensure OpenAI-compatible required fields are present
+                    if 'object' not in model_copy:
+                        model_copy['object'] = 'model'
+                    if 'created' not in model_copy:
+                        model_copy['created'] = current_time
+                    if 'owned_by' not in model_copy:
+                        model_copy['owned_by'] = provider_config.name
                     model_copy['provider'] = provider_id
                     model_copy['type'] = 'provider'
                     model_copy['source'] = 'api_cache'
@@ -755,11 +770,18 @@ async def get_provider_models(provider_id: str, provider_config) -> list:
         try:
             fetched_models = await fetch_provider_models(provider_id)
             if fetched_models:
-                # Add provider prefix to model IDs
+                # Add provider prefix to model IDs and ensure all required fields
                 models = []
                 for model in fetched_models:
                     model_copy = model.copy()
                     model_copy['id'] = f"{provider_id}/{model.get('id', model.get('name', ''))}"
+                    # Ensure OpenAI-compatible required fields are present
+                    if 'object' not in model_copy:
+                        model_copy['object'] = 'model'
+                    if 'created' not in model_copy:
+                        model_copy['created'] = current_time
+                    if 'owned_by' not in model_copy:
+                        model_copy['owned_by'] = provider_config.name
                     model_copy['provider'] = provider_id
                     model_copy['type'] = 'provider'
                     model_copy['source'] = 'api_cache'
@@ -1887,7 +1909,8 @@ async def list_all_models(request: Request):
                 'owned_by': 'aisbf-rotation',
                 'type': 'rotation',
                 'rotation_id': rotation_id,
-                'model_name': rotation_config.model_name
+                'model_name': rotation_config.model_name,
+                'capabilities': getattr(rotation_config, 'capabilities', [])
             })
         except Exception as e:
             logger.warning(f"Error listing rotation {rotation_id}: {e}")
@@ -1903,7 +1926,8 @@ async def list_all_models(request: Request):
                 'type': 'autoselect',
                 'autoselect_id': autoselect_id,
                 'model_name': autoselect_config.model_name,
-                'description': autoselect_config.description
+                'description': autoselect_config.description,
+                'capabilities': getattr(autoselect_config, 'capabilities', [])
             })
         except Exception as e:
             logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
@@ -1936,7 +1960,8 @@ async def v1_list_all_models(request: Request):
                 'owned_by': 'aisbf-rotation',
                 'type': 'rotation',
                 'rotation_id': rotation_id,
-                'model_name': rotation_config.model_name
+                'model_name': rotation_config.model_name,
+                'capabilities': getattr(rotation_config, 'capabilities', [])
             })
         except Exception as e:
             logger.warning(f"Error listing rotation {rotation_id}: {e}")
@@ -1952,7 +1977,8 @@ async def v1_list_all_models(request: Request):
                 'type': 'autoselect',
                 'autoselect_id': autoselect_id,
                 'model_name': autoselect_config.model_name,
-                'description': autoselect_config.description
+                'description': autoselect_config.description,
+                'capabilities': getattr(autoselect_config, 'capabilities', [])
             })
         except Exception as e:
             logger.warning(f"Error listing autoselect {autoselect_id}: {e}")
@@ -2297,8 +2323,9 @@ async def list_rotation_models():
             for model in provider['models']:
                 all_models.append({
                     "id": f"{rotation_id}/{model['name']}",
-                    "name": rotation_id,  # Use rotation name as the model name for selection
+                    "name": rotation_id,
                     "object": "model",
+                    "created": int(time.time()),
                     "owned_by": provider['provider_id'],
                     "rotation_id": rotation_id,
                     "actual_model": model['name'],
@@ -2384,8 +2411,9 @@ async def list_autoselect_models():
         for model_info in autoselect_config.available_models:
             all_models.append({
                 "id": model_info.model_id,
-                "name": autoselect_id,  # Use autoselect name as the model name for selection
+                "name": autoselect_id,
                 "object": "model",
+                "created": int(time.time()),
                 "owned_by": "autoselect",
                 "autoselect_id": autoselect_id,
                 "description": model_info.description,

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "aisbf"
-version = "0.5.0"
+version = "0.6.0"
 description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations"
 readme = "README.md"
 license = "GPL-3.0-or-later"

requirements.txt

@@ -13,6 +13,8 @@ langchain-text-splitters
 tiktoken
 torch
 transformers
+sentence-transformers
+rank-bm25
 jinja2
 itsdangerous
 bs4

setup.py

@@ -49,7 +49,7 @@ class InstallCommand(_install):
 
 setup(
     name="aisbf",
-    version="0.5.0",
+    version="0.6.0",
     author="AISBF Contributors",
     author_email="stefy@nexlab.net",
     description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",
@@ -111,6 +111,7 @@ setup(
             'aisbf/kiro_models.py',
             'aisbf/kiro_parsers.py',
             'aisbf/kiro_utils.py',
+            'aisbf/semantic_classifier.py',
         ]),
         # Install dashboard templates
         ('share/aisbf/templates', [

templates/dashboard/autoselect.html

@@ -131,6 +131,49 @@ function renderAutoselectDetails(autoselectKey) {
             <input type="text" value="${autoselect.model_name}" onchange="updateAutoselect('${autoselectKey}', 'model_name', this.value)" required>
         </div>
         
+        <div class="form-group">
+            <label>Capabilities (comma-separated)</label>
+            <input type="text" value="${autoselect.capabilities ? autoselect.capabilities.join(', ') : ''}" onchange="updateAutoselectCapabilities('${autoselectKey}', this.value)" placeholder="e.g., t2t, reasoning, multimodal">
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${autoselect.nsfw ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'nsfw', this.checked)">
+                NSFW
+            </label>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${autoselect.privacy ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'privacy', this.checked)">
+                Privacy
+            </label>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${autoselect.classify_nsfw ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_nsfw', this.checked)">
+                Classify NSFW
+            </label>
+            <small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_nsfw setting for this autoselection</small>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${autoselect.classify_privacy ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_privacy', this.checked)">
+                Classify Privacy
+            </label>
+            <small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_privacy setting for this autoselection</small>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${autoselect.classify_semantic ? 'checked' : ''} onchange="updateAutoselect('${autoselectKey}', 'classify_semantic', this.checked)">
+                Classify Semantic
+            </label>
+            <small style="color: #a0a0a0; font-size: 12px; display: block; margin-top: 5px;">Override global classify_semantic setting for this autoselection</small>
+        </div>
+        
         <div class="form-group">
             <label>Description</label>
             <textarea onchange="updateAutoselect('${autoselectKey}', 'description', this.value)" style="min-height: 60px;">${autoselect.description || ''}</textarea>
@@ -201,6 +244,20 @@ function renderAutoselectModels(autoselectKey) {
                 <textarea onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'description', this.value)" style="min-height: 80px;" required>${model.description || ''}</textarea>
                 <small style="color: #666; font-size: 12px;">Be specific about when this model should be used (e.g., "Best for programming, code generation, debugging")</small>
             </div>
+            
+            <div class="form-group">
+                <label>
+                    <input type="checkbox" ${model.nsfw ? 'checked' : ''} onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'nsfw', this.checked)">
+                    NSFW
+                </label>
+            </div>
+            
+            <div class="form-group">
+                <label>
+                    <input type="checkbox" ${model.privacy ? 'checked' : ''} onchange="updateAutoselectModel('${autoselectKey}', ${index}, 'privacy', this.checked)">
+                    Privacy
+                </label>
+            </div>
         `;
         
         container.appendChild(modelDiv);
@@ -242,6 +299,18 @@ function updateAutoselect(key, field, value) {
     autoselectConfig[key][field] = value;
 }
 
+function updateAutoselectCapabilities(autoselectKey, value) {
+    const trimmed = value.trim();
+    if (!trimmed) {
+        autoselectConfig[autoselectKey].capabilities = null;
+        return;
+    }
+    
+    // Split by comma and clean up
+    autoselectConfig[autoselectKey].capabilities = 
+        trimmed.split(',').map(s => s.trim()).filter(s => s);
+}
+
 function addAutoselectModel(autoselectKey) {
     if (!autoselectConfig[autoselectKey].available_models) {
         autoselectConfig[autoselectKey].available_models = [];

templates/dashboard/providers.html

@@ -287,6 +287,20 @@ function renderProviderDetails(key) {
             <input type="text" value="${Array.isArray(provider.default_condense_method) ? provider.default_condense_method.join(', ') : (provider.default_condense_method || '')}" onchange="updateProviderCondenseMethod('${key}', this.value)" placeholder="e.g., semantic, conversational">
         </div>
         
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${provider.nsfw ? 'checked' : ''} onchange="updateProvider('${key}', 'nsfw', this.checked)">
+                NSFW
+            </label>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${provider.privacy ? 'checked' : ''} onchange="updateProvider('${key}', 'privacy', this.checked)">
+                Privacy
+            </label>
+        </div>
+        
         <h4 style="margin-top: 20px; margin-bottom: 10px;">Models</h4>
         <div id="models-${key}"></div>
         <button type="button" class="btn btn-secondary" onclick="addModel('${key}')" style="margin-top: 10px;">Add Model</button>
@@ -359,6 +373,20 @@ function renderModels(providerKey) {
                 <label>Condense Method (conversational, semantic, hierarchical, algorithmic)</label>
                 <input type="text" value="${Array.isArray(model.condense_method) ? model.condense_method.join(', ') : (model.condense_method || '')}" onchange="updateModelCondenseMethod('${providerKey}', ${index}, this.value)" placeholder="e.g., semantic, conversational">
             </div>
+            
+            <div class="form-group">
+                <label>
+                    <input type="checkbox" ${model.nsfw ? 'checked' : ''} onchange="updateModel('${providerKey}', ${index}, 'nsfw', this.checked)">
+                    NSFW
+                </label>
+            </div>
+            
+            <div class="form-group">
+                <label>
+                    <input type="checkbox" ${model.privacy ? 'checked' : ''} onchange="updateModel('${providerKey}', ${index}, 'privacy', this.checked)">
+                    Privacy
+                </label>
+            </div>
         `;
         
         container.appendChild(modelDiv);

templates/dashboard/rotations.html

@@ -113,6 +113,11 @@ function renderRotationDetails(rotationKey) {
             </label>
         </div>
         
+        <div class="form-group">
+            <label>Capabilities (comma-separated)</label>
+            <input type="text" value="${rotation.capabilities ? rotation.capabilities.join(', ') : ''}" onchange="updateRotationCapabilities('${rotationKey}', this.value)" placeholder="e.g., code_generation, t2t, reasoning">
+        </div>
+        
         <div class="form-group">
             <label>Default Rate Limit (seconds)</label>
             <input type="number" value="${rotation.default_rate_limit || ''}" onchange="updateRotation('${rotationKey}', 'default_rate_limit', this.value ? parseFloat(this.value) : null)" step="0.1" placeholder="Optional">
@@ -123,6 +128,20 @@ function renderRotationDetails(rotationKey) {
             <input type="number" value="${rotation.default_context_size || ''}" onchange="updateRotation('${rotationKey}', 'default_context_size', this.value ? parseInt(this.value) : null)" placeholder="Optional">
         </div>
         
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${rotation.nsfw ? 'checked' : ''} onchange="updateRotation('${rotationKey}', 'nsfw', this.checked)">
+                NSFW
+            </label>
+        </div>
+        
+        <div class="form-group">
+            <label>
+                <input type="checkbox" ${rotation.privacy ? 'checked' : ''} onchange="updateRotation('${rotationKey}', 'privacy', this.checked)">
+                Privacy
+            </label>
+        </div>
+        
         <h4 style="margin-top: 20px; margin-bottom: 10px;">Providers</h4>
         <div id="providers-${rotationKey}"></div>
         <button type="button" class="btn btn-secondary" onclick="addRotationProvider('${rotationKey}')" style="margin-top: 10px;">Add Provider</button>
@@ -257,6 +276,7 @@ function addRotation() {
     rotationsConfig.rotations[key] = {
         model_name: key,
         notifyerrors: false,
+        capabilities: [],
         providers: []
     };
     
@@ -276,6 +296,18 @@ function updateRotation(key, field, value) {
     rotationsConfig.rotations[key][field] = value;
 }
 
+function updateRotationCapabilities(rotationKey, value) {
+    const trimmed = value.trim();
+    if (!trimmed) {
+        rotationsConfig.rotations[rotationKey].capabilities = null;
+        return;
+    }
+    
+    // Split by comma and clean up
+    rotationsConfig.rotations[rotationKey].capabilities = 
+        trimmed.split(',').map(s => s.trim()).filter(s => s);
+}
+
 function addRotationProvider(rotationKey) {
     if (!rotationsConfig.rotations[rotationKey].providers) {
         rotationsConfig.rotations[rotationKey].providers = [];

templates/dashboard/settings.html

@@ -132,6 +132,50 @@ along with this program.  If not, see <https://www.gnu.org/licenses/>.
         <small style="color: #666; display: block; margin-top: 5px;">Used when autoselect selection_model is set to "internal"</small>
     </div>
     
+    <div class="form-group">
+        <label for="nsfw_classifier">NSFW Classifier Model ID</label>
+        <input type="text" id="nsfw_classifier" name="nsfw_classifier" value="{{ config.internal_model.nsfw_classifier or 'michelleli99/NSFW_text_classifier' }}" required>
+        <small style="color: #666; display: block; margin-top: 5px;">Model used for NSFW content detection</small>
+    </div>
+    
+    <div class="form-group">
+        <label for="privacy_classifier">Privacy Classifier Model ID</label>
+        <input type="text" id="privacy_classifier" name="privacy_classifier" value="{{ config.internal_model.privacy_classifier or 'iiiorg/piiranha-v1-detect-personal-information' }}" required>
+        <small style="color: #666; display: block; margin-top: 5px;">Model used for privacy-sensitive information detection</small>
+    </div>
+    
+    <div class="form-group">
+        <label for="semantic_vectorization">Semantic Vectorization Model ID</label>
+        <input type="text" id="semantic_vectorization" name="semantic_vectorization" value="{{ config.internal_model.semantic_vectorization or 'sentence-transformers/all-MiniLM-L6-v2' }}" required>
+        <small style="color: #666; display: block; margin-top: 5px;">Model used for semantic embedding and vectorization</small>
+    </div>
+    
+    <h3 style="margin: 30px 0 20px;">Content Classification</h3>
+    
+    <div class="form-group">
+        <label>
+            <input type="checkbox" name="classify_nsfw" {% if config.classify_nsfw %}checked{% endif %}>
+            Enable NSFW Classification
+        </label>
+        <small style="color: #666; display: block; margin-top: 5px;">Enable automatic NSFW content detection for model selection</small>
+    </div>
+    
+    <div class="form-group">
+        <label>
+            <input type="checkbox" name="classify_privacy" {% if config.classify_privacy %}checked{% endif %}>
+            Enable Privacy Classification
+        </label>
+        <small style="color: #666; display: block; margin-top: 5px;">Enable automatic privacy-sensitive content detection for model selection</small>
+    </div>
+    
+    <div class="form-group">
+        <label>
+            <input type="checkbox" name="classify_semantic" {% if config.classify_semantic %}checked{% endif %}>
+            Enable Semantic Classification
+        </label>
+        <small style="color: #666; display: block; margin-top: 5px;">Enable semantic content classification using the configured model</small>
+    </div>
+    
     <h3 style="margin: 30px 0 20px;">TOR Hidden Service</h3>
     
     <div id="tor-status" style="margin-bottom: 20px; padding: 15px; background: #0f3460; border-radius: 6px; border-left: 4px solid #16213e;">