v0.4.0 - Configuration refactoring and autoselect enhancements

- Centralized API key storage in providers.json only
- Added support for provider-only rotation entries (auto-selects random model)
- Added default settings hierarchy at provider and rotation levels
- Limited autoselect selection context to 10 messages or 8000 tokens
- Added support for direct provider models in autoselect (rotation/provider/model)
- Added 'internal' keyword for local HuggingFace model selection
- Updated requirements.txt with torch and transformers

AI.PROMPT

@@ -331,6 +331,39 @@ When making changes:
 2. Add to `PROVIDER_HANDLERS` dictionary
 3. Add provider configuration to `config/providers.json`
 
+### Configuration Architecture
+
+**API Key Management:**
+- API keys are stored centrally in `config/providers.json`
+- Each provider definition includes an `api_key` field
+- Rotation and autoselect configurations reference providers by name only
+- The system automatically resolves API keys by matching provider names
+
+**Priority Order for API Keys:**
+1. API key from provider config (providers.json) - highest priority
+2. API key from rotation config (rotations.json) - fallback for backward compatibility
+
+**Model Selection in Rotations:**
+- If models are specified in rotation config: uses those models
+- If no models specified: randomly selects from provider's available models
+- Models from provider config are used with default weight of 1
+
+**Default Settings Hierarchy:**
+Settings can be specified at three levels with the following priority:
+1. Model-specific settings (highest priority)
+2. Rotation default settings
+3. Provider default settings (lowest priority)
+
+**Supported Default Fields:**
+- `default_rate_limit`: Rate limiting between requests
+- `default_max_request_tokens`: Maximum tokens per request
+- `default_rate_limit_TPM`: Tokens per minute limit
+- `default_rate_limit_TPH`: Tokens per hour limit
+- `default_rate_limit_TPD`: Tokens per day limit
+- `default_context_size`: Context window size
+- `default_condense_context`: Context condensation threshold
+- `default_condense_method`: Context condensation method
+
 ### Kiro Gateway Integration
 
 **Overview:**
@@ -353,12 +386,33 @@ Kiro Gateway is a third-party proxy gateway that provides OpenAI and Anthropic-c
 In `config/providers.json`:
 ```json
 {
+  "gemini": {
+    "id": "gemini",
+    "name": "Google AI Studio",
+    "endpoint": "https://generativelanguage.googleapis.com/v1beta",
+    "type": "google",
+    "api_key_required": true,
+    "api_key": "YOUR_GEMINI_API_KEY",
+    "rate_limit": 0,
+    "default_rate_limit": 0,
+    "default_max_request_tokens": 1000000,
+    "default_context_size": 1000000,
+    "models": [
+      {
+        "name": "gemini-2.0-flash",
+        "rate_limit": 0,
+        "max_request_tokens": 1000000,
+        "context_size": 1000000
+      }
+    ]
+  },
   "kiro": {
     "id": "kiro",
     "name": "Kiro Gateway (Amazon Q Developer)",
     "endpoint": "http://localhost:8000/v1",
     "type": "kiro",
     "api_key_required": true,
+    "api_key": "YOUR_KIRO_API_KEY",
     "rate_limit": 0,
     "models": [
       {
@@ -372,20 +426,39 @@ In `config/providers.json`:
 }
 ```
 
-In `config/rotations.json`:
+In `config/rotations.json` (API keys are now referenced from providers.json):
 ```json
 {
+  "coding": {
+    "model_name": "coding",
+    "notifyerrors": false,
+    "default_rate_limit": 0,
+    "default_context_size": 100000,
+    "providers": [
+      {
+        "provider_id": "gemini",
+        "models": [
+          {
+            "name": "gemini-2.0-flash",
+            "weight": 3,
+            "rate_limit": 0
+          }
+        ]
+      },
+      {
+        "provider_id": "openai"
+      }
+    ]
+  },
   "kiro-claude": {
     "model_name": "kiro-claude",
     "providers": [
       {
         "provider_id": "kiro",
-        "api_key": "YOUR_KIRO_GATEWAY_API_KEY",
         "models": [
           {
             "name": "claude-sonnet-4-5",
-            "weight": 3,
-            "rate_limit": 0
+            "weight": 3
           }
         ]
       }
@@ -394,6 +467,8 @@ In `config/rotations.json`:
 }
 ```
 
+**Note:** In the example above, the "openai" provider entry has no models specified, so the system will randomly select from all models defined in the provider's configuration.
+
 **Setup Requirements:**
 1. Kiro Gateway must be running (typically on `http://localhost:8000`)
 2. Kiro Gateway must be configured with valid Kiro credentials (IDE or CLI)
@@ -515,6 +590,20 @@ This AI.PROMPT file is automatically updated when significant changes are made t
 
 ### Recent Updates
 
+**2026-03-22 - Configuration Refactoring**
+- Centralized API key storage in providers.json
+- API keys are now stored only in provider definitions, not in rotation/autoselect configs
+- Rotation and autoselect configurations reference providers by name only
+- Added support for provider-only entries in rotations (no models specified)
+- When no models specified, system randomly selects from provider's available models
+- Added default settings support at provider level (default_rate_limit, default_max_request_tokens, etc.)
+- Added default settings support at rotation level
+- Settings priority: model-specific > rotation defaults > provider defaults
+- Updated ProviderConfig with default setting fields
+- Updated RotationConfig with default setting fields
+- Added _apply_defaults_to_model() method in RotationHandler
+- Updated configuration examples in AI.PROMPT
+
 **2026-02-07 - Version 0.2.7**
 - Added max_request_tokens support for automatic request splitting
 - Updated ProviderConfig to include optional models field with max_request_tokens

aisbf/config.py

@@ -53,10 +53,28 @@ class ProviderConfig(BaseModel):
     api_key: Optional[str] = None  # Optional API key in provider config
     models: Optional[List[ProviderModelConfig]] = None  # Optional list of models with their configs
     kiro_config: Optional[Dict] = None  # Optional Kiro-specific configuration (credentials, region, etc.)
+    # Default settings for models in this provider
+    default_rate_limit: Optional[float] = None
+    default_max_request_tokens: Optional[int] = None
+    default_rate_limit_TPM: Optional[int] = None
+    default_rate_limit_TPH: Optional[int] = None
+    default_rate_limit_TPD: Optional[int] = None
+    default_context_size: Optional[int] = None
+    default_condense_context: Optional[int] = None
+    default_condense_method: Optional[Union[str, List[str]]] = None
 
 class RotationConfig(BaseModel):
     providers: List[Dict]
     notifyerrors: bool = False
+    # Default settings for models in this rotation
+    default_rate_limit: Optional[float] = None
+    default_max_request_tokens: Optional[int] = None
+    default_rate_limit_TPM: Optional[int] = None
+    default_rate_limit_TPH: Optional[int] = None
+    default_rate_limit_TPD: Optional[int] = None
+    default_context_size: Optional[int] = None
+    default_condense_context: Optional[int] = None
+    default_condense_method: Optional[Union[str, List[str]]] = None
 
 class AutoselectModelInfo(BaseModel):
     model_id: str

config/providers.json

@@ -11,6 +11,7 @@
       "endpoint": "https://generativelanguage.googleapis.com/v1beta",
       "type": "google",
       "api_key_required": true,
+      "api_key": "YOUR_GEMINI_API_KEY",
       "rate_limit": 0,
       "models": [
         {
@@ -43,6 +44,7 @@
       "endpoint": "https://api.openai.com/v1",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_OPENAI_API_KEY",
       "rate_limit": 0
     },
     "anthropic": {
@@ -51,6 +53,7 @@
       "endpoint": "https://api.anthropic.com/v1",
       "type": "anthropic",
       "api_key_required": true,
+      "api_key": "YOUR_ANTHROPIC_API_KEY",
       "rate_limit": 0
     },
     "ollama": {
@@ -67,6 +70,7 @@
       "endpoint": "https://your-azure-endpoint.openai.azure.com",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_AZURE_OPENAI_API_KEY",
       "rate_limit": 0
     },
     "cohere": {
@@ -75,6 +79,7 @@
       "endpoint": "https://api.cohere.com/v1",
       "type": "cohere",
       "api_key_required": true,
+      "api_key": "YOUR_COHERE_API_KEY",
       "rate_limit": 0
     },
     "huggingface": {
@@ -83,6 +88,7 @@
       "endpoint": "https://api-inference.huggingface.co",
       "type": "huggingface",
       "api_key_required": true,
+      "api_key": "YOUR_HUGGINGFACE_API_KEY",
       "rate_limit": 0
     },
     "replicate": {
@@ -91,6 +97,7 @@
       "endpoint": "https://api.replicate.com/v1",
       "type": "replicate",
       "api_key_required": true,
+      "api_key": "YOUR_REPLICATE_API_KEY",
       "rate_limit": 0
     },
     "togetherai": {
@@ -99,6 +106,7 @@
       "endpoint": "https://api.together.xyz/v1",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_TOGETHERAI_API_KEY",
       "rate_limit": 0
     },
     "groq": {
@@ -107,6 +115,7 @@
       "endpoint": "https://api.groq.com/openai/v1",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_GROQ_API_KEY",
       "rate_limit": 0
     },
     "mistralai": {
@@ -115,6 +124,7 @@
       "endpoint": "https://api.mistral.ai/v1",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_MISTRALAI_API_KEY",
       "rate_limit": 0
     },
     "stabilityai": {
@@ -123,6 +133,7 @@
       "endpoint": "https://api.stability.ai/v2beta",
       "type": "stabilityai",
       "api_key_required": true,
+      "api_key": "YOUR_STABILITYAI_API_KEY",
       "rate_limit": 0
     },
     "kilo": {
@@ -131,6 +142,7 @@
       "endpoint": "https://kilocode.ai/api/openrouter",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_KILO_API_KEY",
       "rate_limit": 0
     },
     "perplexity": {
@@ -139,6 +151,7 @@
       "endpoint": "https://api.perplexity.ai",
       "type": "openai",
       "api_key_required": true,
+      "api_key": "YOUR_PERPLEXITY_API_KEY",
       "rate_limit": 0
     },
     "poe": {
@@ -147,6 +160,7 @@
       "endpoint": "https://api.poe.com/v1",
       "type": "poe",
       "api_key_required": true,
+      "api_key": "YOUR_POE_API_KEY",
       "rate_limit": 0
     },
     "lanai": {
@@ -155,6 +169,7 @@
       "endpoint": "https://api.lanai.ai/v1",
       "type": "lanai",
       "api_key_required": true,
+      "api_key": "YOUR_LANAI_API_KEY",
       "rate_limit": 0
     },
     "amazon": {
@@ -163,6 +178,7 @@
       "endpoint": "https://api.amazon.com/bedrock/v1",
       "type": "amazon",
       "api_key_required": true,
+      "api_key": "YOUR_AMAZON_API_KEY",
       "rate_limit": 0
     },
     "ibm": {
@@ -171,6 +187,7 @@
       "endpoint": "https://api.ibm.com/watson/v1",
       "type": "ibm",
       "api_key_required": true,
+      "api_key": "YOUR_IBM_API_KEY",
       "rate_limit": 0
     },
     "microsoft": {
@@ -179,6 +196,7 @@
       "endpoint": "https://api.microsoft.com/v1",
       "type": "microsoft",
       "api_key_required": true,
+      "api_key": "YOUR_MICROSOFT_API_KEY",
       "rate_limit": 0
     },
     "kiro": {

config/rotations.json

@@ -7,7 +7,6 @@
       "providers": [
         {
           "provider_id": "gemini",
-          "api_key": "YOUR_GEMINI_API_KEY",
           "models": [
             {
               "name": "gemini-2.0-flash",
@@ -37,7 +36,6 @@
         },
         {
           "provider_id": "openai",
-          "api_key": "YOUR_OPENAI_API_KEY",
           "models": [
             {
               "name": "gpt-4",
@@ -61,7 +59,6 @@
         },
         {
           "provider_id": "anthropic",
-          "api_key": "YOUR_ANTHROPIC_API_KEY",
           "models": [
             {
               "name": "claude-3-5-sonnet-20241022",
@@ -91,7 +88,6 @@
       "providers": [
         {
           "provider_id": "gemini",
-          "api_key": "YOUR_GEMINI_API_KEY",
           "models": [
             {
               "name": "gemini-1.5-pro",
@@ -107,7 +103,6 @@
         },
         {
           "provider_id": "openai",
-          "api_key": "YOUR_OPENAI_API_KEY",
           "models": [
             {
               "name": "gpt-4",
@@ -129,7 +124,6 @@
       "providers": [
         {
           "provider_id": "gemini",
-          "api_key": "YOUR_GEMINI_API_KEY",
           "models": [
             {
               "name": "gemini-2.0-flash",
@@ -151,7 +145,6 @@
       "providers": [
         {
           "provider_id": "kiro",
-          "api_key": "YOUR_KIRO_GATEWAY_API_KEY",
           "models": [
             {
               "name": "claude-sonnet-4-5",

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "aisbf"
-version = "0.3.3"
+version = "0.4.0"
 description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations"
 readme = "README.md"
 license = "GPL-3.0-or-later"

requirements.txt

@@ -10,4 +10,6 @@ google-genai
 openai
 anthropic
 langchain-text-splitters
-tiktoken
+tiktoken
\ No newline at end of file
+torch
+transformers
\ No newline at end of file

setup.py

@@ -49,7 +49,7 @@ class InstallCommand(_install):
 
 setup(
     name="aisbf",
-    version="0.3.3",
+    version="0.4.0",
     author="AISBF Contributors",
     author_email="stefy@nexlab.net",
     description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",

kiro-gateway @ e6f23c22

+Subproject commit e6f23c22fc5e9aa7a22e4c31af56cdc6f859afbd