fix: document security features and harden broker recovery

DOCUMENTATION.md

@@ -254,6 +254,56 @@ Important `runpod_config` fields:
 
 Dashboard support includes runtime refresh, status inspection, public catalog import, and persisted state tracking.
 
+## Security Filters and Prompt Analysis
+
+AISBF includes native request-time prompt analysis and content-safety controls that can be enabled globally and overridden at provider, rotation, autoselect, or model level.
+
+### Prompt-security controls
+
+The `prompt_security` feature group in `aisbf.json` controls:
+
+- `security_scan` - enables local prompt scanning before upstream execution
+- `context_lens` - enables prompt composition analytics and risk telemetry capture
+- `block_high_risk_prompts` - blocks requests whose local prompt analysis resolves to `high` risk
+- `persist_prompt_text` - stores raw prompt text when explicitly enabled
+- `redact_before_persist` - keeps redaction enabled before persistence to avoid storing sensitive content in plain text by default
+- `risk_threshold` - controls the blocking threshold, defaulting to `high`
+
+Default shipped posture:
+- prompt-security scanning: disabled
+- Context Lens analytics: disabled
+- block-high-risk prompts: disabled
+- persist raw prompt text: disabled
+- redact-before-persist: enabled
+
+### Request-time behavior
+
+When enabled, AISBF performs local prompt analysis before proxying the request upstream:
+
+- scans prompts using regex and heuristic detectors for suspicious prompt-injection and policy-evasion patterns
+- computes a risk level and aggregate risk score
+- builds a composition summary including prompt shape, dominant role, system-prompt presence, and tool usage posture
+- stores redacted summaries in prompt analytics tables for later dashboard inspection
+- can stop execution locally when `block_high_risk_prompts` is enabled and the resolved risk level is `high`
+
+### Content classification filters
+
+AISBF also supports request classification flags for:
+
+- NSFW-sensitive traffic via `enable_nsfw_classification`
+- privacy-sensitive traffic via `enable_privacy_classification`
+
+These controls can be applied on providers, rotations, autoselect configurations, and model entries so routing decisions can respect the sensitivity of the content being processed.
+
+### Dashboard visibility
+
+Prompt-security and analytics controls are exposed in the dashboard settings and resource editors:
+
+- global defaults can be configured from dashboard settings
+- provider/model editors expose tri-state overrides for prompt security and Context Lens analytics
+- rotation/autoselect editors expose inherited or explicit overrides for the same controls
+- prompt analysis results appear in the prompt analytics dashboard when scanning or Context Lens capture is enabled
+
 ## AISBF Studio
 
 AISBF Studio is the dashboard-native multimodal workspace exposed at `/dashboard/studio`.
@@ -307,6 +357,17 @@ AISBF includes a built-in marketplace for sharing configured resources between u
 - Settlement support for usage-based sharing
 - User export controls and market visibility filtering
 
+### Publishing model
+
+Listings can be created from:
+
+- full provider configurations
+- specific provider/model pairs
+- rotations
+- autoselect resources
+
+Each listing stores a sanitized configuration snapshot so secrets and local credential material are not exposed through the market export path.
+
 ### Imported references
 
 Users can import market listings as references instead of duplicating the underlying configuration.
@@ -317,6 +378,35 @@ Reference behavior:
 - Availability is tied to the source listing state
 - Dashboard UI clearly distinguishes imported resources from locally owned ones
 
+## CoderAI Integration
+
+AISBF supports CoderAI both as a directly reachable provider and as a brokered remote runtime.
+
+### Transport modes
+
+- direct HTTP mode for OpenAI-compatible endpoints
+- direct WebSocket bridge mode for framed request/response routing
+- broker mode for outbound-only NAT traversal where the CoderAI worker connects into AISBF
+
+### Broker features
+
+- provider-scoped registration tokens
+- global and user-scoped broker endpoints
+- persisted broker session snapshots in `~/.aisbf/coderai_broker_sessions.json`
+- dashboard session visibility with owner, client ID, transport, endpoint, Studio endpoints, and performance telemetry
+- Studio-native proxy forwarding for non-chat endpoints and long-running jobs
+
+### CoderAI dashboard workflow
+
+- create or edit a `coderai` provider
+- choose direct transport or broker mode
+- assign a stable `client_id`
+- generate or rotate the `registration_token`
+- connect the remote worker to the broker endpoint
+- inspect connection health and advertised capabilities from the providers page
+
+For protocol-level implementation details, see `docs/coderai-integration.md`.
+
 ## Wallet System
 
 AISBF includes a comprehensive unified wallet system that manages user fiat balances for subscription payments and provides multiple top-up methods.

README.md

@@ -78,6 +78,7 @@ The dashboard provides:
 - Provider configuration and API key management
 - RunPod runtime controls and CoderAI broker session monitoring
 - Rotation and autoselect model setup
+- Prompt security controls, Context Lens analytics, and NSFW/privacy routing filters
 - AISBF Studio multimodal workflows and pipeline bindings
 - User wallet management and top-up options
 - Token usage analytics, broker telemetry, and cost tracking
@@ -85,6 +86,27 @@ The dashboard provides:
 - SSL/TLS and TOR configuration
 - Multi-user administration
 
+## Featured Capabilities
+
+### Market
+
+- Publish providers, single models, rotations, and autoselect configurations to the built-in AISBF marketplace
+- Import published resources as locked references instead of cloning their full configuration locally
+- Track listing activity, usage settlement, revenue, and admin-side market visibility from the dashboard
+
+### CoderAI
+
+- Use `coderai` providers in direct HTTP mode, direct WebSocket bridge mode, or NAT-friendly broker mode
+- Register remote workers with provider-scoped tokens and inspect broker session status from the dashboard
+- Forward Studio-native endpoints over the CoderAI bridge so chat, multimodal, and long-running jobs can work through the same connection
+
+### Security Filters
+
+- Enable prompt-security scanning to detect suspicious prompt patterns before upstream execution
+- Enable Context Lens analytics to capture prompt composition metadata, risk summaries, and redacted evidence
+- Enable NSFW and privacy classification so AISBF can route or restrict traffic based on content sensitivity
+- Optionally block high-risk prompts and keep persisted prompt text disabled by default while redaction remains enabled
+
 ## API Usage
 
 ### Basic Chat Completion
@@ -119,6 +141,7 @@ For complete documentation, configuration guides, and API reference:
 - **[📚 Full Documentation](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md)** - Comprehensive user and developer guide
 - **[🔧 Installation Guide](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md#installation)** - Detailed setup instructions
 - **[⚙️ Configuration](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md#configuration)** - All configuration options
+- **[🛡️ Security Filters](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md#security-filters-and-prompt-analysis)** - Prompt security, Context Lens analytics, and content classification
 - **[🎛️ Studio Guide](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md#aisbf-studio)** - Multimodal Studio, bindings, and pipelines
 - **[🛒 Marketplace](https://git.nexlab.net/nexlab/aisbf/blob/master/DOCUMENTATION.md#marketplace-and-references)** - Publishing, imports, and settlements
 - **[🤖 CoderAI Broker](https://git.nexlab.net/nexlab/aisbf/blob/master/docs/coderai-integration.md)** - Broker protocol and integration reference

aisbf/coderai_broker.py

@@ -8,6 +8,7 @@ import asyncio
 from collections import deque
 import json
 import logging
+import os
 import time
 import uuid
 from dataclasses import dataclass, field
@@ -235,8 +236,28 @@ class CoderAIBroker:
             ],
             "updated_at": time.time(),
         }
-        with open(self._state_path, 'w') as f:
+        temp_path = self._state_path.with_suffix(self._state_path.suffix + '.tmp')
+        with open(temp_path, 'w') as f:
             json.dump(payload, f, indent=2)
+            f.flush()
+            os.fsync(f.fileno())
+        os.replace(temp_path, self._state_path)
+
+    def _quarantine_invalid_state_file(self) -> Optional[Path]:
+        if not self._state_path.exists():
+            return None
+        timestamp = int(time.time())
+        quarantine_path = self._state_path.with_name(f"{self._state_path.name}.corrupt.{timestamp}")
+        counter = 1
+        while quarantine_path.exists():
+            quarantine_path = self._state_path.with_name(f"{self._state_path.name}.corrupt.{timestamp}.{counter}")
+            counter += 1
+        try:
+            self._state_path.replace(quarantine_path)
+            return quarantine_path
+        except Exception:
+            logger.exception("Failed to quarantine invalid CoderAI broker session state file")
+            return None
 
     def _load_persisted_sessions(self) -> None:
         if not self._state_path.exists():
@@ -245,6 +266,14 @@ class CoderAIBroker:
             with open(self._state_path) as f:
                 payload = json.load(f)
         except Exception as e:
+            quarantine_path = self._quarantine_invalid_state_file()
+            if quarantine_path is not None:
+                logger.warning(
+                    "Failed to load persisted CoderAI broker sessions: %s. Moved invalid state file to %s",
+                    e,
+                    quarantine_path,
+                )
+            else:
                 logger.warning(f"Failed to load persisted CoderAI broker sessions: {e}")
             return
 

aisbf/database.py

@@ -2908,7 +2908,7 @@ class DatabaseManager:
             def _json_extract(column: str, path: str) -> str:
                 if self.db_type == 'sqlite':
                     return f"json_extract({column}, '$.{path}')"
-                return f"JSON_UNQUOTE(JSON_EXTRACT(CAST({column} AS JSON), '$.{path}'))"
+                return f"JSON_UNQUOTE(JSON_EXTRACT({column}, '$.{path}'))"
 
             role_expr = _json_extract('summary_json', 'composition.largest_segment_role')
             shape_expr = _json_extract('summary_json', 'composition.prompt_shape')

tests/test_coderai_broker.py

@@ -2,11 +2,12 @@ import asyncio
 import json
 import sys
 from pathlib import Path
+from types import SimpleNamespace
 
 import pytest
 from fastapi.testclient import TestClient
 
-from aisbf.coderai_broker import broker
+from aisbf.coderai_broker import broker, CoderAIBroker
 from aisbf.config import ProviderConfig, config
 from aisbf.database import DatabaseRegistry
 
@@ -330,3 +331,34 @@ def test_user_registration_rejects_global_provider_token(monkeypatch):
             config.providers.pop("coderai-global-only", None)
         else:
             config.providers["coderai-global-only"] = original_provider
+
+
+def test_load_persisted_sessions_quarantines_corrupt_state_file(tmp_path):
+    broker_instance = CoderAIBroker()
+    broker_instance._state_path = tmp_path / "coderai_broker_sessions.json"
+    broker_instance._state_path.write_text('{"sessions": [invalid]}', encoding="utf-8")
+    broker_instance._cache = SimpleNamespace(
+        broker_node_id=lambda: "node-test",
+        broker_set=lambda *args, **kwargs: None,
+        broker_get=lambda *args, **kwargs: None,
+        broker_delete=lambda *args, **kwargs: None,
+    )
+
+    broker_instance._load_persisted_sessions()
+
+    assert not broker_instance._state_path.exists()
+    quarantined = list(tmp_path.glob("coderai_broker_sessions.json.corrupt.*"))
+    assert len(quarantined) == 1
+
+
+def test_persist_sessions_locked_writes_state_atomically(tmp_path):
+    broker_instance = CoderAIBroker()
+    broker_instance._state_path = tmp_path / "coderai_broker_sessions.json"
+    broker_instance._sessions_by_id = {}
+
+    broker_instance._persist_sessions_locked()
+
+    assert broker_instance._state_path.exists()
+    assert not (tmp_path / "coderai_broker_sessions.json.tmp").exists()
+    payload = json.loads(broker_instance._state_path.read_text(encoding="utf-8"))
+    assert payload["sessions"] == []

tests/test_dashboard_proxy_events.py

@@ -346,6 +346,44 @@ def test_record_prompt_analysis_run_uses_mysql_last_insert_id_fallback():
     assert '"has_tools": true' in insert_params[-1]
 
 
+def test_prompt_analysis_details_mysql_json_extract_uses_text_column_directly():
+    from aisbf.database import DatabaseManager
+
+    class CursorStub:
+        def __init__(self):
+            self.executed = []
+            self._fetchall_calls = 0
+
+        def execute(self, sql, params=None):
+            self.executed.append((sql, params))
+
+        def fetchall(self):
+            self._fetchall_calls += 1
+            return []
+
+        def fetchone(self):
+            return (0, 0, 0, 0)
+
+    class ConnStub:
+        def __init__(self, cursor):
+            self._cursor = cursor
+
+        def cursor(self):
+            return self._cursor
+
+    manager = DatabaseManager.__new__(DatabaseManager)
+    manager.db_type = "mysql"
+    cursor = CursorStub()
+    manager._get_connection = lambda: MysqlConnCtxStub(ConnStub(cursor))
+
+    now = __import__("datetime").datetime.now()
+    manager.get_prompt_analysis_details(start=now, end=now)
+
+    executed_sql = "\n".join(sql for sql, _ in cursor.executed)
+    assert "JSON_UNQUOTE(JSON_EXTRACT(summary_json, '$.composition.largest_segment_role'))" in executed_sql
+    assert "CAST(summary_json AS JSON)" not in executed_sql
+
+
 @pytest.mark.asyncio
 async def test_handle_chat_completion_uses_cache_when_provider_override_enables_it(monkeypatch):
     request = RequestStub()