Add "shodan" last-resort discovery/failover provider (private)

A new special provider, type "shodan", routes OpenAI-compatible requests to a
pool of open model endpoints (Ollama/vLLM/llama.cpp/...) that are auto-discovered
and validated in the background, failing over across them transparently. It is
credential-free by design: client credentials are never forwarded to the
(untrusted) discovered upstreams.

Discovery/validation/live-registry/failover engine is vendored, essentially
verbatim, from Pasquale Minervini's open-router project (@pminervini,
https://github.com/pminervini/open-router, upstream commit 0378a08); full credit
for that engine is his. See aisbf/providers/shodan/open_router_engine.py for
provenance/attribution and docs/shodan-provider.md. AISBF uses only the engine
(not its aiohttp server/CLI layer) via a small adapter (runtime.py, handler.py).

Rotation integration: a new general per-entry flag "last_resort: true" holds an
entry aside and engages it ONLY when every normal provider in the rotation is
unavailable (disabled/rate-limited — the case that otherwise returns HTTP 429)
or has failed during the request's retry loop. The flag works for any provider
type, not just shodan. The special model name "auto" routes to whatever the
discovery pool currently advertises.

- aisbf/providers/shodan/: vendored engine + runtime adapter + handler
- aisbf/providers/__init__.py: register 'shodan'
- aisbf/config.py: ProviderConfig.shodan_config
- aisbf/handlers.py: last_resort partition/engagement + shodan SSE streaming
- requirements.txt: aiohttp>=3.14
- docs/shodan-provider.md: usage, config, security, credit
- tests/providers/test_shodan_provider.py: 21 tests (routing failover,
  model resolution, config build, last_resort partition, singleton)

Full suite: 141 passed (providers + routes + license).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

aisbf/config.py

@@ -125,6 +125,7 @@ class ProviderConfig(BaseModel):
     qwen_config: Optional[Dict] = None  # Optional Qwen-specific configuration - DEPRECATED
     coderai_config: Optional[Dict] = None  # Optional CoderAI-specific configuration
     runpod_config: Optional[Dict] = None  # Optional RunPod-specific configuration
+    shodan_config: Optional[Dict] = None  # Optional "shodan" last-resort discovery configuration
     # Default settings for models in this provider
     default_rate_limit: Optional[float] = None
     default_max_request_tokens: Optional[int] = None

aisbf/handlers.py

@@ -168,6 +168,20 @@ def _provider_in_availability_cooldown(handler) -> bool:
     return bool((failure_until and failure_until > now) or (usage_until and usage_until > now))
 
 
+def _partition_last_resort(models):
+    """Split rotation model entries into (normal, last_resort) by the
+    ``_last_resort`` tag set during the provider scan.
+
+    Last-resort entries (any provider marked ``last_resort: true`` in the
+    rotation, e.g. the discovery-backed 'shodan' provider) are held aside and
+    engaged only when every normal provider is unavailable or has failed —
+    preserving each list's original order.
+    """
+    normal = [m for m in models if not m.get('_last_resort')]
+    last_resort = [m for m in models if m.get('_last_resort')]
+    return normal, last_resort
+
+
 # Per-provider client rate limiting — a global sliding window (all clients
 # combined) that bounds how many requests are forwarded to a given provider,
 # protecting the upstream from hammering (e.g. a client exhausting a per-IP
@@ -3374,6 +3388,14 @@ class RotationHandler:
             logger.info(f"")
             logger.info(f"--- Processing provider: {provider_id} ---")
 
+            # A "last_resort" entry (any provider type, e.g. the discovery-backed
+            # 'shodan' provider) is held aside and engaged only when every normal
+            # provider in this rotation is unavailable or has failed. Its models
+            # are tagged here and partitioned out of the normal pool after the scan.
+            is_last_resort = bool(provider.get('last_resort'))
+            if is_last_resort:
+                logger.info(f"  [LAST RESORT] {provider_id} will be used only if all normal providers are unavailable")
+
             # Meta-provider: this entry forwards to another rotation/autoselect.
             # Each model 'name' is the id of the target rotation/autoselect.
             delegate_kind = self._delegation_kind(provider_id)
@@ -3405,6 +3427,7 @@ class RotationHandler:
                         'api_key': None,
                         '_delegate_kind': delegate_kind,
                         '_delegate_target': tname,
+                        '_last_resort': is_last_resort,
                     })
                     total_models_considered += 1
                     logger.info(f"  [DELEGATION] -> {delegate_kind} '{tname}' (weight {target_weight})")
@@ -3521,6 +3544,7 @@ class RotationHandler:
                 model_with_provider = model.copy()
                 model_with_provider['provider_id'] = provider_id
                 model_with_provider['api_key'] = api_key
+                model_with_provider['_last_resort'] = is_last_resort
                 available_models.append(model_with_provider)
 
         logger.info(f"")
@@ -3530,6 +3554,14 @@ class RotationHandler:
         if skipped_providers:
             logger.info(f"Skipped providers: {', '.join(skipped_providers)}")
         logger.info(f"Total models considered: {total_models_considered}")
+
+        # Partition last-resort models out of the normal pool. They are engaged
+        # only when the normal pool is empty (all providers unavailable) or has
+        # been fully exhausted during the retry loop below.
+        available_models, last_resort_models = _partition_last_resort(available_models)
+        last_resort_engaged = False
+        if last_resort_models:
+            logger.info(f"Last-resort models held aside: {len(last_resort_models)} (normal pool: {len(available_models)})")
         logger.info(f"Total models available: {len(available_models)}")
         
         # Apply NSFW/Privacy content classification filtering
@@ -3616,6 +3648,15 @@ class RotationHandler:
             if rotation_privacy:
                 logger.info(f"Rotation allows Privacy content - keeping models that support it")
         
+        if not available_models and last_resort_models:
+            logger.warning(
+                f"All normal providers in rotation '{rotation_id}' are unavailable; "
+                f"engaging {len(last_resort_models)} last-resort model(s)"
+            )
+            available_models = last_resort_models
+            last_resort_models = []
+            last_resort_engaged = True
+
         if not available_models:
             logger.error("No models available in rotation (all providers may be rate limited)")
             logger.error("All providers in this rotation are currently deactivated")
@@ -3778,7 +3819,21 @@ class RotationHandler:
             
             # Select a model that hasn't been tried yet, or retry a failed model with rate limiting
             remaining_models = [m for m in available_models if m not in tried_models]
-            
+
+            if not remaining_models:
+                # Normal pool exhausted — bring in the last-resort models now (once)
+                # so failover reaches them even when normal providers were initially
+                # available but all failed at request time.
+                if last_resort_models and not last_resort_engaged:
+                    logger.warning(
+                        f"All normal providers in rotation '{rotation_id}' have failed; "
+                        f"engaging {len(last_resort_models)} last-resort model(s)"
+                    )
+                    available_models = available_models + last_resort_models
+                    last_resort_models = []
+                    last_resort_engaged = True
+                    remaining_models = [m for m in available_models if m not in tried_models]
+
             if not remaining_models:
                 logger.error(f"No more models available to try")
                 logger.error(f"All {len(available_models)} models have been attempted")
@@ -4746,7 +4801,9 @@ class RotationHandler:
         # Check if this is a Google or Kilo provider based on configuration
         is_google_provider = provider_type == 'google'
         is_kilo_provider = provider_type in ('kilo', 'kilocode')
-        is_coderai_provider = provider_type == 'coderai'
+        # shodan relays raw OpenAI-compatible SSE bytes from a discovered upstream,
+        # exactly like the coderai/kilo pass-through path, so it shares that branch.
+        is_coderai_provider = provider_type in ('coderai', 'shodan')
         logger.info(f"Creating streaming response for provider type: {provider_type}, is_google: {is_google_provider}, is_kilo: {is_kilo_provider}, is_coderai: {is_coderai_provider}")
         
         # Generate system_fingerprint for this request

aisbf/providers/__init__.py

@@ -44,6 +44,7 @@ from .codex import CodexProviderHandler
 from .coderai import CoderAIProviderHandler
 from .qwen import QwenProviderHandler
 from .runpod import RunpodProviderHandler
+from .shodan import ShodanProviderHandler
 from .preconfigured_openai import (
     # Major inference API providers
     GroqProviderHandler,
@@ -143,6 +144,7 @@ PROVIDER_HANDLERS = {
     'coderai': CoderAIProviderHandler,
     'qwen': QwenProviderHandler,
     'runpod': RunpodProviderHandler,
+    'shodan': ShodanProviderHandler,
 
     # --- pre-configured OpenAI-compatible providers ---
     'groq': GroqProviderHandler,

aisbf/providers/shodan/__init__.py

+"""
+Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+
+AISBF - AI Service Broker Framework || AI Should Be Free
+
+"shodan" last-resort provider package.
+
+The discovery/validation/failover engine in ``open_router_engine.py`` is
+vendored from Pasquale Minervini's open-router project
+(https://github.com/pminervini/open-router); full credit for that engine is his.
+The AISBF-specific adapter (runtime.py, handler.py) wraps it as a provider.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
+from .handler import ShodanProviderHandler
+
+__all__ = ["ShodanProviderHandler"]

aisbf/providers/shodan/handler.py

+"""
+Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
+
+AISBF - AI Service Broker Framework || AI Should Be Free
+
+"shodan" provider handler — a credential-free, last-resort failover provider.
+
+It routes OpenAI-compatible requests to a pool of open model endpoints that are
+discovered and validated by the vendored open-router engine (see
+open_router_engine.py, by Pasquale Minervini / @pminervini). It is intended to
+be added to a rotation as a ``last_resort`` entry: the rotation engages it only
+when every normal provider is disabled or rate limited.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
+import logging
+from typing import Dict, List, Optional, Union
+
+from ..base import BaseProviderHandler
+from ...config import config
+from ...models import Model
+from .runtime import get_runtime
+
+logger = logging.getLogger(__name__)
+
+
+class ShodanProviderHandler(BaseProviderHandler):
+    """Last-resort provider backed by auto-discovered open model endpoints.
+
+    Credential-free by design: the underlying engine deliberately never forwards
+    client credentials to the (untrusted) discovered upstreams, and the provider
+    itself needs no API key.
+    """
+
+    def __init__(
+        self,
+        provider_id: str,
+        api_key: Optional[str] = None,
+        user_id: Optional[int] = None,
+        provider_config=None,
+    ):
+        self.provider_config = (
+            provider_config if provider_config is not None else config.providers.get(provider_id)
+        )
+        super().__init__(provider_id, api_key, user_id=user_id)
+
+    def _shodan_config(self) -> Dict:
+        pc = self.provider_config
+        if pc is None:
+            return {}
+        if isinstance(pc, dict):
+            return pc.get("shodan_config") or {}
+        return getattr(pc, "shodan_config", None) or {}
+
+    def validate_credentials(self) -> bool:
+        # No credentials are used or required — the provider is always "valid".
+        return True
+
+    async def handle_request(
+        self,
+        model: str,
+        messages: List[Dict],
+        max_tokens: Optional[int] = None,
+        temperature: Optional[float] = 1.0,
+        stream: Optional[bool] = False,
+        tools: Optional[List[Dict]] = None,
+        tool_choice: Optional[Union[str, Dict]] = None,
+    ) -> Union[Dict, object]:
+        if self.is_rate_limited():
+            raise Exception("Provider rate limited")
+
+        await self.apply_rate_limit()
+
+        body: Dict = {
+            "model": model,
+            "messages": messages,
+            "stream": bool(stream),
+        }
+        if max_tokens is not None:
+            body["max_tokens"] = max_tokens
+        if temperature is not None:
+            body["temperature"] = temperature
+        if tools is not None:
+            body["tools"] = tools
+        if tool_choice is not None:
+            body["tool_choice"] = tool_choice
+
+        try:
+            runtime = await get_runtime(self._shodan_config())
+            result = await runtime.route("chat/completions", body, bool(stream))
+        except Exception as e:
+            logger.error("ShodanProviderHandler: request failed: %s", e)
+            self.record_failure()
+            raise
+
+        # Streaming returns a lazy async generator; the rotation primes and
+        # consumes it and records success itself (mirrors the OpenAI handler).
+        if not stream:
+            self.record_success()
+        return result
+
+    async def get_models(self) -> List[Model]:
+        try:
+            runtime = await get_runtime(self._shodan_config())
+            model_ids = await runtime.available_models()
+        except Exception as e:
+            logger.error("ShodanProviderHandler: get_models failed: %s", e)
+            raise
+        return [
+            Model(id=mid, name=mid, provider_id=self.provider_id)
+            for mid in model_ids
+        ]

docs/shodan-provider.md

+# Shodan last-resort provider
+
+The `shodan` provider is a **credential-free, last-resort failover provider**. It
+routes OpenAI-compatible requests to a pool of open model endpoints (Ollama,
+vLLM, llama.cpp, LM Studio, …) that are automatically discovered and validated in
+the background. It is meant to be added to a rotation as a `last_resort` entry so
+that the rotation can still answer when every normal provider is disabled or
+rate limited.
+
+## Credit
+
+The discovery, validation, live-registry, and failover-routing **engine** is
+vendored, essentially verbatim, from the **open-router** project by
+**Pasquale Minervini (@pminervini)**:
+
+> https://github.com/pminervini/open-router
+
+Full credit for that engine belongs to Pasquale Minervini. It lives at
+`aisbf/providers/shodan/open_router_engine.py` with provenance and attribution
+in its header, and is kept in sync with upstream rather than edited locally. The
+upstream repository carries no LICENSE file; it is included here with
+attribution and is **not** relicensed under AISBF's GPL. Only the engine (not its
+aiohttp server / CLI layer) is used; AISBF wraps it with a small adapter
+(`runtime.py`, `handler.py`).
+
+## How it works
+
+1. On first use, a single process-wide runtime starts the engine's background
+   discovery + health loops (Shodan facet queries and/or masscan list imports).
+2. Each candidate endpoint is probed and only published after it answers
+   `/v1/models`, advertises a text-generation model, and completes a minimal
+   Chat Completions and Responses request.
+3. A request routed to `shodan` is tried against the validated endpoints in
+   latency order, failing over transparently to the next endpoint on error.
+4. Discovered upstreams are always queried **without credentials** — client
+   `Authorization`/API-key/cookie headers are never forwarded.
+
+## Security note
+
+The embedded engine issues Shodan queries and (optionally) imports masscan scan
+results **from the host running AISBF**. Shodan/masscan results are untrusted
+Internet hosts. Only scan networks you own or are explicitly authorized to test.
+Prefer running discovery on a host and network where this is acceptable.
+
+## Configuring the provider
+
+Add a provider of `type: "shodan"`. It needs no API key. All discovery settings
+live under `shodan_config` and are optional (sensible defaults apply):
+
+```json
+{
+  "id": "shodan",
+  "name": "Shodan last-resort",
+  "type": "shodan",
+  "endpoint": "",
+  "api_key_required": false,
+  "rate_limit": 0,
+  "shodan_config": {
+    "shodan_mode": "public",
+    "shodan_api_key": null,
+    "shodan_max_results": null,
+    "masscan_lists": [],
+    "discovery_interval": 3600,
+    "health_interval": 300,
+    "blacklist_duration": 86400,
+    "workers": 8,
+    "probe_timeout": 5.0,
+    "readiness_timeout": 30.0,
+    "readiness_model_attempts": 3,
+    "upstream_timeout": 300.0
+  }
+}
+```
+
+`shodan_config` keys:
+
+| Key | Default | Meaning |
+| --- | --- | --- |
+| `shodan_mode` | `auto` | `public` (facet scrape, no key), `api` (needs `shodan_api_key`), or `auto` |
+| `shodan_api_key` | `null` | Shodan API key; only used in `api`/`auto` mode; sent only to Shodan |
+| `shodan_max_results` | `null` | Cap on Shodan API results |
+| `masscan_lists` | `[]` | Paths to masscan `list`-format outputs to import (port 11434) |
+| `discovery_interval` | `86400` | Seconds between discovery refreshes (`0` disables the loop) |
+| `health_interval` | `86400` | Seconds between liveness passes (`0` disables the loop) |
+| `blacklist_duration` | `86400` | Seconds a failing endpoint/model stays quarantined |
+| `workers` | `8` | Bounded concurrent probe queue size |
+| `upstream_timeout` | `300` | Per-request upstream socket timeout, in seconds |
+
+## Using it as a rotation last-resort
+
+Mark the rotation entry with `"last_resort": true`. The special `auto` model name
+means "use whatever the discovery pool currently advertises" (recommended, since
+discovered model names change over time):
+
+```json
+{
+  "model_name": "lisa",
+  "providers": [
+    { "provider_id": "codex",       "weight": 10, "enabled": true,
+      "models": [{ "name": "gpt-5.6-luna" }] },
+    { "provider_id": "kilo-stefy",  "weight": 3,  "enabled": true,
+      "models": [{ "name": "kilo-auto/free" }] },
+
+    { "provider_id": "shodan", "last_resort": true,
+      "models": [{ "name": "auto" }] }
+  ]
+}
+```
+
+Behavior:
+
+- While **any** normal provider (here `codex`, `kilo-stefy`) is available, the
+  `shodan` entry is held aside and never selected.
+- When **all** normal providers are unavailable (disabled in the rotation via
+  `enabled: false`, or in a failure/usage cooldown) — the exact case that
+  otherwise returns HTTP 429 — the rotation engages the `last_resort` entries
+  instead.
+- The last-resort pool is also engaged if all normal providers were initially
+  available but every one failed during the request's retry loop.
+
+`last_resort` is a general rotation-entry flag: any provider type can carry it,
+not just `shodan`.
+
+## Notes / limitations
+
+- Discovery warms up in the background; immediately after start the pool may be
+  empty and `shodan` will report no available upstream (the rotation then behaves
+  as if it had no last resort). Give discovery a little time, or seed it with
+  `masscan_lists`.
+- Because discovered upstreams are heterogeneous open servers, capabilities
+  (tools, long context, streaming quirks) vary. It is a best-effort safety net,
+  not a quality-guaranteed provider.

requirements.txt

@@ -43,3 +43,6 @@ mnemonic>=0.20
 bitcoinlib>=0.6.14
 web3>=6.0.0
 eth-account>=0.9.0
+
+# Required by the "shodan" last-resort provider discovery/failover engine
+aiohttp>=3.14