Fix Claude CLI mode: transport, real tool calling via MCP shim, dashboard toggle; bump to 0.99.85

CLI mode never worked. Four independent defects, any one fatal:

- The stdin frame used {"type":"user_message",...}; the CLI expects the
  Anthropic envelope {"type":"user","message":{...}} and silently discards
  anything else, so requests produced no output at all. With
  --input-format stream-json the CLI also ignores a prompt passed as an argv
  positional, so it must go over stdin.
- _handle_cli_streaming_request() took no tools parameter but was called with
  tools=..., raising TypeError on every streaming request.
- The event parser dispatched on a top-level content_block_delta/message_stop,
  but the CLI wraps Anthropic events as {"type":"stream_event","event":{...}},
  so those branches were dead code.
- Tool definitions were passed to --tools, which only selects built-in tools by
  name; a JSON blob there registers nothing.

Tools now reach the model through an MCP stdio shim (claude_mcp_shim.py), which
advertises the caller's definitions via tools/list. It never executes: in the
OpenAI protocol the client runs tools, so the first tool_use ends the turn and
is returned as tool_calls. Past calls/results are replayed as text since each
request is a fresh session, with a system-prompt directive so the model trusts
a replayed result instead of re-calling.

The shim forces one deviation from the intended flag set: --disallowedTools
'mcp__*' is a blanket deny that also blocks the shim, and deny beats
--allowedTools, so it cannot be kept alongside tool calling. It is retained
when no tools are requested; with tools, --strict-mcp-config preserves the same
isolation by loading only our config and ignoring the host's MCP servers.

Consequences of the event model (one assistant event per content block, not per
message): break on message_delta stop_reason == 'tool_use' rather than the first
tool_use, or parallel calls are dropped; dedupe text/arguments across the delta
and assistant paths; and emit sequential tool_call indices, since content-block
indices count text/thinking blocks and leave holes that break client-side
accumulation.

System messages now go to --system-prompt instead of being inlined as user text
the model could ignore. Non-streaming reports real token usage instead of zeros.

Also fix the dashboard toggle that made this unreachable: providers.py imported
_claude_cli_mode from startup, binding a copy of False at import time, while
detection only ever wrote app_state['_claude_cli_mode']. The template therefore
always received False and the use_cli_mode checkbox never rendered. Inject the
value through init() like every other route global, and drop the orphaned
startup global.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

aisbf/__init__.py

@@ -55,7 +55,7 @@ from .auth.qwen import QwenOAuth2
 from .handlers import RequestHandler, RotationHandler, AutoselectHandler
 from .utils import count_messages_tokens, split_messages_into_chunks, get_max_request_tokens_for_model, get_max_completion_tokens_for_model
 
-__version__ = "0.99.83"
+__version__ = "0.99.85"
 __all__ = [
     # Config
     "config",

aisbf/app/startup.py

@@ -28,7 +28,6 @@ _original_argv = None
 payment_service = None
 _initialized = False
 _server_ip_blocked: bool = False
-_claude_cli_mode = False
 _user_handlers_cache = {}
 tor_service = None
 _cache_refresh_task = None

aisbf/providers/claude_mcp_shim.py

+#!/usr/bin/env python3
+"""
+Copyleft (C) 2026 Stefy Lanza <stefy@nexlab.net>
+
+AISBF - AI Service Broker Framework || AI Should Be Free
+
+MCP stdio shim that exposes a caller's OpenAI/Anthropic tool definitions to the
+claude CLI, so the model emits real tool_use blocks instead of describing tool
+calls in prose.
+
+The claude CLI's --tools flag only selects built-in tools by name; there is no
+flag that registers arbitrary function definitions.  MCP is the only injection
+point, so this server advertises the caller's tools via tools/list.
+
+It never executes anything.  In the OpenAI protocol the *client* runs tools and
+posts the results back on the next request, so the broker has no way to satisfy
+a call mid-request.  The provider terminates the CLI as soon as it sees a
+tool_use block and returns it to the client as tool_calls; tools/call is
+therefore normally never reached.  It is implemented anyway (returning a
+sentinel) so that a race — the CLI dispatching the call before we tear it down —
+fails loudly in the transcript rather than hanging the subprocess.
+
+Run as a standalone script; stdlib only, so it works under any interpreter the
+CLI can spawn.  Tool definitions are read from the JSON file named by
+AISBF_MCP_TOOLS_FILE.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
+import json
+import os
+import sys
+
+SERVER_NAME = 'aisbf'
+DEFAULT_PROTOCOL_VERSION = '2024-11-05'
+CALL_SENTINEL = (
+    'AISBF: this tool is executed by the calling client, not here. '
+    'The broker should have already returned this call to the client.'
+)
+
+
+def _log(message: str) -> None:
+    sys.stderr.write(f'[aisbf-mcp-shim] {message}\n')
+    sys.stderr.flush()
+
+
+def _send(msg: dict) -> None:
+    sys.stdout.write(json.dumps(msg) + '\n')
+    sys.stdout.flush()
+
+
+def _load_tools() -> list:
+    """
+    Load tool definitions and convert them to the MCP tools/list schema.
+
+    Accepts Anthropic-style (input_schema) or MCP-style (inputSchema) entries.
+    """
+    path = os.environ.get('AISBF_MCP_TOOLS_FILE', '')
+    if not path:
+        _log('AISBF_MCP_TOOLS_FILE not set; serving no tools')
+        return []
+    try:
+        with open(path, 'r') as fh:
+            raw = json.load(fh)
+    except Exception as exc:
+        _log(f'failed to read {path}: {exc}')
+        return []
+
+    tools = []
+    for entry in raw if isinstance(raw, list) else []:
+        if not isinstance(entry, dict):
+            continue
+        name = entry.get('name')
+        if not name:
+            continue
+        schema = entry.get('inputSchema') or entry.get('input_schema') or {}
+        # MCP requires an object schema; a bare/empty schema is rejected by
+        # some clients, so normalise to a valid empty object schema.
+        if not isinstance(schema, dict) or schema.get('type') != 'object':
+            schema = {'type': 'object', 'properties': {}}
+        tools.append({
+            'name': name,
+            'description': entry.get('description', '') or '',
+            'inputSchema': schema,
+        })
+    _log(f'serving {len(tools)} tool(s): {[t["name"] for t in tools]}')
+    return tools
+
+
+def main() -> None:
+    tools = _load_tools()
+
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            req = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+
+        method = req.get('method')
+        req_id = req.get('id')
+
+        # Notifications carry no id and require no response.
+        if req_id is None:
+            continue
+
+        if method == 'initialize':
+            params = req.get('params') or {}
+            protocol = params.get('protocolVersion') or DEFAULT_PROTOCOL_VERSION
+            _send({'jsonrpc': '2.0', 'id': req_id, 'result': {
+                'protocolVersion': protocol,
+                'capabilities': {'tools': {'listChanged': False}},
+                'serverInfo': {'name': SERVER_NAME, 'version': '1.0.0'},
+            }})
+        elif method == 'tools/list':
+            _send({'jsonrpc': '2.0', 'id': req_id,
+                   'result': {'tools': tools}})
+        elif method == 'tools/call':
+            params = req.get('params') or {}
+            _log(f'unexpected tools/call for {params.get("name")!r} '
+                 f'- broker should have terminated the turn first')
+            _send({'jsonrpc': '2.0', 'id': req_id, 'result': {
+                'content': [{'type': 'text', 'text': CALL_SENTINEL}],
+                'isError': True,
+            }})
+        elif method == 'ping':
+            _send({'jsonrpc': '2.0', 'id': req_id, 'result': {}})
+        else:
+            _send({'jsonrpc': '2.0', 'id': req_id, 'error': {
+                'code': -32601, 'message': f'method not found: {method}',
+            }})
+
+
+if __name__ == '__main__':
+    main()

aisbf/routes/dashboard/providers.py

@@ -12,7 +12,7 @@ from aisbf.studio import build_studio_catalog, stamp_inferred_capabilities, seri
 from aisbf.studio_adapters import serialize_studio_adapter_choices, serialize_studio_adapter_profile_choices, effective_studio_adapter, infer_studio_adapter_profile
 from aisbf.studio_services import studio_service
 from aisbf.app.templates import url_for, get_base_url
-from aisbf.app.startup import _reload_global_config, _apply_condense_defaults_provider, _apply_condense_defaults_rotation, _providers_json_path, _rotations_json_path, _autoselect_json_path, _claude_cli_mode
+from aisbf.app.startup import _reload_global_config, _apply_condense_defaults_provider, _apply_condense_defaults_rotation, _providers_json_path, _rotations_json_path, _autoselect_json_path
 from aisbf.app.middleware import _is_local_client
 from aisbf.app.model_cache import fetch_provider_models
 from aisbf.routes.auth import require_dashboard_auth, require_api_auth, require_api_admin, require_admin
@@ -23,6 +23,9 @@ router = APIRouter()
 _config = None
 _templates = None
 _server_config = None
+# Injected by init(); imported from startup it would bind a copy of False at
+# import time, before detection has run.
+_claude_cli_mode = False
 
 logger = logging.getLogger(__name__)
 
@@ -309,11 +312,12 @@ def _resource_change_event(existing_config: dict | None, new_config: dict | None
         return f"{base_saved}_updated"
     return base_removed
 
-def init(config, templates, server_config=None):
-    global _config, _templates, _server_config
+def init(config, templates, server_config=None, claude_cli_mode=False):
+    global _config, _templates, _server_config, _claude_cli_mode
     _config = config
     _templates = templates
     _server_config = server_config
+    _claude_cli_mode = claude_cli_mode
 
 
 def _get_templates():

main.py

@@ -227,7 +227,8 @@ def _init_all_routers():
     _api_routes.init(config, _get_user_handler, _app_state['rotation_handler'])
     _mcp_routes.init(server_config, _get_user_handler)
     _user_api_routes.init(config, _get_user_handler)
-    _dash_providers.init(config, templates, server_config)
+    _dash_providers.init(config, templates, server_config,
+                         _app_state.get('_claude_cli_mode', False))
     _dash_settings.init(config, templates)
     _dash_admin.init(config, templates)
     _dash_payments.init(config, templates)

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "aisbf"
-version = "0.99.83"
+version = "0.99.85"
 description = "AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations"
 readme = "README.md"
 license = "GPL-3.0-or-later"

setup.py

@@ -106,7 +106,7 @@ class InstallCommand(_install):
 
 setup(
     name="aisbf",
-    version="0.99.83",
+    version="0.99.85",
     author="AISBF Contributors",
     author_email="stefy@nexlab.net",
     description="AISBF - AI Service Broker Framework || AI Should Be Free - A modular proxy server for managing multiple AI provider integrations",