- 09 Feb, 2026 1 commit
-
-
Stefy Lanza (nextime / spora ) authored
Fix Google provider tool formatting - pass function declarations directly instead of wrapping in dict
-
- 08 Feb, 2026 39 commits
-
-
Stefy Lanza (nextime / spora ) authored
Changed line 511 to check list length instead of boolean evaluation. This fixes the bug where tool calls extracted from Google chunks were not being sent to clients because empty lists were being treated as falsy. Before: 'tool_calls': delta_tool_calls if delta_tool_calls else None After: 'tool_calls': delta_tool_calls if len(delta_tool_calls) > 0 else None
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
- Added extraction of function_call from Google chunk parts - Convert Google function_call to OpenAI-compatible tool_calls format - Track accumulated_tool_calls to calculate deltas - Include tool_calls in the delta when present - Send chunks when there are new tool calls, not just text
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
- Removed stray 'd' identifier on line 494 - Fixed indentation issues on lines 494 and 537 - File now compiles without errors
-
Stefy Lanza (nextime / spora ) authored
- Fixed tool conversion to create single function_declarations array - This prevents Google from detecting malformed tool use - Tools are now properly converted from OpenAI to Google format
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Improve error message formatting by replacing semicolon separators with newlines for better readability
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
- Detect and unwrap responses wrapped in 'assistant: [{"type": "text", "text": "..."}]' format - Use extracted text for response content instead of raw accumulated text - Fix variable scoping issue with tool_match variable - Update token counting to use final_text when available -
Stefy Lanza (nextime / spora ) authored
Instead of collecting all chunks and sending a modified response: - Stream chunks normally as they come (with deltas like before) - Only at the END, if tool call pattern detected, send additional chunk with tool_calls - Then send final chunk with usage statistics This preserves the original streaming behavior while adding tool call detection.
-
Stefy Lanza (nextime / spora ) authored
When the model returns a response in the format: assistant: [{'type': 'text', 'text': '...'}] but without a tool call, extract just the text content instead of returning the raw wrapper format. -
Stefy Lanza (nextime / spora ) authored
The model returns literal \n (backslash-n) instead of actual newlines. This breaks JSON parsing because {\n is not valid JSON syntax. Use codecs.decode with 'unicode_escape' to convert escape sequences to actual characters before parsing. -
Stefy Lanza (nextime / spora ) authored
- Log accumulated response text (first 500 and last 200 chars) - Log extracted tool JSON with length and byte details - Log ASCII codes for first 20 chars to detect encoding issues - Log JSON parse errors with position details - Log success/failure of JSON parsing attempts
-
Stefy Lanza (nextime / spora ) authored
- Use brace counting for robust JSON extraction - Try JSON first, then fix common issues (single quotes, trailing commas) - Extract final assistant text using regex after tool JSON - Remove complex nested parsing that was failing with escaped quotes
-
Stefy Lanza (nextime / spora ) authored
- Models may return single quotes instead of double quotes - Fall back to ast.literal_eval when JSON parsing fails - Handle both JSON and Python-style literals in streaming responses
-
Stefy Lanza (nextime / spora ) authored
- Detect tool calls in accumulated streaming text after all chunks received - Parse nested 'assistant: [...]' format with tool calls inside - Parse simple 'tool: {...}' format - Convert detected tool calls to OpenAI-compatible format - Send tool_calls in first chunk, then final assistant text - Proper handling of finish_reason in final chunk -
Stefy Lanza (nextime / spora ) authored
- Detect when entire response is wrapped in 'assistant: [...]' - Parse nested 'tool: {...}' inside the assistant text - Extract final assistant text from nested structure - Handle multi-line JSON content with proper brace counting - More robust parsing for complex nested formats -
Stefy Lanza (nextime / spora ) authored
- Detect '"content": "..." } assistant: [...]' pattern - Extract tool content and convert to write action - Extract assistant text from JSON array - Handle multi-line content with newlines - More robust tool call detection for various text formats
-
Stefy Lanza (nextime / spora ) authored
- Detect 'tool: {...}' pattern in Google model text responses - Parse and convert to OpenAI-compatible tool_calls format - Extract assistant text from 'assistant: [...]' format if present - Handle both 'action' and 'name' fields for tool identification - Convert arguments to JSON string for OpenAI compatibility This fixes issues where models return tool calls as text instead of using proper function_call attributes. -
Stefy Lanza (nextime / spora ) authored
- Google provider now yields raw chunk objects instead of pre-formatted SSE bytes - The handlers.py handles the conversion to OpenAI-compatible format - This fixes the issue where clients weren't receiving streaming responses Note: Server must be restarted to pick up this change
-
Stefy Lanza (nextime / spora ) authored
- Import count_messages_tokens from utils module - Fixes 'name count_messages_tokens is not defined' error in Google streaming handler
-
Stefy Lanza (nextime / spora ) authored
- Add 'condensation' section to providers.json for specifying dedicated provider/model - Add CondensationConfig model to config.py - Add _load_condensation() and get_condensation() methods - Update ContextManager to use dedicated condensation handler when configured - Update handlers to pass condensation config to ContextManager - Allows using smaller/faster model for context condensation operations This addresses the issue where conversational and semantic condensation methods were using the same model as the main request, which was inefficient. Now users can configure a dedicated provider and model for condensation operations, typically using a smaller/faster model to reduce costs and improve performance.
-
Stefy Lanza (nextime / spora ) authored
- Accumulate all streaming chunks before parsing - Parse complete response at end of stream - Detect and convert tool calls from accumulated text content - Fixes issue where tool calls were returned as text instead of tool_calls structure
-
Stefy Lanza (nextime / spora ) authored
- Add logic to detect and parse tool calls from text content - Some models return tool calls as JSON in text instead of using function_call attribute - Handles both Google-style (action) and OpenAI-style (function/name) tool calls - Clears response_text when tool_calls are detected
-
Stefy Lanza (nextime / spora ) authored
- Changed all references from 'notify_errors' to 'notifyerrors' to match RotationConfig model - Fixes issue where notifyerrors setting was not being properly detected
-
Stefy Lanza (nextime / spora ) authored
- Add notifyerrors field with default value False - Fixes issue where notifyerrors was always detected as False - Allows rotation to return error as normal message instead of HTTP 503
-
Stefy Lanza (nextime / spora ) authored
- Get stream parameter from request_data to determine response type - Return StreamingResponse if original request was streaming - Return dict if original request was non-streaming - Fixes notifyerrors not working for streaming requests in retry exhausted case
-
Stefy Lanza (nextime / spora ) authored
- Add stream parameter to handle_rotation_request() - When notifyerrors is enabled, return StreamingResponse if original request was streaming - Return dict if original request was non-streaming - Fixes issue where autoselect handler expects StreamingResponse but was getting dict
-
Stefy Lanza (nextime / spora ) authored
- RotationConfig is a Pydantic model, not a dictionary - Use getattr() to safely access notifyerrors attribute - Fixes AttributeError when accessing rotation_config attributes
-
Stefy Lanza (nextime / spora ) authored
- Add 'notifyerrors' field to rotation configuration (default: false) - When enabled, return errors as normal messages instead of HTTP 503 - Allows clients to consume error messages normally without HTTP errors - Update handlers.py to check notifyerrors setting and return appropriate response
-