Improve error message formatting by replacing semicolon separators with newlines for better readability

- Detect and unwrap responses wrapped in 'assistant: [{"type": "text", "text": "..."}]' format
- Use extracted text for response content instead of raw accumulated text
- Fix variable scoping issue with tool_match variable
- Update token counting to use final_text when available

Instead of collecting all chunks and sending a modified response:
- Stream chunks normally as they come (with deltas like before)
- Only at the END, if tool call pattern detected, send additional chunk with tool_calls
- Then send final chunk with usage statistics

This preserves the original streaming behavior while adding tool call detection.

When the model returns a response in the format:
assistant: [{'type': 'text', 'text': '...'}]
but without a tool call, extract just the text content
instead of returning the raw wrapper format.

The model returns literal \n (backslash-n) instead of actual newlines.
This breaks JSON parsing because {\n is not valid JSON syntax.
Use codecs.decode with 'unicode_escape' to convert escape sequences
to actual characters before parsing.

- Log accumulated response text (first 500 and last 200 chars)
- Log extracted tool JSON with length and byte details
- Log ASCII codes for first 20 chars to detect encoding issues
- Log JSON parse errors with position details
- Log success/failure of JSON parsing attempts

- Use brace counting for robust JSON extraction
- Try JSON first, then fix common issues (single quotes, trailing commas)
- Extract final assistant text using regex after tool JSON
- Remove complex nested parsing that was failing with escaped quotes

- Models may return single quotes instead of double quotes
- Fall back to ast.literal_eval when JSON parsing fails
- Handle both JSON and Python-style literals in streaming responses

- Detect tool calls in accumulated streaming text after all chunks received
- Parse nested 'assistant: [...]' format with tool calls inside
- Parse simple 'tool: {...}' format
- Convert detected tool calls to OpenAI-compatible format
- Send tool_calls in first chunk, then final assistant text
- Proper handling of finish_reason in final chunk

- Detect when entire response is wrapped in 'assistant: [...]'
- Parse nested 'tool: {...}' inside the assistant text
- Extract final assistant text from nested structure
- Handle multi-line JSON content with proper brace counting
- More robust parsing for complex nested formats

- Detect '"content": "..." } assistant: [...]' pattern
- Extract tool content and convert to write action
- Extract assistant text from JSON array
- Handle multi-line content with newlines
- More robust tool call detection for various text formats

- Detect 'tool: {...}' pattern in Google model text responses
- Parse and convert to OpenAI-compatible tool_calls format
- Extract assistant text from 'assistant: [...]' format if present
- Handle both 'action' and 'name' fields for tool identification
- Convert arguments to JSON string for OpenAI compatibility

This fixes issues where models return tool calls as text instead of
using proper function_call attributes.

- Google provider now yields raw chunk objects instead of pre-formatted SSE bytes
- The handlers.py handles the conversion to OpenAI-compatible format
- This fixes the issue where clients weren't receiving streaming responses

Note: Server must be restarted to pick up this change

- Import count_messages_tokens from utils module
- Fixes 'name count_messages_tokens is not defined' error in Google streaming handler

- Add 'condensation' section to providers.json for specifying dedicated provider/model
- Add CondensationConfig model to config.py
- Add _load_condensation() and get_condensation() methods
- Update ContextManager to use dedicated condensation handler when configured
- Update handlers to pass condensation config to ContextManager
- Allows using smaller/faster model for context condensation operations

This addresses the issue where conversational and semantic condensation
methods were using the same model as the main request, which was
inefficient. Now users can configure a dedicated provider and model
for condensation operations, typically using a smaller/faster model to reduce
costs and improve performance.

- Accumulate all streaming chunks before parsing
- Parse complete response at end of stream
- Detect and convert tool calls from accumulated text content
- Fixes issue where tool calls were returned as text instead of tool_calls structure

- Add logic to detect and parse tool calls from text content
- Some models return tool calls as JSON in text instead of using function_call attribute
- Handles both Google-style (action) and OpenAI-style (function/name) tool calls
- Clears response_text when tool_calls are detected

- Changed all references from 'notify_errors' to 'notifyerrors' to match RotationConfig model
- Fixes issue where notifyerrors setting was not being properly detected

- Add notifyerrors field with default value False
- Fixes issue where notifyerrors was always detected as False
- Allows rotation to return error as normal message instead of HTTP 503

- Get stream parameter from request_data to determine response type
- Return StreamingResponse if original request was streaming
- Return dict if original request was non-streaming
- Fixes notifyerrors not working for streaming requests in retry exhausted case

- Add stream parameter to handle_rotation_request()
- When notifyerrors is enabled, return StreamingResponse if original request was streaming
- Return dict if original request was non-streaming
- Fixes issue where autoselect handler expects StreamingResponse but was getting dict

- RotationConfig is a Pydantic model, not a dictionary
- Use getattr() to safely access notifyerrors attribute
- Fixes AttributeError when accessing rotation_config attributes

- Add 'notifyerrors' field to rotation configuration (default: false)
- When enabled, return errors as normal messages instead of HTTP 503
- Allows clients to consume error messages normally without HTTP errors
- Update handlers.py to check notifyerrors setting and return appropriate response

- Add detailed error information including provider status
- Show cooldown remaining time for rate-limited providers
- Display failure counts for each provider
- Provide structured error response with rotation_id, attempted models, and details

- Move RotationHandler import inside method to avoid circular dependency
- Import RotationHandler only when needed in ContextManager.__init__

- Add 'condensation' section to providers.json for dedicated provider/model
- Support rotation-based condensation by specifying rotation ID in model field
- Update ContextManager to use dedicated condensation handler
- Update handlers to pass condensation configuration
- Bump version to 0.3.2

- Pass effective_context as parameter to stream_generator functions
- Update _create_streaming_response signature to accept effective_context
- Update all calls to _create_streaming_response to pass effective_context
- Track accumulated response text for token counting in streaming
- Calculate completion tokens for Google responses (since Google doesn't provide them)
- Calculate completion tokens for non-Google providers when they don't provide token counts
- Include prompt_tokens, completion_tokens, total_tokens, and effective_context in final chunk
- Fixes 'name effective_context is not defined' error in streaming responses
- Fixes issue where streaming responses had null token counts

- Enable PRAGMA journal_mode=WAL for better concurrent access
- Set PRAGMA busy_timeout=5000 (5 seconds) for concurrent access
- WAL mode allows multiple readers and one writer simultaneously

- Import initialize_database from aisbf.database
- Call initialize_database() in main() to create/recreate database
- Clean up old token usage records to prevent database bloat

- Create aisbf/database.py with DatabaseManager class
- Track context dimensions (context_size, condense_context, condense_method, effective_context)
- Track token usage for rate limiting (TPM, TPH, TPD)
- Auto-create database at ~/.aisbf/aisbf.db if it doesn't exist
- Clean up old token usage records to prevent database bloat
- Export database module in __init__.py
- Update setup.py to include database.py in package data

- Add context_size, condense_context, and condense_method fields to Model class
- Create new context.py module with ContextManager and condensation methods
- Implement hierarchical, conversational, semantic, and algoritmic condensation
- Calculate and report effective_context for all requests
- Update handlers.py to apply context condensation when configured
- Update providers.json and rotations.json with example context configurations
- Update README.md and DOCUMENTATION.md with context management documentation
- Export context module and utilities in __init__.py