-
Your Name authored
- Add ResponseCache class with multiple backend support (memory, Redis, SQLite, MySQL) - Implement LRU eviction for memory backend with configurable max size - Add SHA256-based cache key generation for request deduplication - Implement TTL-based expiration (default: 600 seconds) - Add cache statistics tracking (hits, misses, hit rate, evictions) - Integrate caching into RequestHandler, RotationHandler, and AutoselectHandler - Add granular cache control at model, provider, rotation, and autoselect levels - Implement hierarchical configuration: Model > Provider > Rotation > Autoselect > Global - Add dashboard endpoints for cache statistics (/dashboard/response-cache/stats) and clearing (/dashboard/response-cache/clear) - Add response cache initialization in main.py startup event - Skip caching for streaming requests - Add comprehensive test suite (test_response_cache.py) with 6 test scenarios - Update configuration models with enable_response_cache fields - Update TODO.md to mark Response Caching as completed - Update CHANGELOG.md with response caching features Files created: - aisbf/response_cache.py (740+ lines) - test_response_cache.py (comprehensive test suite) Files modified: - aisbf/handlers.py (cache integration and _should_cache_response helper) - aisbf/config.py (ResponseCacheConfig and enable_response_cache fields) - config/aisbf.json (response_cache configuration section) - main.py (response cache initialization) - TODO.md (mark task as completed) - CHANGELOG.md (document new features)
f04ae15d