• Your Name's avatar
    feat: implement response caching with granular control · f04ae15d
    Your Name authored
    - Add ResponseCache class with multiple backend support (memory, Redis, SQLite, MySQL)
    - Implement LRU eviction for memory backend with configurable max size
    - Add SHA256-based cache key generation for request deduplication
    - Implement TTL-based expiration (default: 600 seconds)
    - Add cache statistics tracking (hits, misses, hit rate, evictions)
    - Integrate caching into RequestHandler, RotationHandler, and AutoselectHandler
    - Add granular cache control at model, provider, rotation, and autoselect levels
    - Implement hierarchical configuration: Model > Provider > Rotation > Autoselect > Global
    - Add dashboard endpoints for cache statistics (/dashboard/response-cache/stats) and clearing (/dashboard/response-cache/clear)
    - Add response cache initialization in main.py startup event
    - Skip caching for streaming requests
    - Add comprehensive test suite (test_response_cache.py) with 6 test scenarios
    - Update configuration models with enable_response_cache fields
    - Update TODO.md to mark Response Caching as completed
    - Update CHANGELOG.md with response caching features
    
    Files created:
    - aisbf/response_cache.py (740+ lines)
    - test_response_cache.py (comprehensive test suite)
    
    Files modified:
    - aisbf/handlers.py (cache integration and _should_cache_response helper)
    - aisbf/config.py (ResponseCacheConfig and enable_response_cache fields)
    - config/aisbf.json (response_cache configuration section)
    - main.py (response cache initialization)
    - TODO.md (mark task as completed)
    - CHANGELOG.md (document new features)
    f04ae15d
test_response_cache.py 10.4 KB