Add reports synchronization feature with API endpoint and web interface

- Implement /api/reports/sync endpoint for client report data synchronization
- Add database models: ReportSync, Bet, BetDetail, ExtractionStats, ReportSyncLog
- Create web interface at /reports with filtering, pagination, and export (PDF/XLSX/CSV)
- Add sync logs interface at /sync-logs with comprehensive search and filtering
- Implement comprehensive logging for all sync operations
- Add migration 010 for existing systems to create all reports tables
- Update navigation menu with Reports and Sync Logs links
- Add weasyprint dependency for PDF generation
- Include test suite for reports sync API
- Add complete documentation in REPORTS_FEATURE_README.md

REPORTS_FEATURE_README.md

+# Reports Synchronization Feature
+
+## Overview
+
+This feature implements server-side support for synchronizing report data from MbetterClient applications to the server, along with a comprehensive web interface for viewing, filtering, and exporting reports.
+
+## Features Implemented
+
+### 1. Database Models
+
+Four new database models have been added to store report data:
+
+- **ReportSync**: Tracks each synchronization operation from clients
+  - Stores sync metadata (sync_id, client_id, timestamps)
+  - Stores summary statistics (total_payin, total_payout, net_profit, etc.)
+  - Links to bets and extraction stats
+
+- **Bet**: Individual bet records
+  - Unique UUID for deduplication
+  - Links to sync and client
+  - Stores bet details (amount, datetime, paid status)
+
+- **BetDetail**: Individual bet details within a bet
+  - Links to parent bet
+  - Stores match-specific information (match_id, outcome, amount, result)
+
+- **ExtractionStats**: Match extraction statistics
+  - Stores redistribution and payout data
+  - Includes CAP application details
+  - Stores result breakdown by outcome
+
+### 2. API Endpoint
+
+**POST /api/reports/sync**
+
+Accepts report data from clients with the following features:
+
+- **Authentication**: Requires valid API token (Bearer token)
+- **Validation**: Validates all required fields and data types
+- **Idempotency**: Handles duplicate sync_id gracefully
+- **Deduplication**: Prevents duplicate bet UUIDs
+- **Error Handling**: Comprehensive error responses with details
+
+#### Request Format
+
+```json
+{
+  "sync_id": "sync_20260201_082615_a1b2c3d4",
+  "client_id": "abc123def456",
+  "sync_timestamp": "2026-02-01T08:26:15.123456",
+  "date_range": "today",
+  "start_date": "2026-02-01T00:00:00",
+  "end_date": "2026-02-01T08:26:15",
+  "bets": [...],
+  "extraction_stats": [...],
+  "summary": {...}
+}
+```
+
+#### Response Format
+
+**Success (200 OK)**:
+```json
+{
+  "success": true,
+  "synced_count": 45,
+  "message": "Report data synchronized successfully",
+  "server_timestamp": "2026-02-01T08:26:20.123456"
+}
+```
+
+**Error (400 Bad Request)**:
+```json
+{
+  "success": false,
+  "error": "Invalid request format",
+  "details": "Missing required field: sync_id"
+}
+```
+
+**Error (401 Unauthorized)**:
+```json
+{
+  "success": false,
+  "error": "Authentication required",
+  "details": "Invalid or expired bearer token"
+}
+```
+
+### 3. Web Interface
+
+#### Reports List Page (`/reports`)
+
+Features:
+- **Filtering**:
+  - Filter by Client ID
+  - Filter by Date Range (today, yesterday, week, all)
+  - Filter by Start Date and End Date
+  - Sort by various fields (sync_timestamp, client_id, totals, etc.)
+  - Sort order (ascending/descending)
+
+- **Pagination**:
+  - Configurable per-page count (default: 20, max: 100)
+  - Page navigation with ellipsis for large page counts
+  - Shows current range and total count
+
+- **Export**:
+  - Export to CSV
+  - Export to Excel (XLSX)
+  - Export to PDF
+  - Exports respect current filters
+
+- **Display**:
+  - Summary table showing key metrics
+  - Color-coded profit (green for positive, red for negative)
+  - Click to view detailed report
+
+#### Report Detail Page (`/reports/<sync_id>`)
+
+Features:
+- **Summary Section**:
+  - Sync metadata (ID, client, timestamps)
+  - Key statistics (bets, matches)
+  - Financial summary (payin, payout, profit)
+
+- **Tabs**:
+  - **Bets Tab**: Shows all bets with expandable details
+    - Bet UUID, fixture ID, datetime
+    - Total amount, bet count
+    - Paid/paid out status indicators
+    - Expandable bet details (match-level information)
+  
+  - **Extraction Stats Tab**: Shows match statistics
+    - Match ID, fixture ID, datetime
+    - Total bets, collected, redistributed
+    - Actual and extraction results
+    - CAP application status
+    - Expandable details (under/over bets, result breakdown)
+
+### 4. Export Functionality
+
+Three export formats are supported:
+
+#### CSV Export
+- Comma-separated values
+- Compatible with Excel, Google Sheets, etc.
+- Includes all filtered data
+- Filename: `reports_export.csv`
+
+#### Excel Export (XLSX)
+- Native Excel format
+- Preserves formatting and data types
+- Includes all filtered data
+- Filename: `reports_export.xlsx`
+
+#### PDF Export
+- Professional PDF format
+- Styled tables with headers
+- Includes generation timestamp
+- Filename: `reports_export.pdf`
+
+Export data includes:
+- Summary rows for each sync
+- Bet detail rows
+- Extraction stat rows
+
+## Installation & Setup
+
+### 1. Install Dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+New dependency added:
+- `weasyprint==60.2` - For PDF generation
+
+### 2. Run Database Migration
+
+```bash
+python -c "from app.database.migrations.add_reports_tables import upgrade; from app import create_app; app = create_app(); upgrade(app.db)"
+```
+
+Or use the migration system:
+```bash
+python run_migration.py
+```
+
+### 3. Restart Application
+
+```bash
+python main.py
+```
+
+The new tables will be created automatically on first run.
+
+## Usage
+
+### For Clients
+
+Clients can now synchronize reports by sending POST requests to `/api/reports/sync`:
+
+```python
+import requests
+
+sync_data = {
+    "sync_id": "sync_20260201_082615_a1b2c3d4",
+    "client_id": "your_client_id",
+    "sync_timestamp": "2026-02-01T08:26:15.123456",
+    "date_range": "today",
+    "start_date": "2026-02-01T00:00:00",
+    "end_date": "2026-02-01T08:26:15",
+    "bets": [...],
+    "extraction_stats": [...],
+    "summary": {...}
+}
+
+headers = {
+    "Content-Type": "application/json",
+    "Authorization": "Bearer YOUR_API_TOKEN"
+}
+
+response = requests.post(
+    "http://your-server.com/api/reports/sync",
+    json=sync_data,
+    headers=headers
+)
+
+print(response.json())
+```
+
+### For Web Users
+
+1. Navigate to `/reports` in your browser
+2. Apply filters as needed:
+   - Select Client ID from dropdown
+   - Choose Date Range or specify custom dates
+   - Select Sort By and Order
+3. Click "Apply Filters"
+4. View reports in the table
+5. Click "View" to see detailed information
+6. Use "Export" dropdown to download filtered data
+
+## Testing
+
+A comprehensive test suite is provided in `test_reports_sync.py`:
+
+```bash
+# Update API_TOKEN in the script
+python test_reports_sync.py
+```
+
+Tests include:
+1. **Authentication Tests**: Verify token requirements
+2. **Validation Tests**: Check input validation
+3. **Sync Endpoint Tests**: Test successful sync
+4. **Idempotency Tests**: Verify duplicate handling
+
+## Security Considerations
+
+1. **Authentication**: All sync requests require valid API token
+2. **Authorization**: Users can only view reports from their own clients (admins see all)
+3. **Input Validation**: All inputs are validated before processing
+4. **SQL Injection Protection**: Uses SQLAlchemy parameterized queries
+5. **Rate Limiting**: Can be added via Flask-Limit (not implemented yet)
+
+## Performance Considerations
+
+1. **Indexes**: All foreign keys and frequently queried fields are indexed
+2. **Pagination**: Large datasets are paginated to prevent memory issues
+3. **Deduplication**: Duplicate sync_id and bet UUID checks prevent redundant data
+4. **Efficient Queries**: Uses SQLAlchemy's query optimization
+
+## Future Enhancements
+
+Potential improvements:
+1. **Rate Limiting**: Add rate limiting to prevent abuse
+2. **Real-time Updates**: WebSocket support for live report updates
+3. **Advanced Analytics**: Charts and graphs for report trends
+4. **Scheduled Reports**: Email reports on schedule
+5. **Custom Export Templates**: User-defined export formats
+6. **Data Retention**: Automatic cleanup of old reports
+
+## Troubleshooting
+
+### Common Issues
+
+**Issue**: Reports not appearing in web interface
+- **Solution**: Check that client_id matches your API token's client activity
+
+**Issue**: Export fails
+- **Solution**: Ensure weasyprint is installed and system fonts are available
+
+**Issue**: Duplicate sync_id errors
+- **Solution**: This is expected behavior - idempotency prevents duplicate data
+
+**Issue**: Large sync requests timeout
+- **Solution**: Increase timeout in client or reduce batch size
+
+## API Specification
+
+Full API specification is available in `REPORTS_SYNC_API_SPECIFICATION.txt`
+
+## Support
+
+For issues or questions:
+1. Check the logs in the application
+2. Review the test suite output
+3. Consult the API specification document
+4. Contact the development team
+
+## Changelog
+
+### Version 1.0 (2026-02-01)
+- Initial implementation
+- Database models for reports
+- API endpoint for sync
+- Web interface with filtering
+- Export functionality (CSV, XLSX, PDF)
+- Comprehensive test suite
\ No newline at end of file

app/templates/base.html

@@ -179,6 +179,8 @@
             <a href="{{ url_for('main.dashboard') }}">Dashboard</a>
             <a href="{{ url_for('main.fixtures') }}">Fixtures</a>
             <a href="{{ url_for('main.clients') }}">Clients</a>
+            <a href="{{ url_for('main.reports') }}">Reports</a>
+            <a href="{{ url_for('main.sync_logs') }}">Sync Logs</a>
             <a href="{{ url_for('main.uploads') }}">Uploads</a>
             <a href="{{ url_for('main.statistics') }}">Statistics</a>
             <a href="{{ url_for('main.user_tokens') }}">API Tokens</a>

requirements.txt

@@ -20,4 +20,5 @@ watchdog==6.0.0
 click==8.1.7
 colorlog==6.8.2
 marshmallow==3.23.1
-redis==5.2.0
\ No newline at end of file
+redis==5.2.0
+weasyprint==60.2
\ No newline at end of file