Add reports synchronization feature with API endpoint and web interface

- Implement /api/reports/sync endpoint for client report data synchronization
- Add database models: ReportSync, Bet, BetDetail, ExtractionStats, ReportSyncLog
- Create web interface at /reports with filtering, pagination, and export (PDF/XLSX/CSV)
- Add sync logs interface at /sync-logs with comprehensive search and filtering
- Implement comprehensive logging for all sync operations
- Add migration 010 for existing systems to create all reports tables
- Update navigation menu with Reports and Sync Logs links
- Add weasyprint dependency for PDF generation
- Include test suite for reports sync API
- Add complete documentation in REPORTS_FEATURE_README.md
parent 82e2c0a2
# Reports Synchronization Feature
## Overview
This feature implements server-side support for synchronizing report data from MbetterClient applications to the server, along with a comprehensive web interface for viewing, filtering, and exporting reports.
## Features Implemented
### 1. Database Models
Four new database models have been added to store report data:
- **ReportSync**: Tracks each synchronization operation from clients
- Stores sync metadata (sync_id, client_id, timestamps)
- Stores summary statistics (total_payin, total_payout, net_profit, etc.)
- Links to bets and extraction stats
- **Bet**: Individual bet records
- Unique UUID for deduplication
- Links to sync and client
- Stores bet details (amount, datetime, paid status)
- **BetDetail**: Individual bet details within a bet
- Links to parent bet
- Stores match-specific information (match_id, outcome, amount, result)
- **ExtractionStats**: Match extraction statistics
- Stores redistribution and payout data
- Includes CAP application details
- Stores result breakdown by outcome
### 2. API Endpoint
**POST /api/reports/sync**
Accepts report data from clients with the following features:
- **Authentication**: Requires valid API token (Bearer token)
- **Validation**: Validates all required fields and data types
- **Idempotency**: Handles duplicate sync_id gracefully
- **Deduplication**: Prevents duplicate bet UUIDs
- **Error Handling**: Comprehensive error responses with details
#### Request Format
```json
{
"sync_id": "sync_20260201_082615_a1b2c3d4",
"client_id": "abc123def456",
"sync_timestamp": "2026-02-01T08:26:15.123456",
"date_range": "today",
"start_date": "2026-02-01T00:00:00",
"end_date": "2026-02-01T08:26:15",
"bets": [...],
"extraction_stats": [...],
"summary": {...}
}
```
#### Response Format
**Success (200 OK)**:
```json
{
"success": true,
"synced_count": 45,
"message": "Report data synchronized successfully",
"server_timestamp": "2026-02-01T08:26:20.123456"
}
```
**Error (400 Bad Request)**:
```json
{
"success": false,
"error": "Invalid request format",
"details": "Missing required field: sync_id"
}
```
**Error (401 Unauthorized)**:
```json
{
"success": false,
"error": "Authentication required",
"details": "Invalid or expired bearer token"
}
```
### 3. Web Interface
#### Reports List Page (`/reports`)
Features:
- **Filtering**:
- Filter by Client ID
- Filter by Date Range (today, yesterday, week, all)
- Filter by Start Date and End Date
- Sort by various fields (sync_timestamp, client_id, totals, etc.)
- Sort order (ascending/descending)
- **Pagination**:
- Configurable per-page count (default: 20, max: 100)
- Page navigation with ellipsis for large page counts
- Shows current range and total count
- **Export**:
- Export to CSV
- Export to Excel (XLSX)
- Export to PDF
- Exports respect current filters
- **Display**:
- Summary table showing key metrics
- Color-coded profit (green for positive, red for negative)
- Click to view detailed report
#### Report Detail Page (`/reports/<sync_id>`)
Features:
- **Summary Section**:
- Sync metadata (ID, client, timestamps)
- Key statistics (bets, matches)
- Financial summary (payin, payout, profit)
- **Tabs**:
- **Bets Tab**: Shows all bets with expandable details
- Bet UUID, fixture ID, datetime
- Total amount, bet count
- Paid/paid out status indicators
- Expandable bet details (match-level information)
- **Extraction Stats Tab**: Shows match statistics
- Match ID, fixture ID, datetime
- Total bets, collected, redistributed
- Actual and extraction results
- CAP application status
- Expandable details (under/over bets, result breakdown)
### 4. Export Functionality
Three export formats are supported:
#### CSV Export
- Comma-separated values
- Compatible with Excel, Google Sheets, etc.
- Includes all filtered data
- Filename: `reports_export.csv`
#### Excel Export (XLSX)
- Native Excel format
- Preserves formatting and data types
- Includes all filtered data
- Filename: `reports_export.xlsx`
#### PDF Export
- Professional PDF format
- Styled tables with headers
- Includes generation timestamp
- Filename: `reports_export.pdf`
Export data includes:
- Summary rows for each sync
- Bet detail rows
- Extraction stat rows
## Installation & Setup
### 1. Install Dependencies
```bash
pip install -r requirements.txt
```
New dependency added:
- `weasyprint==60.2` - For PDF generation
### 2. Run Database Migration
```bash
python -c "from app.database.migrations.add_reports_tables import upgrade; from app import create_app; app = create_app(); upgrade(app.db)"
```
Or use the migration system:
```bash
python run_migration.py
```
### 3. Restart Application
```bash
python main.py
```
The new tables will be created automatically on first run.
## Usage
### For Clients
Clients can now synchronize reports by sending POST requests to `/api/reports/sync`:
```python
import requests
sync_data = {
"sync_id": "sync_20260201_082615_a1b2c3d4",
"client_id": "your_client_id",
"sync_timestamp": "2026-02-01T08:26:15.123456",
"date_range": "today",
"start_date": "2026-02-01T00:00:00",
"end_date": "2026-02-01T08:26:15",
"bets": [...],
"extraction_stats": [...],
"summary": {...}
}
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer YOUR_API_TOKEN"
}
response = requests.post(
"http://your-server.com/api/reports/sync",
json=sync_data,
headers=headers
)
print(response.json())
```
### For Web Users
1. Navigate to `/reports` in your browser
2. Apply filters as needed:
- Select Client ID from dropdown
- Choose Date Range or specify custom dates
- Select Sort By and Order
3. Click "Apply Filters"
4. View reports in the table
5. Click "View" to see detailed information
6. Use "Export" dropdown to download filtered data
## Testing
A comprehensive test suite is provided in `test_reports_sync.py`:
```bash
# Update API_TOKEN in the script
python test_reports_sync.py
```
Tests include:
1. **Authentication Tests**: Verify token requirements
2. **Validation Tests**: Check input validation
3. **Sync Endpoint Tests**: Test successful sync
4. **Idempotency Tests**: Verify duplicate handling
## Security Considerations
1. **Authentication**: All sync requests require valid API token
2. **Authorization**: Users can only view reports from their own clients (admins see all)
3. **Input Validation**: All inputs are validated before processing
4. **SQL Injection Protection**: Uses SQLAlchemy parameterized queries
5. **Rate Limiting**: Can be added via Flask-Limit (not implemented yet)
## Performance Considerations
1. **Indexes**: All foreign keys and frequently queried fields are indexed
2. **Pagination**: Large datasets are paginated to prevent memory issues
3. **Deduplication**: Duplicate sync_id and bet UUID checks prevent redundant data
4. **Efficient Queries**: Uses SQLAlchemy's query optimization
## Future Enhancements
Potential improvements:
1. **Rate Limiting**: Add rate limiting to prevent abuse
2. **Real-time Updates**: WebSocket support for live report updates
3. **Advanced Analytics**: Charts and graphs for report trends
4. **Scheduled Reports**: Email reports on schedule
5. **Custom Export Templates**: User-defined export formats
6. **Data Retention**: Automatic cleanup of old reports
## Troubleshooting
### Common Issues
**Issue**: Reports not appearing in web interface
- **Solution**: Check that client_id matches your API token's client activity
**Issue**: Export fails
- **Solution**: Ensure weasyprint is installed and system fonts are available
**Issue**: Duplicate sync_id errors
- **Solution**: This is expected behavior - idempotency prevents duplicate data
**Issue**: Large sync requests timeout
- **Solution**: Increase timeout in client or reduce batch size
## API Specification
Full API specification is available in `REPORTS_SYNC_API_SPECIFICATION.txt`
## Support
For issues or questions:
1. Check the logs in the application
2. Review the test suite output
3. Consult the API specification document
4. Contact the development team
## Changelog
### Version 1.0 (2026-02-01)
- Initial implementation
- Database models for reports
- API endpoint for sync
- Web interface with filtering
- Export functionality (CSV, XLSX, PDF)
- Comprehensive test suite
\ No newline at end of file
================================================================================
REPORTS SYNCHRONIZATION API SPECIFICATION
================================================================================
This document describes the API endpoint for synchronizing report data from the
MbetterClient application to the server. The implementation is designed to handle
unreliable and unstable network connections with automatic retry mechanisms.
================================================================================
ENDPOINT DETAILS
================================================================================
URL: /api/reports/sync
Method: POST
Authentication: Bearer Token (required)
Content-Type: application/json
Timeout: 60 seconds (recommended for large payloads)
================================================================================
REQUEST FORMAT
================================================================================
The client sends a POST request with the following JSON structure:
{
"sync_id": "sync_20260201_082615_a1b2c3d4",
"client_id": "abc123def456",
"sync_timestamp": "2026-02-01T08:26:15.123456",
"date_range": "today",
"start_date": "2026-02-01T00:00:00",
"end_date": "2026-02-01T08:26:15",
"bets": [
{
"uuid": "550e8400-e29b-41d4-a716-446655440000",
"fixture_id": "fixture_20260201_001",
"bet_datetime": "2026-02-01T08:15:30.123456",
"paid": false,
"paid_out": false,
"total_amount": 500.00,
"bet_count": 3,
"details": [
{
"match_id": 123,
"match_number": 1,
"outcome": "WIN1",
"amount": 200.00,
"win_amount": 0.00,
"result": "pending"
},
{
"match_id": 124,
"match_number": 2,
"outcome": "X1",
"amount": 150.00,
"win_amount": 0.00,
"result": "pending"
},
{
"match_id": 125,
"match_number": 3,
"outcome": "WIN2",
"amount": 150.00,
"win_amount": 0.00,
"result": "pending"
}
]
}
],
"extraction_stats": [
{
"match_id": 123,
"fixture_id": "fixture_20260201_001",
"match_datetime": "2026-02-01T08:00:00.123456",
"total_bets": 45,
"total_amount_collected": 15000.00,
"total_redistributed": 10500.00,
"actual_result": "WIN1",
"extraction_result": "WIN1",
"cap_applied": true,
"cap_percentage": 70.0,
"under_bets": 20,
"under_amount": 6000.00,
"over_bets": 25,
"over_amount": 9000.00,
"result_breakdown": {
"WIN1": {"bets": 20, "amount": 6000.00, "coefficient": 2.5},
"X1": {"bets": 10, "amount": 3000.00, "coefficient": 3.0},
"WIN2": {"bets": 15, "amount": 6000.00, "coefficient": 2.0}
}
}
],
"summary": {
"total_payin": 15000.00,
"total_payout": 10500.00,
"net_profit": 4500.00,
"total_bets": 45,
"total_matches": 1
}
}
================================================================================
REQUEST FIELDS DESCRIPTION
================================================================================
Top-Level Fields:
----------------
sync_id (string, required)
- Unique identifier for this sync operation
- Format: "sync_YYYYMMDD_HHMMSS_<random_8_chars>"
- Used for tracking and deduplication on server side
client_id (string, required)
- Unique identifier for the client machine
- Generated from rustdesk_id or machine ID
- Used to identify source of data
sync_timestamp (string, required)
- ISO 8601 format timestamp when sync was initiated
- Format: "YYYY-MM-DDTHH:MM:SS.ffffff"
date_range (string, required)
- Time range for the report data
- Values: "today", "yesterday", "week", "all"
start_date (string, required)
- ISO 8601 format start of date range
- Format: "YYYY-MM-DDTHH:MM:SS"
end_date (string, required)
- ISO 8601 format end of date range
- Format: "YYYY-MM-DDTHH:MM:SS"
bets (array, required)
- Array of bet records for the specified date range
- Excludes cancelled bets
extraction_stats (array, required)
- Array of extraction statistics for matches
- Contains redistribution and payout data
summary (object, required)
- Summary statistics for the entire sync
- Aggregated totals for quick reporting
Bet Object Fields:
-----------------
uuid (string, required)
- Unique identifier for the bet
- UUID v4 format
fixture_id (string, required)
- Fixture identifier the bet belongs to
- Links bet to specific fixture
bet_datetime (string, required)
- ISO 8601 format timestamp when bet was placed
- Format: "YYYY-MM-DDTHH:MM:SS.ffffff"
paid (boolean, required)
- Whether the bet has been marked as paid
- true = paid, false = unpaid
paid_out (boolean, required)
- Whether winnings have been paid out
- true = paid out, false = not paid out
total_amount (float, required)
- Total amount of all bet details in this bet
- Sum of all detail amounts
bet_count (integer, required)
- Number of bet details in this bet
- Excludes cancelled details
details (array, required)
- Array of individual bet details
Bet Detail Object Fields:
-------------------------
match_id (integer, required)
- Database ID of the match
- Foreign key reference
match_number (integer, required)
- Match number from fixture
- Human-readable match identifier
outcome (string, required)
- Outcome type that was bet on
- Examples: "WIN1", "X1", "WIN2", "UNDER", "OVER"
amount (float, required)
- Amount bet on this outcome
- Positive decimal value
win_amount (float, required)
- Amount won for this bet detail
- 0.00 if not won or pending
result (string, required)
- Result status of the bet detail
- Values: "pending", "won", "lost", "cancelled"
Extraction Stats Object Fields:
------------------------------
match_id (integer, required)
- Database ID of the match
- Foreign key reference
fixture_id (string, required)
- Fixture identifier
- Links stats to specific fixture
match_datetime (string, required)
- ISO 8601 format timestamp of match
- Format: "YYYY-MM-DDTHH:MM:SS.ffffff"
total_bets (integer, required)
- Total number of bets placed on this match
- Excludes cancelled bets
total_amount_collected (float, required)
- Total amount collected from all bets
- Sum of all bet amounts
total_redistributed (float, required)
- Total amount redistributed to winners
- Payout amount after CAP application
actual_result (string, required)
- Actual match result
- Examples: "WIN1", "X1", "WIN2", "RET1", "RET2"
extraction_result (string, required)
- Result used for extraction calculations
- May differ from actual_result in some cases
cap_applied (boolean, required)
- Whether CAP percentage was applied
- true = CAP applied, false = no CAP
cap_percentage (float, optional)
- CAP percentage used for calculations
- Example: 70.0 for 70% CAP
under_bets (integer, required)
- Number of UNDER bets placed
- Only applicable for UNDER/OVER matches
under_amount (float, required)
- Total amount bet on UNDER
- Only applicable for UNDER/OVER matches
over_bets (integer, required)
- Number of OVER bets placed
- Only applicable for UNDER/OVER matches
over_amount (float, required)
- Total amount bet on OVER
- Only applicable for UNDER/OVER matches
result_breakdown (object, required)
- Detailed breakdown of bets by outcome
- Structure: {"OUTCOME_NAME": {"bets": N, "amount": X.XX, "coefficient": Y.YY}}
Summary Object Fields:
---------------------
total_payin (float, required)
- Total amount collected from all bets
- Sum of all bet amounts
total_payout (float, required)
- Total amount redistributed to winners
- Sum of all extraction stats total_redistributed
net_profit (float, required)
- Net profit for the period
- Calculated as: total_payin - total_payout
total_bets (integer, required)
- Total number of bet details
- Excludes cancelled bets
total_matches (integer, required)
- Number of matches with extraction stats
- Count of extraction_stats array
================================================================================
SUCCESS RESPONSE FORMAT
================================================================================
HTTP Status: 200 OK
Content-Type: application/json
{
"success": true,
"synced_count": 45,
"message": "Report data synchronized successfully",
"server_timestamp": "2026-02-01T08:26:20.123456"
}
Success Response Fields:
---------------------
success (boolean, required)
- Always true for successful sync
- Indicates operation completed successfully
synced_count (integer, required)
- Number of items successfully synced
- Total count of bets and stats processed
message (string, required)
- Human-readable success message
- Description of sync operation
server_timestamp (string, required)
- ISO 8601 format timestamp on server
- When the server processed the sync
================================================================================
ERROR RESPONSE FORMAT
================================================================================
HTTP Status: 400 Bad Request
Content-Type: application/json
{
"success": false,
"error": "Invalid request format",
"details": "Missing required field: sync_id"
}
HTTP Status: 401 Unauthorized
Content-Type: application/json
{
"success": false,
"error": "Authentication required",
"details": "Invalid or expired bearer token"
}
HTTP Status: 429 Too Many Requests
Content-Type: application/json
{
"success": false,
"error": "Rate limit exceeded",
"details": "Too many sync requests. Please try again later.",
"retry_after": 60
}
HTTP Status: 500 Internal Server Error
Content-Type: application/json
{
"success": false,
"error": "Internal server error",
"details": "An unexpected error occurred while processing sync"
}
Error Response Fields:
--------------------
success (boolean, required)
- Always false for errors
- Indicates operation failed
error (string, required)
- Error type or category
- Human-readable error identifier
details (string, optional)
- Detailed error description
- Additional context about the error
retry_after (integer, optional)
- Seconds to wait before retrying
- Only present for rate limit errors
================================================================================
CLIENT RETRY BEHAVIOR
================================================================================
The client implements robust retry mechanisms for unreliable connections:
1. Exponential Backoff:
- Base backoff: 60 seconds
- Formula: backoff_time = 60 * (2 ^ retry_count)
- Example: 60s, 120s, 240s, 480s, 960s
2. Maximum Retries:
- Per sync attempt: 3 retries
- Per queued item: 5 retries total
- After max retries: item marked as failed
3. Offline Queue:
- Failed syncs are queued for retry
- Queue stored in: <user_data_dir>/sync_queue/reports_sync_queue.json
- Maximum queue size: 1000 items
- Queue persists across application restarts
4. Connection Error Handling:
- Timeout: Retry with backoff
- Connection Error: Retry with backoff
- HTTP 429: Wait retry_after seconds, then retry
- HTTP 401: Stop retrying (authentication issue)
- HTTP 5xx: Retry with backoff
5. Queue Processing:
- Automatic processing on scheduled interval (default: 1 hour)
- FIFO order (oldest items first)
- Skips items waiting for backoff period
- Removes completed/failed items after processing
================================================================================
SERVER IMPLEMENTATION REQUIREMENTS
================================================================================
1. Authentication:
- Validate Bearer token from Authorization header
- Return 401 if token is invalid or expired
- Token should identify the client machine
2. Data Validation:
- Validate all required fields are present
- Validate data types match specifications
- Validate date ranges are valid
- Return 400 with specific error details on validation failure
3. Data Storage:
- Store bets data with deduplication based on uuid
- Store extraction stats with deduplication based on match_id
- Use sync_id for tracking and audit trail
- Store client_id for data source identification
4. Idempotency:
- Handle duplicate sync requests gracefully
- Use sync_id to detect duplicates
- Return success for already-synced data
5. Rate Limiting:
- Implement rate limiting to prevent abuse
- Return 429 with retry_after header when limit exceeded
- Recommended: 1 sync per minute per client
6. Response Format:
- Always return JSON with success field
- Include appropriate HTTP status codes
- Provide helpful error messages for debugging
7. Data Integrity:
- Validate numeric values are positive where required
- Validate timestamps are in valid ISO 8601 format
- Validate UUIDs are valid UUID v4 format
- Validate result values are from allowed set
================================================================================
SECURITY CONSIDERATIONS
================================================================================
1. Authentication:
- All requests must include valid Bearer token
- Tokens should be cryptographically secure
- Implement token expiration and refresh mechanism
2. Data Encryption:
- Consider encrypting sensitive data in transit (HTTPS)
- Validate SSL/TLS certificates
- Use secure cipher suites
3. Input Validation:
- Sanitize all input data
- Prevent SQL injection
- Validate data types and ranges
- Limit payload size (recommended: 10MB max)
4. Audit Logging:
- Log all sync requests with sync_id
- Log client_id for tracking
- Log timestamps for audit trail
- Log errors for troubleshooting
================================================================================
TESTING RECOMMENDATIONS
================================================================================
1. Unit Tests:
- Test with valid request data
- Test with missing required fields
- Test with invalid data types
- Test with duplicate sync_id
2. Integration Tests:
- Test with authentication
- Test without authentication
- Test with rate limiting
- Test with large payloads
3. Network Tests:
- Test with slow connections
- Test with intermittent failures
- Test with timeout scenarios
- Test retry behavior
4. Load Tests:
- Test with multiple concurrent clients
- Test with large data volumes
- Test with rapid sync requests
- Test queue overflow scenarios
================================================================================
VERSION HISTORY
================================================================================
Version 1.0 - 2026-02-01
- Initial specification
- Basic sync functionality
- Retry mechanisms
- Offline queue support
================================================================================
CONTACT
================================================================================
For questions or issues regarding this API specification, please contact the
development team.
Document generated: 2026-02-01
Last updated: 2026-02-01
\ No newline at end of file
...@@ -914,4 +914,356 @@ def api_track_client(): ...@@ -914,4 +914,356 @@ def api_track_client():
except Exception as e: except Exception as e:
logger.error(f"API track client error: {str(e)}") logger.error(f"API track client error: {str(e)}")
return jsonify({'error': 'Failed to track client'}), 500 return jsonify({'error': 'Failed to track client'}), 500
\ No newline at end of file
@bp.route('/reports/sync', methods=['POST'])
@csrf.exempt
def api_reports_sync():
"""Synchronize report data from clients"""
import time
start_time = time.time()
try:
from app.models import ReportSync, Bet, BetDetail, ExtractionStats, APIToken, ReportSyncLog
from app.auth.jwt_utils import validate_api_token, extract_token_from_request
from datetime import datetime
import uuid as uuid_lib
# Get request metadata for logging
ip_address = request.headers.get('X-Forwarded-For', request.remote_addr)
user_agent = request.headers.get('User-Agent')
request_size = len(request.data) if request.data else 0
# Authenticate using API token
token = extract_token_from_request()
if not token:
# Log failed authentication
log_entry = ReportSyncLog(
sync_id='unknown',
client_id='unknown',
sync_timestamp=datetime.utcnow(),
operation_type='error',
status='failed',
error_message='Authentication required',
error_details={'reason': 'API token required'},
ip_address=ip_address,
user_agent=user_agent,
request_size=request_size,
processing_time_ms=int((time.time() - start_time) * 1000)
)
db.session.add(log_entry)
db.session.commit()
return jsonify({
'success': False,
'error': 'Authentication required',
'details': 'API token required'
}), 401
user, api_token = validate_api_token(token)
if not user or not user.is_active:
# Log failed authentication
log_entry = ReportSyncLog(
sync_id='unknown',
client_id='unknown',
sync_timestamp=datetime.utcnow(),
operation_type='error',
status='failed',
error_message='Authentication failed',
error_details={'reason': 'Invalid or expired API token'},
ip_address=ip_address,
user_agent=user_agent,
request_size=request_size,
processing_time_ms=int((time.time() - start_time) * 1000)
)
db.session.add(log_entry)
db.session.commit()
return jsonify({
'success': False,
'error': 'Authentication failed',
'details': 'Invalid or expired API token'
}), 401
# Get request data
data = request.get_json()
if not data:
# Log invalid request
log_entry = ReportSyncLog(
sync_id='unknown',
client_id='unknown',
sync_timestamp=datetime.utcnow(),
operation_type='error',
status='failed',
error_message='Invalid request format',
error_details={'reason': 'No JSON data provided'},
ip_address=ip_address,
user_agent=user_agent,
request_size=request_size,
processing_time_ms=int((time.time() - start_time) * 1000)
)
db.session.add(log_entry)
db.session.commit()
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': 'No JSON data provided'
}), 400
# Validate required fields
required_fields = ['sync_id', 'client_id', 'sync_timestamp', 'date_range', 'start_date', 'end_date', 'bets', 'extraction_stats', 'summary']
for field in required_fields:
if field not in data:
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': f'Missing required field: {field}'
}), 400
# Check for duplicate sync_id (idempotency)
existing_sync = ReportSync.query.filter_by(sync_id=data['sync_id']).first()
if existing_sync:
# Log duplicate sync
log_entry = ReportSyncLog(
sync_id=data['sync_id'],
client_id=data['client_id'],
sync_timestamp=sync_timestamp,
operation_type='duplicate_sync',
status='success',
bets_processed=0,
bets_new=0,
bets_duplicate=0,
stats_processed=0,
stats_new=0,
stats_updated=0,
total_payin=existing_sync.total_payin,
total_payout=existing_sync.total_payout,
net_profit=existing_sync.net_profit,
total_bets=existing_sync.total_bets,
total_matches=existing_sync.total_matches,
ip_address=ip_address,
user_agent=user_agent,
request_size=request_size,
processing_time_ms=int((time.time() - start_time) * 1000)
)
db.session.add(log_entry)
db.session.commit()
logger.info(f"Duplicate sync_id {data['sync_id']} detected, returning success")
return jsonify({
'success': True,
'synced_count': existing_sync.total_bets + existing_sync.total_matches,
'message': 'Report data already synchronized',
'server_timestamp': datetime.utcnow().isoformat()
}), 200
# Parse timestamps
try:
sync_timestamp = datetime.fromisoformat(data['sync_timestamp'])
start_date = datetime.fromisoformat(data['start_date'])
end_date = datetime.fromisoformat(data['end_date'])
except ValueError as e:
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': f'Invalid datetime format: {str(e)}'
}), 400
# Validate summary data
summary = data['summary']
if not all(key in summary for key in ['total_payin', 'total_payout', 'net_profit', 'total_bets', 'total_matches']):
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': 'Missing required fields in summary'
}), 400
# Create report sync record
report_sync = ReportSync(
sync_id=data['sync_id'],
client_id=data['client_id'],
sync_timestamp=sync_timestamp,
date_range=data['date_range'],
start_date=start_date,
end_date=end_date,
total_payin=summary['total_payin'],
total_payout=summary['total_payout'],
net_profit=summary['net_profit'],
total_bets=summary['total_bets'],
total_matches=summary['total_matches']
)
db.session.add(report_sync)
# Process bets
bets_count = 0
for bet_data in data['bets']:
# Validate bet UUID
try:
uuid_lib.UUID(bet_data['uuid'])
except ValueError:
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': f'Invalid UUID format: {bet_data.get("uuid", "unknown")}'
}), 400
# Check for duplicate bet UUID
existing_bet = Bet.query.filter_by(uuid=bet_data['uuid']).first()
if existing_bet:
logger.warning(f"Duplicate bet UUID {bet_data['uuid']} detected, skipping")
continue
# Parse bet datetime
try:
bet_datetime = datetime.fromisoformat(bet_data['bet_datetime'])
except ValueError:
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': f'Invalid bet_datetime format: {bet_data.get("bet_datetime", "unknown")}'
}), 400
# Create bet record
bet = Bet(
uuid=bet_data['uuid'],
sync_id=report_sync.id,
client_id=data['client_id'],
fixture_id=bet_data['fixture_id'],
bet_datetime=bet_datetime,
paid=bet_data.get('paid', False),
paid_out=bet_data.get('paid_out', False),
total_amount=bet_data['total_amount'],
bet_count=bet_data['bet_count']
)
db.session.add(bet)
bets_count += 1
# Process bet details
for detail_data in bet_data.get('details', []):
bet_detail = BetDetail(
bet_id=bet.id,
match_id=detail_data['match_id'],
match_number=detail_data['match_number'],
outcome=detail_data['outcome'],
amount=detail_data['amount'],
win_amount=detail_data.get('win_amount', 0.00),
result=detail_data['result']
)
db.session.add(bet_detail)
# Process extraction stats
stats_count = 0
stats_new = 0
stats_updated = 0
for stats_data in data['extraction_stats']:
# Check for duplicate stats (match_id + client_id)
existing_stats = ExtractionStats.query.filter_by(
match_id=stats_data['match_id'],
client_id=data['client_id']
).first()
if existing_stats:
# Update existing stats
existing_stats.sync_id = report_sync.id
existing_stats.fixture_id = stats_data['fixture_id']
existing_stats.match_datetime = datetime.fromisoformat(stats_data['match_datetime'])
existing_stats.total_bets = stats_data['total_bets']
existing_stats.total_amount_collected = stats_data['total_amount_collected']
existing_stats.total_redistributed = stats_data['total_redistributed']
existing_stats.actual_result = stats_data['actual_result']
existing_stats.extraction_result = stats_data['extraction_result']
existing_stats.cap_applied = stats_data.get('cap_applied', False)
existing_stats.cap_percentage = stats_data.get('cap_percentage')
existing_stats.under_bets = stats_data.get('under_bets', 0)
existing_stats.under_amount = stats_data.get('under_amount', 0.00)
existing_stats.over_bets = stats_data.get('over_bets', 0)
existing_stats.over_amount = stats_data.get('over_amount', 0.00)
existing_stats.result_breakdown = stats_data.get('result_breakdown')
stats_count += 1
stats_updated += 1
else:
# Create new stats record
try:
match_datetime = datetime.fromisoformat(stats_data['match_datetime'])
except ValueError:
return jsonify({
'success': False,
'error': 'Invalid request format',
'details': f'Invalid match_datetime format: {stats_data.get("match_datetime", "unknown")}'
}), 400
extraction_stats = ExtractionStats(
sync_id=report_sync.id,
client_id=data['client_id'],
match_id=stats_data['match_id'],
fixture_id=stats_data['fixture_id'],
match_datetime=match_datetime,
total_bets=stats_data['total_bets'],
total_amount_collected=stats_data['total_amount_collected'],
total_redistributed=stats_data['total_redistributed'],
actual_result=stats_data['actual_result'],
extraction_result=stats_data['extraction_result'],
cap_applied=stats_data.get('cap_applied', False),
cap_percentage=stats_data.get('cap_percentage'),
under_bets=stats_data.get('under_bets', 0),
under_amount=stats_data.get('under_amount', 0.00),
over_bets=stats_data.get('over_bets', 0),
over_amount=stats_data.get('over_amount', 0.00),
result_breakdown=stats_data.get('result_breakdown')
)
db.session.add(extraction_stats)
stats_count += 1
stats_new += 1
# Commit all changes
db.session.commit()
# Log successful sync
log_entry = ReportSyncLog(
sync_id=data['sync_id'],
client_id=data['client_id'],
sync_timestamp=sync_timestamp,
operation_type='new_sync',
status='success',
bets_processed=bets_count,
bets_new=bets_count,
bets_duplicate=0,
stats_processed=stats_count,
stats_new=stats_new,
stats_updated=stats_updated,
total_payin=summary['total_payin'],
total_payout=summary['total_payout'],
net_profit=summary['net_profit'],
total_bets=summary['total_bets'],
total_matches=summary['total_matches'],
ip_address=ip_address,
user_agent=user_agent,
request_size=request_size,
processing_time_ms=int((time.time() - start_time) * 1000)
)
db.session.add(log_entry)
db.session.commit()
# Update API token last used
if api_token:
api_token.update_last_used(request.remote_addr)
logger.info(f"Report sync {data['sync_id']} completed successfully from client {data['client_id']}: {bets_count} bets, {stats_count} stats ({stats_new} new, {stats_updated} updated)")
return jsonify({
'success': True,
'synced_count': bets_count + stats_count,
'message': 'Report data synchronized successfully',
'server_timestamp': datetime.utcnow().isoformat()
}), 200
except Exception as e:
db.session.rollback()
logger.error(f"Reports sync error: {str(e)}")
return jsonify({
'success': False,
'error': 'Internal server error',
'details': 'An unexpected error occurred while processing sync'
}), 500
\ No newline at end of file
...@@ -565,6 +565,216 @@ class Migration_009_CreateClientActivityTable(Migration): ...@@ -565,6 +565,216 @@ class Migration_009_CreateClientActivityTable(Migration):
def can_rollback(self) -> bool: def can_rollback(self) -> bool:
return True return True
class Migration_010_CreateReportsTables(Migration):
"""Create reports synchronization tables for client report data"""
def __init__(self):
super().__init__("010", "Create reports synchronization tables for client report data")
def up(self):
"""Create all reports-related tables"""
try:
inspector = inspect(db.engine)
# Create report_syncs table
if 'report_syncs' not in inspector.get_table_names():
create_report_syncs_sql = '''
CREATE TABLE report_syncs (
id INT AUTO_INCREMENT PRIMARY KEY,
sync_id VARCHAR(255) NOT NULL UNIQUE,
client_id VARCHAR(255) NOT NULL,
sync_timestamp DATETIME NOT NULL,
date_range VARCHAR(50) NOT NULL,
start_date DATETIME NOT NULL,
end_date DATETIME NOT NULL,
total_payin DECIMAL(15,2) NOT NULL DEFAULT 0.00,
total_payout DECIMAL(15,2) NOT NULL DEFAULT 0.00,
net_profit DECIMAL(15,2) NOT NULL DEFAULT 0.00,
total_bets INT NOT NULL DEFAULT 0,
total_matches INT NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_report_syncs_sync_id (sync_id),
INDEX idx_report_syncs_client_id (client_id),
INDEX idx_report_syncs_sync_timestamp (sync_timestamp),
INDEX idx_report_syncs_date_range (date_range),
INDEX idx_report_syncs_created_at (created_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
'''
with db.engine.connect() as conn:
conn.execute(text(create_report_syncs_sql))
conn.commit()
logger.info("Created report_syncs table")
else:
logger.info("report_syncs table already exists, skipping")
# Create bets table
if 'bets' not in inspector.get_table_names():
create_bets_sql = '''
CREATE TABLE bets (
id INT AUTO_INCREMENT PRIMARY KEY,
report_sync_id INT NOT NULL,
uuid VARCHAR(36) NOT NULL UNIQUE,
fixture_id VARCHAR(255) NOT NULL,
bet_datetime DATETIME NOT NULL,
paid BOOLEAN NOT NULL DEFAULT FALSE,
paid_out BOOLEAN NOT NULL DEFAULT FALSE,
total_amount DECIMAL(15,2) NOT NULL DEFAULT 0.00,
bet_count INT NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_bets_report_sync_id (report_sync_id),
INDEX idx_bets_uuid (uuid),
INDEX idx_bets_fixture_id (fixture_id),
INDEX idx_bets_bet_datetime (bet_datetime),
INDEX idx_bets_paid (paid),
INDEX idx_bets_paid_out (paid_out),
FOREIGN KEY (report_sync_id) REFERENCES report_syncs(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
'''
with db.engine.connect() as conn:
conn.execute(text(create_bets_sql))
conn.commit()
logger.info("Created bets table")
else:
logger.info("bets table already exists, skipping")
# Create bet_details table
if 'bet_details' not in inspector.get_table_names():
create_bet_details_sql = '''
CREATE TABLE bet_details (
id INT AUTO_INCREMENT PRIMARY KEY,
bet_id INT NOT NULL,
match_id INT NOT NULL,
match_number INT NOT NULL,
outcome VARCHAR(50) NOT NULL,
amount DECIMAL(15,2) NOT NULL DEFAULT 0.00,
win_amount DECIMAL(15,2) NOT NULL DEFAULT 0.00,
result VARCHAR(50) NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_bet_details_bet_id (bet_id),
INDEX idx_bet_details_match_id (match_id),
INDEX idx_bet_details_outcome (outcome),
INDEX idx_bet_details_result (result),
FOREIGN KEY (bet_id) REFERENCES bets(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
'''
with db.engine.connect() as conn:
conn.execute(text(create_bet_details_sql))
conn.commit()
logger.info("Created bet_details table")
else:
logger.info("bet_details table already exists, skipping")
# Create extraction_stats table
if 'extraction_stats' not in inspector.get_table_names():
create_extraction_stats_sql = '''
CREATE TABLE extraction_stats (
id INT AUTO_INCREMENT PRIMARY KEY,
report_sync_id INT NOT NULL,
match_id INT NOT NULL,
fixture_id VARCHAR(255) NOT NULL,
match_datetime DATETIME NOT NULL,
total_bets INT NOT NULL DEFAULT 0,
total_amount_collected DECIMAL(15,2) NOT NULL DEFAULT 0.00,
total_redistributed DECIMAL(15,2) NOT NULL DEFAULT 0.00,
actual_result VARCHAR(50) NOT NULL,
extraction_result VARCHAR(50) NOT NULL,
cap_applied BOOLEAN NOT NULL DEFAULT FALSE,
cap_percentage DECIMAL(5,2),
under_bets INT NOT NULL DEFAULT 0,
under_amount DECIMAL(15,2) NOT NULL DEFAULT 0.00,
over_bets INT NOT NULL DEFAULT 0,
over_amount DECIMAL(15,2) NOT NULL DEFAULT 0.00,
result_breakdown JSON,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_extraction_stats_report_sync_id (report_sync_id),
INDEX idx_extraction_stats_match_id (match_id),
INDEX idx_extraction_stats_fixture_id (fixture_id),
INDEX idx_extraction_stats_match_datetime (match_datetime),
INDEX idx_extraction_stats_actual_result (actual_result),
FOREIGN KEY (report_sync_id) REFERENCES report_syncs(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
'''
with db.engine.connect() as conn:
conn.execute(text(create_extraction_stats_sql))
conn.commit()
logger.info("Created extraction_stats table")
else:
logger.info("extraction_stats table already exists, skipping")
# Create report_sync_logs table
if 'report_sync_logs' not in inspector.get_table_names():
create_report_sync_logs_sql = '''
CREATE TABLE report_sync_logs (
id INT AUTO_INCREMENT PRIMARY KEY,
sync_id VARCHAR(255) NOT NULL,
client_id VARCHAR(255) NOT NULL,
sync_timestamp DATETIME NOT NULL,
operation_type ENUM('new_sync', 'duplicate_sync', 'update_stats', 'error') NOT NULL,
status ENUM('success', 'failed') NOT NULL,
bets_processed INT NOT NULL DEFAULT 0,
bets_new INT NOT NULL DEFAULT 0,
bets_duplicate INT NOT NULL DEFAULT 0,
stats_processed INT NOT NULL DEFAULT 0,
stats_new INT NOT NULL DEFAULT 0,
stats_updated INT NOT NULL DEFAULT 0,
total_payin DECIMAL(15,2) NOT NULL DEFAULT 0.00,
total_payout DECIMAL(15,2) NOT NULL DEFAULT 0.00,
net_profit DECIMAL(15,2) NOT NULL DEFAULT 0.00,
total_bets INT NOT NULL DEFAULT 0,
total_matches INT NOT NULL DEFAULT 0,
error_message TEXT,
error_details JSON,
ip_address VARCHAR(45),
user_agent TEXT,
request_size INT,
processing_time_ms INT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_report_sync_logs_sync_id (sync_id),
INDEX idx_report_sync_logs_client_id (client_id),
INDEX idx_report_sync_logs_sync_timestamp (sync_timestamp),
INDEX idx_report_sync_logs_operation_type (operation_type),
INDEX idx_report_sync_logs_status (status),
INDEX idx_report_sync_logs_created_at (created_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
'''
with db.engine.connect() as conn:
conn.execute(text(create_report_sync_logs_sql))
conn.commit()
logger.info("Created report_sync_logs table")
else:
logger.info("report_sync_logs table already exists, skipping")
logger.info("Migration 010: All reports tables created successfully")
return True
except Exception as e:
logger.error(f"Migration 010 failed: {str(e)}")
raise
def down(self):
"""Drop all reports-related tables"""
try:
with db.engine.connect() as conn:
# Drop in reverse order of creation (due to foreign keys)
conn.execute(text("DROP TABLE IF EXISTS report_sync_logs"))
conn.execute(text("DROP TABLE IF EXISTS extraction_stats"))
conn.execute(text("DROP TABLE IF EXISTS bet_details"))
conn.execute(text("DROP TABLE IF EXISTS bets"))
conn.execute(text("DROP TABLE IF EXISTS report_syncs"))
conn.commit()
logger.info("Dropped all reports tables")
return True
except Exception as e:
logger.error(f"Rollback of migration 010 failed: {str(e)}")
raise
def can_rollback(self) -> bool:
return True
class MigrationManager: class MigrationManager:
"""Manages database migrations and versioning""" """Manages database migrations and versioning"""
...@@ -579,6 +789,7 @@ class MigrationManager: ...@@ -579,6 +789,7 @@ class MigrationManager:
Migration_007_AddDoneToStatusEnum(), Migration_007_AddDoneToStatusEnum(),
Migration_008_AddRemoteDomainSetting(), Migration_008_AddRemoteDomainSetting(),
Migration_009_CreateClientActivityTable(), Migration_009_CreateClientActivityTable(),
Migration_010_CreateReportsTables(),
] ]
def ensure_version_table(self): def ensure_version_table(self):
......
...@@ -1527,4 +1527,699 @@ def clients(): ...@@ -1527,4 +1527,699 @@ def clients():
except Exception as e: except Exception as e:
logger.error(f"Clients page error: {str(e)}") logger.error(f"Clients page error: {str(e)}")
flash('Error loading clients', 'error') flash('Error loading clients', 'error')
return render_template('main/clients.html', clients=[]) return render_template('main/clients.html', clients=[])
\ No newline at end of file
@csrf.exempt
@bp.route('/reports')
@login_required
@require_active_user
def reports():
"""Reports page with filtering, pagination, and export"""
try:
from app.models import ReportSync, Bet, ExtractionStats
from sqlalchemy import func, and_, or_
# Get filter parameters
client_id_filter = request.args.get('client_id', '').strip()
date_range_filter = request.args.get('date_range', '').strip()
start_date_filter = request.args.get('start_date', '').strip()
end_date_filter = request.args.get('end_date', '').strip()
sort_by = request.args.get('sort_by', 'sync_timestamp')
sort_order = request.args.get('sort_order', 'desc')
export_format = request.args.get('export', '').strip()
# Pagination
page = request.args.get('page', 1, type=int)
per_page = min(request.args.get('per_page', 20, type=int), 100)
# Base query
if current_user.is_admin:
query = ReportSync.query
else:
# Non-admin users can only see reports from their own clients
from app.models import APIToken
user_token_ids = [t.id for t in APIToken.query.filter_by(user_id=current_user.id).all()]
if user_token_ids:
# Get client_ids from ClientActivity for this user's tokens
from app.models import ClientActivity
client_ids = [c.rustdesk_id for c in ClientActivity.query.filter(
ClientActivity.api_token_id.in_(user_token_ids)
).all()]
if client_ids:
query = ReportSync.query.filter(ReportSync.client_id.in_(client_ids))
else:
query = ReportSync.query.filter(ReportSync.client_id == 'none')
else:
query = ReportSync.query.filter(ReportSync.client_id == 'none')
# Apply filters
if client_id_filter:
query = query.filter(ReportSync.client_id == client_id_filter)
if start_date_filter:
try:
start_date = datetime.strptime(start_date_filter, '%Y-%m-%d')
query = query.filter(ReportSync.start_date >= start_date)
except ValueError:
pass
if end_date_filter:
try:
end_date = datetime.strptime(end_date_filter, '%Y-%m-%d')
# Include the entire end date
end_date = end_date.replace(hour=23, minute=59, second=59)
query = query.filter(ReportSync.end_date <= end_date)
except ValueError:
pass
# Sorting
if hasattr(ReportSync, sort_by):
sort_column = getattr(ReportSync, sort_by)
if sort_order == 'asc':
query = query.order_by(sort_column.asc())
else:
query = query.order_by(sort_column.desc())
else:
query = query.order_by(ReportSync.sync_timestamp.desc())
# Handle export
if export_format:
return export_reports(query, export_format)
# Pagination
reports_pagination = query.paginate(page=page, per_page=per_page, error_out=False)
# Get unique client IDs for filter dropdown
if current_user.is_admin:
all_client_ids = db.session.query(ReportSync.client_id).distinct().all()
else:
if user_token_ids:
from app.models import ClientActivity
all_client_ids = db.session.query(ClientActivity.rustdesk_id).filter(
ClientActivity.api_token_id.in_(user_token_ids)
).distinct().all()
else:
all_client_ids = []
client_ids = [cid[0] for cid in all_client_ids if cid[0]]
return render_template('main/reports.html',
reports=reports_pagination.items,
pagination=reports_pagination,
client_ids=client_ids,
filters={
'client_id': client_id_filter,
'date_range': date_range_filter,
'start_date': start_date_filter,
'end_date': end_date_filter,
'sort_by': sort_by,
'sort_order': sort_order
})
except Exception as e:
logger.error(f"Reports page error: {str(e)}")
flash('Error loading reports', 'error')
return render_template('main/reports.html', reports=[], pagination=None, client_ids=[])
@csrf.exempt
@bp.route('/reports/<int:sync_id>')
@login_required
@require_active_user
def report_detail(sync_id):
"""Detailed view of a specific report sync"""
try:
from app.models import ReportSync, Bet, ExtractionStats
# Get report sync
if current_user.is_admin:
report = ReportSync.query.get_or_404(sync_id)
else:
# Check if user has access to this client
from app.models import APIToken, ClientActivity
user_token_ids = [t.id for t in APIToken.query.filter_by(user_id=current_user.id).all()]
if user_token_ids:
client_ids = [c.rustdesk_id for c in ClientActivity.query.filter(
ClientActivity.api_token_id.in_(user_token_ids)
).all()]
report = ReportSync.query.filter(
and_(ReportSync.id == sync_id, ReportSync.client_id.in_(client_ids))
).first_or_404()
else:
report = ReportSync.query.filter_by(id=sync_id, client_id='none').first_or_404()
# Get bets for this sync
bets = Bet.query.filter_by(sync_id=sync_id).all()
# Get extraction stats for this sync
stats = ExtractionStats.query.filter_by(sync_id=sync_id).all()
return render_template('main/report_detail.html',
report=report,
bets=bets,
stats=stats)
except Exception as e:
logger.error(f"Report detail error: {str(e)}")
flash('Error loading report details', 'error')
return redirect(url_for('main.reports'))
def export_reports(query, export_format):
"""Export reports to various formats"""
try:
from app.models import ReportSync, Bet, ExtractionStats
import pandas as pd
from flask import Response
import io
# Get all reports matching the query
reports = query.all()
if not reports:
flash('No reports to export', 'warning')
return redirect(url_for('main.reports'))
@csrf.exempt
@bp.route('/sync-logs')
@login_required
@require_active_user
def sync_logs():
"""Sync logs page with filtering and search"""
try:
from app.models import ReportSyncLog
from sqlalchemy import and_, or_
# Check for export request
export_format = request.args.get('export')
if export_format:
return export_sync_logs(export_format)
# Get filter parameters
client_id_filter = request.args.get('client_id', '').strip()
operation_type_filter = request.args.get('operation_type', '').strip()
status_filter = request.args.get('status', '').strip()
search_query = request.args.get('search', '').strip()
start_date_filter = request.args.get('start_date', '').strip()
end_date_filter = request.args.get('end_date', '').strip()
sort_by = request.args.get('sort_by', 'created_at')
sort_order = request.args.get('sort_order', 'desc')
# Pagination
page = request.args.get('page', 1, type=int)
per_page = min(request.args.get('per_page', 50, type=int), 200)
def export_sync_logs(export_format):
"""Export sync logs to CSV, XLSX, or PDF"""
try:
from app.models import ReportSyncLog
from sqlalchemy import and_, or_
from flask import make_response, send_file
import pandas as pd
from io import BytesIO
from datetime import datetime
# Get filter parameters
client_id_filter = request.args.get('client_id', '').strip()
operation_type_filter = request.args.get('operation_type', '').strip()
status_filter = request.args.get('status', '').strip()
search_query = request.args.get('search', '').strip()
start_date_filter = request.args.get('start_date', '').strip()
end_date_filter = request.args.get('end_date', '').strip()
# Base query
if current_user.is_admin:
query = ReportSyncLog.query
else:
# Non-admin users can only see logs from their own clients
from app.models import APIToken, ClientActivity
user_token_ids = [t.id for t in APIToken.query.filter_by(user_id=current_user.id).all()]
if user_token_ids:
client_ids = [c.rustdesk_id for c in ClientActivity.query.filter(
ClientActivity.api_token_id.in_(user_token_ids)
).all()]
if client_ids:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id.in_(client_ids))
else:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id == 'none')
else:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id == 'none')
# Apply filters
if client_id_filter:
query = query.filter(ReportSyncLog.client_id == client_id_filter)
if operation_type_filter:
query = query.filter(ReportSyncLog.operation_type == operation_type_filter)
if status_filter:
query = query.filter(ReportSyncLog.status == status_filter)
if start_date_filter:
try:
start_date = datetime.strptime(start_date_filter, '%Y-%m-%d')
query = query.filter(ReportSyncLog.created_at >= start_date)
except ValueError:
pass
if end_date_filter:
try:
end_date = datetime.strptime(end_date_filter, '%Y-%m-%d')
end_date = end_date.replace(hour=23, minute=59, second=59)
query = query.filter(ReportSyncLog.created_at <= end_date)
except ValueError:
pass
# Search functionality
if search_query:
search_pattern = f"%{search_query}%"
query = query.filter(
or_(
ReportSyncLog.sync_id.ilike(search_pattern),
ReportSyncLog.client_id.ilike(search_pattern),
ReportSyncLog.error_message.ilike(search_pattern)
)
)
# Get all logs (no pagination for export)
logs = query.order_by(ReportSyncLog.created_at.desc()).all()
if export_format == 'csv':
# Export to CSV
output = BytesIO()
data = []
for log in logs:
data.append({
'Sync ID': log.sync_id,
'Client ID': log.client_id,
'Sync Timestamp': log.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if log.sync_timestamp else '',
'Operation Type': log.operation_type,
'Status': log.status,
'Bets Processed': log.bets_processed,
'Bets New': log.bets_new,
'Bets Duplicate': log.bets_duplicate,
'Stats Processed': log.stats_processed,
'Stats New': log.stats_new,
'Stats Updated': log.stats_updated,
'Total Payin': float(log.total_payin) if log.total_payin else 0.0,
'Total Payout': float(log.total_payout) if log.total_payout else 0.0,
'Net Profit': float(log.net_profit) if log.net_profit else 0.0,
'Total Bets': log.total_bets,
'Total Matches': log.total_matches,
'Error Message': log.error_message or '',
'IP Address': log.ip_address or '',
'User Agent': log.user_agent or '',
'Request Size (bytes)': log.request_size or 0,
'Processing Time (ms)': log.processing_time_ms or 0,
'Created At': log.created_at.strftime('%Y-%m-%d %H:%M:%S') if log.created_at else ''
})
df = pd.DataFrame(data)
df.to_csv(output, index=False)
output.seek(0)
response = make_response(output.read())
response.headers['Content-Type'] = 'text/csv'
response.headers['Content-Disposition'] = 'attachment; filename=sync_logs_export.csv'
return response
elif export_format == 'xlsx':
# Export to Excel
output = BytesIO()
data = []
for log in logs:
data.append({
'Sync ID': log.sync_id,
'Client ID': log.client_id,
'Sync Timestamp': log.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if log.sync_timestamp else '',
'Operation Type': log.operation_type,
'Status': log.status,
'Bets Processed': log.bets_processed,
'Bets New': log.bets_new,
'Bets Duplicate': log.bets_duplicate,
'Stats Processed': log.stats_processed,
'Stats New': log.stats_new,
'Stats Updated': log.stats_updated,
'Total Payin': float(log.total_payin) if log.total_payin else 0.0,
'Total Payout': float(log.total_payout) if log.total_payout else 0.0,
'Net Profit': float(log.net_profit) if log.net_profit else 0.0,
'Total Bets': log.total_bets,
'Total Matches': log.total_matches,
'Error Message': log.error_message or '',
'IP Address': log.ip_address or '',
'User Agent': log.user_agent or '',
'Request Size (bytes)': log.request_size or 0,
'Processing Time (ms)': log.processing_time_ms or 0,
'Created At': log.created_at.strftime('%Y-%m-%d %H:%M:%S') if log.created_at else ''
})
df = pd.DataFrame(data)
with pd.ExcelWriter(output, engine='openpyxl') as writer:
df.to_excel(writer, index=False, sheet_name='Sync Logs')
output.seek(0)
response = make_response(output.read())
response.headers['Content-Type'] = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
response.headers['Content-Disposition'] = 'attachment; filename=sync_logs_export.xlsx'
return response
elif export_format == 'pdf':
# Export to PDF
from weasyprint import HTML, CSS
import os
# Create HTML table
html_content = f"""
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<style>
body {{ font-family: Arial, sans-serif; margin: 20px; }}
h1 {{ color: #333; border-bottom: 2px solid #007bff; padding-bottom: 10px; }}
table {{ width: 100%; border-collapse: collapse; margin-top: 20px; }}
th {{ background-color: #007bff; color: white; padding: 10px; text-align: left; font-weight: bold; }}
td {{ padding: 8px; border: 1px solid #ddd; }}
tr:nth-child(even) {{ background-color: #f9f9f9; }}
.success {{ color: #28a745; }}
.failed {{ color: #dc3545; }}
.warning {{ color: #ffc107; }}
.info {{ color: #17a2b8; }}
</style>
</head>
<body>
<h1>Sync Logs Export</h1>
<p><strong>Generated:</strong> {datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')}</p>
<p><strong>Total Records:</strong> {len(logs)}</p>
<table>
<thead>
<tr>
<th>Sync ID</th>
<th>Client ID</th>
<th>Sync Timestamp</th>
<th>Operation Type</th>
<th>Status</th>
<th>Bets Processed</th>
<th>Stats Processed</th>
<th>Total Payin</th>
<th>Total Payout</th>
<th>Net Profit</th>
<th>Processing Time</th>
<th>Created At</th>
</tr>
</thead>
<tbody>
"""
for log in logs:
status_class = 'success' if log.status == 'success' else 'failed'
operation_class = {
'new_sync': 'info',
'duplicate_sync': 'info',
'update_stats': 'warning',
'error': 'danger'
}.get(log.operation_type, 'info')
html_content += f"""
<tr>
<td>{log.sync_id}</td>
<td>{log.client_id}</td>
<td>{log.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if log.sync_timestamp else ''}</td>
<td class="{operation_class}">{log.operation_type.replace('_', ' ').title()}</td>
<td class="{status_class}">{log.status.title()}</td>
<td>{log.bets_processed}</td>
<td>{log.stats_processed}</td>
<td>{float(log.total_payin) if log.total_payin else 0.0:.2f}</td>
<td>{float(log.total_payout) if log.total_payout else 0.0:.2f}</td>
<td>{float(log.net_profit) if log.net_profit else 0.0:.2f}</td>
<td>{log.processing_time_ms}ms</td>
<td>{log.created_at.strftime('%Y-%m-%d %H:%M:%S') if log.created_at else ''}</td>
</tr>
"""
html_content += """
</tbody>
</table>
</body>
</html>
"""
# Generate PDF
pdf_bytes = HTML(string=html_content).write_pdf()
response = make_response(pdf_bytes)
response.headers['Content-Type'] = 'application/pdf'
response.headers['Content-Disposition'] = 'attachment; filename=sync_logs_export.pdf'
return response
else:
flash('Invalid export format', 'error')
return redirect(url_for('main.sync_logs'))
except Exception as e:
logger.error(f"Export sync logs error: {str(e)}")
flash('Error exporting sync logs', 'error')
return redirect(url_for('main.sync_logs'))
# Base query
if current_user.is_admin:
query = ReportSyncLog.query
else:
# Non-admin users can only see logs from their own clients
from app.models import APIToken, ClientActivity
user_token_ids = [t.id for t in APIToken.query.filter_by(user_id=current_user.id).all()]
if user_token_ids:
client_ids = [c.rustdesk_id for c in ClientActivity.query.filter(
ClientActivity.api_token_id.in_(user_token_ids)
).all()]
if client_ids:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id.in_(client_ids))
else:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id == 'none')
else:
query = ReportSyncLog.query.filter(ReportSyncLog.client_id == 'none')
# Apply filters
if client_id_filter:
query = query.filter(ReportSyncLog.client_id == client_id_filter)
if operation_type_filter:
query = query.filter(ReportSyncLog.operation_type == operation_type_filter)
if status_filter:
query = query.filter(ReportSyncLog.status == status_filter)
if start_date_filter:
try:
start_date = datetime.strptime(start_date_filter, '%Y-%m-%d')
query = query.filter(ReportSyncLog.created_at >= start_date)
except ValueError:
pass
if end_date_filter:
try:
end_date = datetime.strptime(end_date_filter, '%Y-%m-%d')
# Include the entire end date
end_date = end_date.replace(hour=23, minute=59, second=59)
query = query.filter(ReportSyncLog.created_at <= end_date)
except ValueError:
pass
# Search functionality
if search_query:
search_pattern = f"%{search_query}%"
query = query.filter(
or_(
ReportSyncLog.sync_id.ilike(search_pattern),
ReportSyncLog.client_id.ilike(search_pattern),
ReportSyncLog.error_message.ilike(search_pattern)
)
)
# Sorting
if hasattr(ReportSyncLog, sort_by):
sort_column = getattr(ReportSyncLog, sort_by)
if sort_order == 'asc':
query = query.order_by(sort_column.asc())
else:
query = query.order_by(sort_column.desc())
else:
query = query.order_by(ReportSyncLog.created_at.desc())
# Pagination
logs_pagination = query.paginate(page=page, per_page=per_page, error_out=False)
# Get unique client IDs for filter dropdown
if current_user.is_admin:
all_client_ids = db.session.query(ReportSyncLog.client_id).distinct().all()
else:
if user_token_ids:
all_client_ids = db.session.query(ClientActivity.rustdesk_id).filter(
ClientActivity.api_token_id.in_(user_token_ids)
).distinct().all()
else:
all_client_ids = []
client_ids = [cid[0] for cid in all_client_ids if cid[0]]
return render_template('main/sync_logs.html',
logs=logs_pagination.items,
pagination=logs_pagination,
client_ids=client_ids,
filters={
'client_id': client_id_filter,
'operation_type': operation_type_filter,
'status': status_filter,
'search': search_query,
'start_date': start_date_filter,
'end_date': end_date_filter,
'sort_by': sort_by,
'sort_order': sort_order
})
except Exception as e:
logger.error(f"Sync logs page error: {str(e)}")
flash('Error loading sync logs', 'error')
return render_template('main/sync_logs.html', logs=[], pagination=None, client_ids=[])
# Prepare data for export
export_data = []
for report in reports:
# Get bets and stats for this report
bets = Bet.query.filter_by(sync_id=report.id).all()
stats = ExtractionStats.query.filter_by(sync_id=report.id).all()
# Add summary row
export_data.append({
'Type': 'Summary',
'Sync ID': report.sync_id,
'Client ID': report.client_id,
'Sync Timestamp': report.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if report.sync_timestamp else '',
'Date Range': report.date_range,
'Start Date': report.start_date.strftime('%Y-%m-%d') if report.start_date else '',
'End Date': report.end_date.strftime('%Y-%m-%d') if report.end_date else '',
'Total Payin': float(report.total_payin) if report.total_payin else 0.0,
'Total Payout': float(report.total_payout) if report.total_payout else 0.0,
'Net Profit': float(report.net_profit) if report.net_profit else 0.0,
'Total Bets': report.total_bets,
'Total Matches': report.total_matches,
'Bet UUID': '',
'Match ID': '',
'Outcome': '',
'Amount': '',
'Result': ''
})
# Add bet details
for bet in bets:
for detail in bet.details:
export_data.append({
'Type': 'Bet Detail',
'Sync ID': report.sync_id,
'Client ID': report.client_id,
'Sync Timestamp': report.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if report.sync_timestamp else '',
'Date Range': report.date_range,
'Start Date': report.start_date.strftime('%Y-%m-%d') if report.start_date else '',
'End Date': report.end_date.strftime('%Y-%m-%d') if report.end_date else '',
'Total Payin': '',
'Total Payout': '',
'Net Profit': '',
'Total Bets': '',
'Total Matches': '',
'Bet UUID': bet.uuid,
'Match ID': detail.match_id,
'Outcome': detail.outcome,
'Amount': float(detail.amount) if detail.amount else 0.0,
'Result': detail.result
})
# Add extraction stats
for stat in stats:
export_data.append({
'Type': 'Extraction Stats',
'Sync ID': report.sync_id,
'Client ID': report.client_id,
'Sync Timestamp': report.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if report.sync_timestamp else '',
'Date Range': report.date_range,
'Start Date': report.start_date.strftime('%Y-%m-%d') if report.start_date else '',
'End Date': report.end_date.strftime('%Y-%m-%d') if report.end_date else '',
'Total Payin': float(stat.total_amount_collected) if stat.total_amount_collected else 0.0,
'Total Payout': float(stat.total_redistributed) if stat.total_redistributed else 0.0,
'Net Profit': '',
'Total Bets': stat.total_bets,
'Total Matches': '',
'Bet UUID': '',
'Match ID': stat.match_id,
'Outcome': stat.actual_result,
'Amount': '',
'Result': stat.extraction_result
})
# Create DataFrame
df = pd.DataFrame(export_data)
# Export based on format
if export_format == 'csv':
output = io.StringIO()
df.to_csv(output, index=False)
output.seek(0)
return Response(
output.getvalue(),
mimetype='text/csv',
headers={'Content-Disposition': 'attachment; filename=reports_export.csv'}
)
elif export_format == 'xlsx':
output = io.BytesIO()
with pd.ExcelWriter(output, engine='openpyxl') as writer:
df.to_excel(writer, index=False, sheet_name='Reports')
output.seek(0)
return Response(
output.getvalue(),
mimetype='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
headers={'Content-Disposition': 'attachment; filename=reports_export.xlsx'}
)
elif export_format == 'pdf':
# For PDF, we'll create a simple HTML table and convert it
from weasyprint import HTML
import tempfile
# Create HTML table
html_table = df.to_html(index=False, classes='table table-striped table-bordered')
html_content = f"""
<!DOCTYPE html>
<html>
<head>
<style>
body {{ font-family: Arial, sans-serif; margin: 20px; }}
table {{ border-collapse: collapse; width: 100%; margin-bottom: 20px; }}
th, td {{ border: 1px solid #ddd; padding: 8px; text-align: left; }}
th {{ background-color: #4CAF50; color: white; }}
tr:nth-child(even) {{ background-color: #f2f2f2; }}
</style>
</head>
<body>
<h1>Reports Export</h1>
<p>Generated on: {datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')}</p>
{html_table}
</body>
</html>
"""
# Generate PDF
pdf_bytes = HTML(string=html_content).write_pdf()
return Response(
pdf_bytes,
mimetype='application/pdf',
headers={'Content-Disposition': 'attachment; filename=reports_export.pdf'}
)
else:
flash('Invalid export format', 'error')
return redirect(url_for('main.reports'))
except Exception as e:
logger.error(f"Export reports error: {str(e)}")
flash('Error exporting reports', 'error')
return redirect(url_for('main.reports'))
\ No newline at end of file
...@@ -836,4 +836,253 @@ class ClientActivity(db.Model): ...@@ -836,4 +836,253 @@ class ClientActivity(db.Model):
api_token = db.relationship('APIToken', backref='client_activity', lazy='select') api_token = db.relationship('APIToken', backref='client_activity', lazy='select')
def __repr__(self): def __repr__(self):
return f'<ClientActivity {self.rustdesk_id} via token {self.api_token_id}>' return f'<ClientActivity {self.rustdesk_id} via token {self.api_token_id}>'
\ No newline at end of file
class ReportSync(db.Model):
"""Track report synchronization operations from clients"""
__tablename__ = 'report_syncs'
id = db.Column(db.Integer, primary_key=True)
sync_id = db.Column(db.String(255), unique=True, nullable=False, index=True)
client_id = db.Column(db.String(255), nullable=False, index=True)
sync_timestamp = db.Column(db.DateTime, nullable=False, index=True)
date_range = db.Column(db.String(50), nullable=False)
start_date = db.Column(db.DateTime, nullable=False)
end_date = db.Column(db.DateTime, nullable=False)
# Summary statistics
total_payin = db.Column(db.Numeric(15, 2), default=0.00)
total_payout = db.Column(db.Numeric(15, 2), default=0.00)
net_profit = db.Column(db.Numeric(15, 2), default=0.00)
total_bets = db.Column(db.Integer, default=0)
total_matches = db.Column(db.Integer, default=0)
# Metadata
created_at = db.Column(db.DateTime, default=datetime.utcnow)
# Relationships
bets = db.relationship('Bet', backref='sync', lazy='dynamic', cascade='all, delete-orphan')
extraction_stats = db.relationship('ExtractionStats', backref='sync', lazy='dynamic', cascade='all, delete-orphan')
def to_dict(self):
"""Convert to dictionary for JSON serialization"""
return {
'id': self.id,
'sync_id': self.sync_id,
'client_id': self.client_id,
'sync_timestamp': self.sync_timestamp.isoformat() if self.sync_timestamp else None,
'date_range': self.date_range,
'start_date': self.start_date.isoformat() if self.start_date else None,
'end_date': self.end_date.isoformat() if self.end_date else None,
'total_payin': float(self.total_payin) if self.total_payin else 0.0,
'total_payout': float(self.total_payout) if self.total_payout else 0.0,
'net_profit': float(self.net_profit) if self.net_profit else 0.0,
'total_bets': self.total_bets,
'total_matches': self.total_matches,
'created_at': self.created_at.isoformat() if self.created_at else None
}
def __repr__(self):
return f'<ReportSync {self.sync_id} from {self.client_id}>'
class Bet(db.Model):
"""Individual bet records from client reports"""
__tablename__ = 'bets'
id = db.Column(db.Integer, primary_key=True)
uuid = db.Column(db.String(36), unique=True, nullable=False, index=True)
sync_id = db.Column(db.Integer, db.ForeignKey('report_syncs.id'), nullable=False, index=True)
client_id = db.Column(db.String(255), nullable=False, index=True)
fixture_id = db.Column(db.String(255), nullable=False, index=True)
bet_datetime = db.Column(db.DateTime, nullable=False, index=True)
paid = db.Column(db.Boolean, default=False)
paid_out = db.Column(db.Boolean, default=False)
total_amount = db.Column(db.Numeric(15, 2), nullable=False)
bet_count = db.Column(db.Integer, nullable=False)
# Metadata
created_at = db.Column(db.DateTime, default=datetime.utcnow)
# Relationships
details = db.relationship('BetDetail', backref='bet', lazy='dynamic', cascade='all, delete-orphan')
def to_dict(self):
"""Convert to dictionary for JSON serialization"""
return {
'id': self.id,
'uuid': self.uuid,
'sync_id': self.sync_id,
'client_id': self.client_id,
'fixture_id': self.fixture_id,
'bet_datetime': self.bet_datetime.isoformat() if self.bet_datetime else None,
'paid': self.paid,
'paid_out': self.paid_out,
'total_amount': float(self.total_amount) if self.total_amount else 0.0,
'bet_count': self.bet_count,
'created_at': self.created_at.isoformat() if self.created_at else None
}
def __repr__(self):
return f'<Bet {self.uuid} amount={self.total_amount}>'
class BetDetail(db.Model):
"""Individual bet details within a bet"""
__tablename__ = 'bet_details'
id = db.Column(db.Integer, primary_key=True)
bet_id = db.Column(db.Integer, db.ForeignKey('bets.id'), nullable=False, index=True)
match_id = db.Column(db.Integer, nullable=False, index=True)
match_number = db.Column(db.Integer, nullable=False)
outcome = db.Column(db.String(50), nullable=False)
amount = db.Column(db.Numeric(15, 2), nullable=False)
win_amount = db.Column(db.Numeric(15, 2), default=0.00)
result = db.Column(db.String(50), nullable=False)
# Metadata
created_at = db.Column(db.DateTime, default=datetime.utcnow)
def to_dict(self):
"""Convert to dictionary for JSON serialization"""
return {
'id': self.id,
'bet_id': self.bet_id,
'match_id': self.match_id,
'match_number': self.match_number,
'outcome': self.outcome,
'amount': float(self.amount) if self.amount else 0.0,
'win_amount': float(self.win_amount) if self.win_amount else 0.0,
'result': self.result,
'created_at': self.created_at.isoformat() if self.created_at else None
}
def __repr__(self):
return f'<BetDetail match={self.match_id} outcome={self.outcome} amount={self.amount}>'
class ExtractionStats(db.Model):
"""Extraction statistics for matches from client reports"""
__tablename__ = 'extraction_stats'
id = db.Column(db.Integer, primary_key=True)
sync_id = db.Column(db.Integer, db.ForeignKey('report_syncs.id'), nullable=False, index=True)
client_id = db.Column(db.String(255), nullable=False, index=True)
match_id = db.Column(db.Integer, nullable=False, index=True)
fixture_id = db.Column(db.String(255), nullable=False, index=True)
match_datetime = db.Column(db.DateTime, nullable=False)
total_bets = db.Column(db.Integer, nullable=False)
total_amount_collected = db.Column(db.Numeric(15, 2), nullable=False)
total_redistributed = db.Column(db.Numeric(15, 2), nullable=False)
actual_result = db.Column(db.String(50), nullable=False)
extraction_result = db.Column(db.String(50), nullable=False)
cap_applied = db.Column(db.Boolean, default=False)
cap_percentage = db.Column(db.Numeric(5, 2))
under_bets = db.Column(db.Integer, default=0)
under_amount = db.Column(db.Numeric(15, 2), default=0.00)
over_bets = db.Column(db.Integer, default=0)
over_amount = db.Column(db.Numeric(15, 2), default=0.00)
result_breakdown = db.Column(db.JSON)
# Metadata
created_at = db.Column(db.DateTime, default=datetime.utcnow)
def to_dict(self):
"""Convert to dictionary for JSON serialization"""
return {
'id': self.id,
'sync_id': self.sync_id,
'client_id': self.client_id,
'match_id': self.match_id,
'fixture_id': self.fixture_id,
'match_datetime': self.match_datetime.isoformat() if self.match_datetime else None,
'total_bets': self.total_bets,
'total_amount_collected': float(self.total_amount_collected) if self.total_amount_collected else 0.0,
'total_redistributed': float(self.total_redistributed) if self.total_redistributed else 0.0,
'actual_result': self.actual_result,
'extraction_result': self.extraction_result,
'cap_applied': self.cap_applied,
'cap_percentage': float(self.cap_percentage) if self.cap_percentage else None,
'under_bets': self.under_bets,
'under_amount': float(self.under_amount) if self.under_amount else 0.0,
'over_bets': self.over_bets,
'over_amount': float(self.over_amount) if self.over_amount else 0.0,
'result_breakdown': self.result_breakdown,
'created_at': self.created_at.isoformat() if self.created_at else None
}
def __repr__(self):
return f'<ExtractionStats match={self.match_id} collected={self.total_amount_collected}>'
class ReportSyncLog(db.Model):
"""Log all report synchronization operations"""
__tablename__ = 'report_sync_logs'
id = db.Column(db.Integer, primary_key=True)
sync_id = db.Column(db.String(255), nullable=False, index=True)
client_id = db.Column(db.String(255), nullable=False, index=True)
sync_timestamp = db.Column(db.DateTime, nullable=False, index=True)
# Operation details
operation_type = db.Column(db.Enum('new_sync', 'duplicate_sync', 'update_stats', 'error', name='sync_operation_type'),
nullable=False, index=True)
status = db.Column(db.Enum('success', 'failed', name='sync_status'), nullable=False, index=True)
# Statistics
bets_processed = db.Column(db.Integer, default=0)
bets_new = db.Column(db.Integer, default=0)
bets_duplicate = db.Column(db.Integer, default=0)
stats_processed = db.Column(db.Integer, default=0)
stats_new = db.Column(db.Integer, default=0)
stats_updated = db.Column(db.Integer, default=0)
# Summary data
total_payin = db.Column(db.Numeric(15, 2), default=0.00)
total_payout = db.Column(db.Numeric(15, 2), default=0.00)
net_profit = db.Column(db.Numeric(15, 2), default=0.00)
total_bets = db.Column(db.Integer, default=0)
total_matches = db.Column(db.Integer, default=0)
# Error information
error_message = db.Column(db.Text)
error_details = db.Column(db.JSON)
# Request metadata
ip_address = db.Column(db.String(45))
user_agent = db.Column(db.Text)
request_size = db.Column(db.Integer) # Size in bytes
# Processing time
processing_time_ms = db.Column(db.Integer) # Processing time in milliseconds
# Metadata
created_at = db.Column(db.DateTime, default=datetime.utcnow, index=True)
def to_dict(self):
"""Convert to dictionary for JSON serialization"""
return {
'id': self.id,
'sync_id': self.sync_id,
'client_id': self.client_id,
'sync_timestamp': self.sync_timestamp.isoformat() if self.sync_timestamp else None,
'operation_type': self.operation_type,
'status': self.status,
'bets_processed': self.bets_processed,
'bets_new': self.bets_new,
'bets_duplicate': self.bets_duplicate,
'stats_processed': self.stats_processed,
'stats_new': self.stats_new,
'stats_updated': self.stats_updated,
'total_payin': float(self.total_payin) if self.total_payin else 0.0,
'total_payout': float(self.total_payout) if self.total_payout else 0.0,
'net_profit': float(self.net_profit) if self.net_profit else 0.0,
'total_bets': self.total_bets,
'total_matches': self.total_matches,
'error_message': self.error_message,
'error_details': self.error_details,
'ip_address': self.ip_address,
'user_agent': self.user_agent,
'request_size': self.request_size,
'processing_time_ms': self.processing_time_ms,
'created_at': self.created_at.isoformat() if self.created_at else None
}
def __repr__(self):
return f'<ReportSyncLog {self.sync_id} {self.operation_type} {self.status}>'
\ No newline at end of file
...@@ -179,6 +179,8 @@ ...@@ -179,6 +179,8 @@
<a href="{{ url_for('main.dashboard') }}">Dashboard</a> <a href="{{ url_for('main.dashboard') }}">Dashboard</a>
<a href="{{ url_for('main.fixtures') }}">Fixtures</a> <a href="{{ url_for('main.fixtures') }}">Fixtures</a>
<a href="{{ url_for('main.clients') }}">Clients</a> <a href="{{ url_for('main.clients') }}">Clients</a>
<a href="{{ url_for('main.reports') }}">Reports</a>
<a href="{{ url_for('main.sync_logs') }}">Sync Logs</a>
<a href="{{ url_for('main.uploads') }}">Uploads</a> <a href="{{ url_for('main.uploads') }}">Uploads</a>
<a href="{{ url_for('main.statistics') }}">Statistics</a> <a href="{{ url_for('main.statistics') }}">Statistics</a>
<a href="{{ url_for('main.user_tokens') }}">API Tokens</a> <a href="{{ url_for('main.user_tokens') }}">API Tokens</a>
......
{% extends "base.html" %}
{% block title %}Report Details - {{ report.sync_id }}{% endblock %}
{% block content %}
<div class="container-fluid">
<div class="row">
<div class="col-12">
<div class="d-flex justify-content-between align-items-center mb-4">
<div>
<h1>Report Details</h1>
<nav aria-label="breadcrumb">
<ol class="breadcrumb">
<li class="breadcrumb-item"><a href="{{ url_for('main.reports') }}">Reports</a></li>
<li class="breadcrumb-item active">{{ report.sync_id }}</li>
</ol>
</nav>
</div>
<a href="{{ url_for('main.reports') }}" class="btn btn-secondary">
<i class="fas fa-arrow-left"></i> Back to Reports
</a>
</div>
<!-- Summary Card -->
<div class="card mb-4">
<div class="card-header">
<h5 class="mb-0"><i class="fas fa-chart-line"></i> Summary</h5>
</div>
<div class="card-body">
<div class="row">
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Sync ID</h6>
<p class="mb-0"><code>{{ report.sync_id }}</code></p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Client ID</h6>
<p class="mb-0">{{ report.client_id }}</p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Sync Timestamp</h6>
<p class="mb-0">{{ report.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if report.sync_timestamp else '' }}</p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Date Range</h6>
<p class="mb-0">{{ report.date_range }}</p>
</div>
</div>
</div>
<div class="row mt-3">
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Start Date</h6>
<p class="mb-0">{{ report.start_date.strftime('%Y-%m-%d') if report.start_date else '' }}</p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">End Date</h6>
<p class="mb-0">{{ report.end_date.strftime('%Y-%m-%d') if report.end_date else '' }}</p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Total Bets</h6>
<p class="mb-0">{{ report.total_bets }}</p>
</div>
</div>
<div class="col-md-3">
<div class="p-3 bg-light rounded">
<h6 class="text-muted mb-1">Total Matches</h6>
<p class="mb-0">{{ report.total_matches }}</p>
</div>
</div>
</div>
<div class="row mt-3">
<div class="col-md-4">
<div class="p-3 bg-primary text-white rounded">
<h6 class="mb-1">Total Payin</h6>
<h3 class="mb-0">{{ "{:,.2f}".format(report.total_payin) if report.total_payin else '0.00' }}</h3>
</div>
</div>
<div class="col-md-4">
<div class="p-3 bg-info text-white rounded">
<h6 class="mb-1">Total Payout</h6>
<h3 class="mb-0">{{ "{:,.2f}".format(report.total_payout) if report.total_payout else '0.00' }}</h3>
</div>
</div>
<div class="col-md-4">
<div class="p-3 {% if report.net_profit >= 0 %}bg-success{% else %}bg-danger{% endif %} text-white rounded">
<h6 class="mb-1">Net Profit</h6>
<h3 class="mb-0">{{ "{:,.2f}".format(report.net_profit) if report.net_profit else '0.00' }}</h3>
</div>
</div>
</div>
</div>
</div>
<!-- Tabs for Bets and Extraction Stats -->
<ul class="nav nav-tabs mb-3" id="reportTabs" role="tablist">
<li class="nav-item" role="presentation">
<button class="nav-link active" id="bets-tab" data-bs-toggle="tab" data-bs-target="#bets" type="button" role="tab">
<i class="fas fa-ticket-alt"></i> Bets ({{ bets|length }})
</button>
</li>
<li class="nav-item" role="presentation">
<button class="nav-link" id="stats-tab" data-bs-toggle="tab" data-bs-target="#stats" type="button" role="tab">
<i class="fas fa-chart-bar"></i> Extraction Stats ({{ stats|length }})
</button>
</li>
</ul>
<div class="tab-content" id="reportTabsContent">
<!-- Bets Tab -->
<div class="tab-pane fade show active" id="bets" role="tabpanel">
<div class="card">
<div class="card-body">
{% if bets %}
<div class="table-responsive">
<table class="table table-striped table-hover">
<thead>
<tr>
<th>Bet UUID</th>
<th>Fixture ID</th>
<th>Bet DateTime</th>
<th>Total Amount</th>
<th>Bet Count</th>
<th>Paid</th>
<th>Paid Out</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for bet in bets %}
<tr>
<td><code>{{ bet.uuid }}</code></td>
<td>{{ bet.fixture_id }}</td>
<td>{{ bet.bet_datetime.strftime('%Y-%m-%d %H:%M:%S') if bet.bet_datetime else '' }}</td>
<td class="text-end">{{ "{:,.2f}".format(bet.total_amount) if bet.total_amount else '0.00' }}</td>
<td class="text-center">{{ bet.bet_count }}</td>
<td class="text-center">
{% if bet.paid %}
<i class="fas fa-check-circle text-success"></i>
{% else %}
<i class="fas fa-times-circle text-danger"></i>
{% endif %}
</td>
<td class="text-center">
{% if bet.paid_out %}
<i class="fas fa-check-circle text-success"></i>
{% else %}
<i class="fas fa-times-circle text-danger"></i>
{% endif %}
</td>
<td>
<button type="button" class="btn btn-sm btn-info" data-bs-toggle="collapse" data-bs-target="#bet-details-{{ bet.id }}">
<i class="fas fa-list"></i> Details
</button>
</td>
</tr>
<tr>
<td colspan="8" class="p-0">
<div class="collapse" id="bet-details-{{ bet.id }}">
<div class="p-3 bg-light">
<h6>Bet Details</h6>
<table class="table table-sm">
<thead>
<tr>
<th>Match ID</th>
<th>Match Number</th>
<th>Outcome</th>
<th>Amount</th>
<th>Win Amount</th>
<th>Result</th>
</tr>
</thead>
<tbody>
{% for detail in bet.details %}
<tr>
<td>{{ detail.match_id }}</td>
<td>{{ detail.match_number }}</td>
<td>{{ detail.outcome }}</td>
<td class="text-end">{{ "{:,.2f}".format(detail.amount) if detail.amount else '0.00' }}</td>
<td class="text-end">{{ "{:,.2f}".format(detail.win_amount) if detail.win_amount else '0.00' }}</td>
<td>
<span class="badge bg-{% if detail.result == 'won' %}success{% elif detail.result == 'lost' %}danger{% elif detail.result == 'pending' %}warning{% else %}secondary{% endif %}">
{{ detail.result }}
</span>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="text-center py-5">
<i class="fas fa-ticket-alt fa-3x text-muted mb-3"></i>
<h4 class="text-muted">No bets found</h4>
</div>
{% endif %}
</div>
</div>
</div>
<!-- Extraction Stats Tab -->
<div class="tab-pane fade" id="stats" role="tabpanel">
<div class="card">
<div class="card-body">
{% if stats %}
<div class="table-responsive">
<table class="table table-striped table-hover">
<thead>
<tr>
<th>Match ID</th>
<th>Fixture ID</th>
<th>Match DateTime</th>
<th>Total Bets</th>
<th>Total Collected</th>
<th>Total Redistributed</th>
<th>Actual Result</th>
<th>Extraction Result</th>
<th>CAP Applied</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for stat in stats %}
<tr>
<td>{{ stat.match_id }}</td>
<td>{{ stat.fixture_id }}</td>
<td>{{ stat.match_datetime.strftime('%Y-%m-%d %H:%M:%S') if stat.match_datetime else '' }}</td>
<td class="text-center">{{ stat.total_bets }}</td>
<td class="text-end">{{ "{:,.2f}".format(stat.total_amount_collected) if stat.total_amount_collected else '0.00' }}</td>
<td class="text-end">{{ "{:,.2f}".format(stat.total_redistributed) if stat.total_redistributed else '0.00' }}</td>
<td>{{ stat.actual_result }}</td>
<td>{{ stat.extraction_result }}</td>
<td class="text-center">
{% if stat.cap_applied %}
<i class="fas fa-check-circle text-success"></i> {{ stat.cap_percentage }}%
{% else %}
<i class="fas fa-times-circle text-danger"></i>
{% endif %}
</td>
<td>
<button type="button" class="btn btn-sm btn-info" data-bs-toggle="collapse" data-bs-target="#stats-details-{{ stat.id }}">
<i class="fas fa-list"></i> Details
</button>
</td>
</tr>
<tr>
<td colspan="10" class="p-0">
<div class="collapse" id="stats-details-{{ stat.id }}">
<div class="p-3 bg-light">
<h6>Extraction Details</h6>
<div class="row">
<div class="col-md-6">
<table class="table table-sm">
<tr>
<th>Under Bets:</th>
<td>{{ stat.under_bets }}</td>
</tr>
<tr>
<th>Under Amount:</th>
<td>{{ "{:,.2f}".format(stat.under_amount) if stat.under_amount else '0.00' }}</td>
</tr>
<tr>
<th>Over Bets:</th>
<td>{{ stat.over_bets }}</td>
</tr>
<tr>
<th>Over Amount:</th>
<td>{{ "{:,.2f}".format(stat.over_amount) if stat.over_amount else '0.00' }}</td>
</tr>
</table>
</div>
<div class="col-md-6">
<h6>Result Breakdown</h6>
{% if stat.result_breakdown %}
<table class="table table-sm">
<thead>
<tr>
<th>Outcome</th>
<th>Bets</th>
<th>Amount</th>
<th>Coefficient</th>
</tr>
</thead>
<tbody>
{% for outcome, data in stat.result_breakdown.items() %}
<tr>
<td>{{ outcome }}</td>
<td>{{ data.bets }}</td>
<td>{{ "{:,.2f}".format(data.amount) if data.amount else '0.00' }}</td>
<td>{{ data.coefficient }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% else %}
<p class="text-muted">No breakdown data available</p>
{% endif %}
</div>
</div>
</div>
</div>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="text-center py-5">
<i class="fas fa-chart-bar fa-3x text-muted mb-3"></i>
<h4 class="text-muted">No extraction stats found</h4>
</div>
{% endif %}
</div>
</div>
</div>
</div>
</div>
</div>
</div>
{% endblock %}
\ No newline at end of file
{% extends "base.html" %}
{% block title %}Reports{% endblock %}
{% block content %}
<div class="container-fluid">
<div class="row">
<div class="col-12">
<div class="d-flex justify-content-between align-items-center mb-4">
<h1>Reports</h1>
<div class="btn-group">
<button type="button" class="btn btn-primary dropdown-toggle" data-bs-toggle="dropdown">
<i class="fas fa-download"></i> Export
</button>
<ul class="dropdown-menu">
<li><a class="dropdown-item" href="{{ url_for('main.reports', export='csv', **filters) }}">Export as CSV</a></li>
<li><a class="dropdown-item" href="{{ url_for('main.reports', export='xlsx', **filters) }}">Export as Excel</a></li>
<li><a class="dropdown-item" href="{{ url_for('main.reports', export='pdf', **filters) }}">Export as PDF</a></li>
</ul>
</div>
</div>
<!-- Filters -->
<div class="card mb-4">
<div class="card-header">
<h5 class="mb-0"><i class="fas fa-filter"></i> Filters</h5>
</div>
<div class="card-body">
<form method="GET" action="{{ url_for('main.reports') }}" class="row g-3">
<div class="col-md-3">
<label for="client_id" class="form-label">Client ID</label>
<select class="form-select" id="client_id" name="client_id">
<option value="">All Clients</option>
{% for cid in client_ids %}
<option value="{{ cid }}" {% if filters.client_id == cid %}selected{% endif %}>{{ cid }}</option>
{% endfor %}
</select>
</div>
<div class="col-md-2">
<label for="date_range" class="form-label">Date Range</label>
<select class="form-select" id="date_range" name="date_range">
<option value="">All</option>
<option value="today" {% if filters.date_range == 'today' %}selected{% endif %}>Today</option>
<option value="yesterday" {% if filters.date_range == 'yesterday' %}selected{% endif %}>Yesterday</option>
<option value="week" {% if filters.date_range == 'week' %}selected{% endif %}>This Week</option>
<option value="all" {% if filters.date_range == 'all' %}selected{% endif %}>All Time</option>
</select>
</div>
<div class="col-md-2">
<label for="start_date" class="form-label">Start Date</label>
<input type="date" class="form-control" id="start_date" name="start_date" value="{{ filters.start_date }}">
</div>
<div class="col-md-2">
<label for="end_date" class="form-label">End Date</label>
<input type="date" class="form-control" id="end_date" name="end_date" value="{{ filters.end_date }}">
</div>
<div class="col-md-2">
<label for="sort_by" class="form-label">Sort By</label>
<select class="form-select" id="sort_by" name="sort_by">
<option value="sync_timestamp" {% if filters.sort_by == 'sync_timestamp' %}selected{% endif %}>Sync Timestamp</option>
<option value="client_id" {% if filters.sort_by == 'client_id' %}selected{% endif %}>Client ID</option>
<option value="total_payin" {% if filters.sort_by == 'total_payin' %}selected{% endif %}>Total Payin</option>
<option value="total_payout" {% if filters.sort_by == 'total_payout' %}selected{% endif %}>Total Payout</option>
<option value="net_profit" {% if filters.sort_by == 'net_profit' %}selected{% endif %}>Net Profit</option>
<option value="total_bets" {% if filters.sort_by == 'total_bets' %}selected{% endif %}>Total Bets</option>
</select>
</div>
<div class="col-md-1">
<label for="sort_order" class="form-label">Order</label>
<select class="form-select" id="sort_order" name="sort_order">
<option value="desc" {% if filters.sort_order == 'desc' %}selected{% endif %}>Desc</option>
<option value="asc" {% if filters.sort_order == 'asc' %}selected{% endif %}>Asc</option>
</select>
</div>
<div class="col-12">
<button type="submit" class="btn btn-primary">
<i class="fas fa-search"></i> Apply Filters
</button>
<a href="{{ url_for('main.reports') }}" class="btn btn-secondary">
<i class="fas fa-times"></i> Clear Filters
</a>
</div>
</form>
</div>
</div>
<!-- Reports Table -->
<div class="card">
<div class="card-body">
{% if reports %}
<div class="table-responsive">
<table class="table table-striped table-hover">
<thead>
<tr>
<th>Sync ID</th>
<th>Client ID</th>
<th>Sync Timestamp</th>
<th>Date Range</th>
<th>Total Payin</th>
<th>Total Payout</th>
<th>Net Profit</th>
<th>Total Bets</th>
<th>Total Matches</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for report in reports %}
<tr>
<td><code>{{ report.sync_id }}</code></td>
<td>{{ report.client_id }}</td>
<td>{{ report.sync_timestamp.strftime('%Y-%m-%d %H:%M:%S') if report.sync_timestamp else '' }}</td>
<td>{{ report.date_range }}</td>
<td class="text-end">{{ "{:,.2f}".format(report.total_payin) if report.total_payin else '0.00' }}</td>
<td class="text-end">{{ "{:,.2f}".format(report.total_payout) if report.total_payout else '0.00' }}</td>
<td class="text-end {% if report.net_profit >= 0 %}text-success{% else %}text-danger{% endif %}">
{{ "{:,.2f}".format(report.net_profit) if report.net_profit else '0.00' }}
</td>
<td class="text-center">{{ report.total_bets }}</td>
<td class="text-center">{{ report.total_matches }}</td>
<td>
<a href="{{ url_for('main.report_detail', sync_id=report.id) }}" class="btn btn-sm btn-info">
<i class="fas fa-eye"></i> View
</a>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<!-- Pagination -->
{% if pagination and pagination.pages > 1 %}
<nav aria-label="Page navigation">
<ul class="pagination justify-content-center">
{% if pagination.has_prev %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.reports', page=pagination.prev_num, **filters) }}">Previous</a>
</li>
{% endif %}
{% for page_num in pagination.iter_pages() %}
{% if page_num %}
{% if page_num == pagination.page %}
<li class="page-item active"><span class="page-link">{{ page_num }}</span></li>
{% else %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.reports', page=page_num, **filters) }}">{{ page_num }}</a>
</li>
{% endif %}
{% else %}
<li class="page-item disabled"><span class="page-link">...</span></li>
{% endif %}
{% endfor %}
{% if pagination.has_next %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.reports', page=pagination.next_num, **filters) }}">Next</a>
</li>
{% endif %}
</ul>
</nav>
<p class="text-center text-muted">
Showing {{ pagination.per_page * (pagination.page - 1) + 1 }} to
{{ min(pagination.per_page * pagination.page, pagination.total) }} of
{{ pagination.total }} reports
</p>
{% endif %}
{% else %}
<div class="text-center py-5">
<i class="fas fa-file-alt fa-3x text-muted mb-3"></i>
<h4 class="text-muted">No reports found</h4>
<p class="text-muted">Try adjusting your filters or wait for clients to sync reports.</p>
</div>
{% endif %}
</div>
</div>
</div>
</div>
</div>
{% endblock %}
\ No newline at end of file
{% extends "base.html" %}
{% block title %}Sync Logs{% endblock %}
{% block content %}
<div class="container-fluid">
<div class="row">
<div class="col-12">
<div class="d-flex justify-content-between align-items-center mb-4">
<h1>Sync Logs</h1>
<div class="btn-group">
<button type="button" class="btn btn-primary dropdown-toggle" data-bs-toggle="dropdown">
<i class="fas fa-download"></i> Export
</button>
<ul class="dropdown-menu">
<li><a class="dropdown-item" href="{{ url_for('main.sync_logs', export='csv', **filters) }}">Export as CSV</a></li>
<li><a class="dropdown-item" href="{{ url_for('main.sync_logs', export='xlsx', **filters) }}">Export as Excel</a></li>
<li><a class="dropdown-item" href="{{ url_for('main.sync_logs', export='pdf', **filters) }}">Export as PDF</a></li>
</ul>
</div>
</div>
<!-- Filters -->
<div class="card mb-4">
<div class="card-header">
<h5 class="mb-0"><i class="fas fa-filter"></i> Filters</h5>
</div>
<div class="card-body">
<form method="GET" action="{{ url_for('main.sync_logs') }}" class="row g-3">
<div class="col-md-2">
<label for="client_id" class="form-label">Client ID</label>
<select class="form-select" id="client_id" name="client_id">
<option value="">All Clients</option>
{% for cid in client_ids %}
<option value="{{ cid }}" {% if filters.client_id == cid %}selected{% endif %}>{{ cid }}</option>
{% endfor %}
</select>
</div>
<div class="col-md-2">
<label for="operation_type" class="form-label">Operation Type</label>
<select class="form-select" id="operation_type" name="operation_type">
<option value="">All Types</option>
<option value="new_sync" {% if filters.operation_type == 'new_sync' %}selected{% endif %}>New Sync</option>
<option value="duplicate_sync" {% if filters.operation_type == 'duplicate_sync' %}selected{% endif %}>Duplicate Sync</option>
<option value="update_stats" {% if filters.operation_type == 'update_stats' %}selected{% endif %}>Update Stats</option>
<option value="error" {% if filters.operation_type == 'error' %}selected{% endif %}>Error</option>
</select>
</div>
<div class="col-md-2">
<label for="status" class="form-label">Status</label>
<select class="form-select" id="status" name="status">
<option value="">All Statuses</option>
<option value="success" {% if filters.status == 'success' %}selected{% endif %}>Success</option>
<option value="failed" {% if filters.status == 'failed' %}selected{% endif %}>Failed</option>
</select>
</div>
<div class="col-md-2">
<label for="start_date" class="form-label">Start Date</label>
<input type="date" class="form-control" id="start_date" name="start_date" value="{{ filters.start_date }}">
</div>
<div class="col-md-2">
<label for="end_date" class="form-label">End Date</label>
<input type="date" class="form-control" id="end_date" name="end_date" value="{{ filters.end_date }}">
</div>
<div class="col-md-2">
<label for="search" class="form-label">Search</label>
<input type="text" class="form-control" id="search" name="search" value="{{ filters.search }}" placeholder="Sync ID, Client ID, or Error">
</div>
<div class="col-12">
<button type="submit" class="btn btn-primary">
<i class="fas fa-search"></i> Apply Filters
</button>
<a href="{{ url_for('main.sync_logs') }}" class="btn btn-secondary">
<i class="fas fa-times"></i> Clear Filters
</a>
</div>
</form>
</div>
</div>
<!-- Logs Table -->
<div class="card">
<div class="card-body">
{% if logs %}
<div class="table-responsive">
<table class="table table-striped table-hover">
<thead>
<tr>
<th>Sync ID</th>
<th>Client ID</th>
<th>Operation Type</th>
<th>Status</th>
<th>Bets Processed</th>
<th>Bets New</th>
<th>Stats Processed</th>
<th>Stats New</th>
<th>Stats Updated</th>
<th>Total Payin</th>
<th>Total Payout</th>
<th>Net Profit</th>
<th>Processing Time</th>
<th>Created At</th>
<th>IP Address</th>
</tr>
</thead>
<tbody>
{% for log in logs %}
<tr class="{% if log.status == 'failed' %}table-danger{% elif log.operation_type == 'error' %}table-warning{% endif %}">
<td><code>{{ log.sync_id }}</code></td>
<td>{{ log.client_id }}</td>
<td>
<span class="badge bg-{% if log.operation_type == 'new_sync' %}primary{% elif log.operation_type == 'duplicate_sync' %}info{% elif log.operation_type == 'update_stats' %}warning{% else %}danger{% endif %}">
{{ log.operation_type|replace('_', ' ')|title }}
</span>
</td>
<td>
{% if log.status == 'success' %}
<i class="fas fa-check-circle text-success"></i> Success
{% else %}
<i class="fas fa-times-circle text-danger"></i> Failed
{% endif %}
</td>
<td class="text-center">{{ log.bets_processed }}</td>
<td class="text-center">{{ log.bets_new }}</td>
<td class="text-center">{{ log.stats_processed }}</td>
<td class="text-center">{{ log.stats_new }}</td>
<td class="text-center">{{ log.stats_updated }}</td>
<td class="text-end">{{ "{:,.2f}".format(log.total_payin) if log.total_payin else '0.00' }}</td>
<td class="text-end">{{ "{:,.2f}".format(log.total_payout) if log.total_payout else '0.00' }}</td>
<td class="text-end {% if log.net_profit >= 0 %}text-success{% else %}text-danger{% endif %}">
{{ "{:,.2f}".format(log.net_profit) if log.net_profit else '0.00' }}
</td>
<td class="text-center">{{ log.processing_time_ms }}ms</td>
<td>{{ log.created_at.strftime('%Y-%m-%d %H:%M:%S') if log.created_at else '' }}</td>
<td>{{ log.ip_address or '-' }}</td>
</tr>
{% if log.error_message %}
<tr class="table-light">
<td colspan="16" class="p-2">
<div class="alert alert-danger mb-0">
<strong>Error:</strong> {{ log.error_message }}
{% if log.error_details %}
<br>
<small><code>{{ log.error_details }}</code></small>
{% endif %}
</div>
</td>
</tr>
{% endif %}
{% endfor %}
</tbody>
</table>
</div>
<!-- Pagination -->
{% if pagination and pagination.pages > 1 %}
<nav aria-label="Page navigation">
<ul class="pagination justify-content-center">
{% if pagination.has_prev %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.sync_logs', page=pagination.prev_num, **filters) }}">Previous</a>
</li>
{% endif %}
{% for page_num in pagination.iter_pages() %}
{% if page_num %}
{% if page_num == pagination.page %}
<li class="page-item active"><span class="page-link">{{ page_num }}</span></li>
{% else %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.sync_logs', page=page_num, **filters) }}">{{ page_num }}</a>
</li>
{% endif %}
{% else %}
<li class="page-item disabled"><span class="page-link">...</span></li>
{% endif %}
{% endfor %}
{% if pagination.has_next %}
<li class="page-item">
<a class="page-link" href="{{ url_for('main.sync_logs', page=pagination.next_num, **filters) }}">Next</a>
</li>
{% endif %}
</ul>
</nav>
<p class="text-center text-muted">
Showing {{ pagination.per_page * (pagination.page - 1) + 1 }} to
{{ min(pagination.per_page * pagination.page, pagination.total) }} of
{{ pagination.total }} logs
</p>
{% endif %}
{% else %}
<div class="text-center py-5">
<i class="fas fa-history fa-3x text-muted mb-3"></i>
<h4 class="text-muted">No sync logs found</h4>
<p class="text-muted">Try adjusting your filters or wait for clients to sync reports.</p>
</div>
{% endif %}
</div>
</div>
</div>
</div>
</div>
{% endblock %}
\ No newline at end of file
...@@ -20,4 +20,5 @@ watchdog==6.0.0 ...@@ -20,4 +20,5 @@ watchdog==6.0.0
click==8.1.7 click==8.1.7
colorlog==6.8.2 colorlog==6.8.2
marshmallow==3.23.1 marshmallow==3.23.1
redis==5.2.0 redis==5.2.0
\ No newline at end of file weasyprint==60.2
\ No newline at end of file
"""
Test script for reports synchronization API
"""
import requests
import json
from datetime import datetime, timedelta
# Configuration
API_URL = "http://localhost:5000/api/reports/sync"
API_TOKEN = "your-api-token-here" # Replace with actual token
def create_test_sync_data():
"""Create test sync data matching the API specification"""
now = datetime.utcnow()
sync_data = {
"sync_id": f"sync_{now.strftime('%Y%m%d_%H%M%S')}_test1234",
"client_id": "test_client_001",
"sync_timestamp": now.isoformat(),
"date_range": "today",
"start_date": now.replace(hour=0, minute=0, second=0).isoformat(),
"end_date": now.isoformat(),
"bets": [
{
"uuid": "550e8400-e29b-41d4-a716-4466554400001",
"fixture_id": "fixture_test_001",
"bet_datetime": (now - timedelta(minutes=30)).isoformat(),
"paid": False,
"paid_out": False,
"total_amount": 500.00,
"bet_count": 3,
"details": [
{
"match_id": 123,
"match_number": 1,
"outcome": "WIN1",
"amount": 200.00,
"win_amount": 0.00,
"result": "pending"
},
{
"match_id": 124,
"match_number": 2,
"outcome": "X1",
"amount": 150.00,
"win_amount": 0.00,
"result": "pending"
},
{
"match_id": 125,
"match_number": 3,
"outcome": "WIN2",
"amount": 150.00,
"win_amount": 0.00,
"result": "pending"
}
]
}
],
"extraction_stats": [
{
"match_id": 123,
"fixture_id": "fixture_test_001",
"match_datetime": (now - timedelta(hours=1)).isoformat(),
"total_bets": 45,
"total_amount_collected": 15000.00,
"total_redistributed": 10500.00,
"actual_result": "WIN1",
"extraction_result": "WIN1",
"cap_applied": True,
"cap_percentage": 70.0,
"under_bets": 20,
"under_amount": 6000.00,
"over_bets": 25,
"over_amount": 9000.00,
"result_breakdown": {
"WIN1": {"bets": 20, "amount": 6000.00, "coefficient": 2.5},
"X1": {"bets": 10, "amount": 3000.00, "coefficient": 3.0},
"WIN2": {"bets": 15, "amount": 6000.00, "coefficient": 2.0}
}
}
],
"summary": {
"total_payin": 15000.00,
"total_payout": 10500.00,
"net_profit": 4500.00,
"total_bets": 45,
"total_matches": 1
}
}
return sync_data
def test_sync_endpoint():
"""Test the reports sync endpoint"""
print("=" * 80)
print("Testing Reports Synchronization API")
print("=" * 80)
# Create test data
sync_data = create_test_sync_data()
print(f"\n1. Preparing sync data:")
print(f" Sync ID: {sync_data['sync_id']}")
print(f" Client ID: {sync_data['client_id']}")
print(f" Bets: {len(sync_data['bets'])}")
print(f" Extraction Stats: {len(sync_data['extraction_stats'])}")
print(f" Total Payin: {sync_data['summary']['total_payin']}")
print(f" Total Payout: {sync_data['summary']['total_payout']}")
print(f" Net Profit: {sync_data['summary']['net_profit']}")
# Prepare headers
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {API_TOKEN}"
}
print(f"\n2. Sending POST request to: {API_URL}")
print(f" Headers: Content-Type: application/json, Authorization: Bearer ***")
try:
# Send request
response = requests.post(API_URL, json=sync_data, headers=headers, timeout=60)
print(f"\n3. Response received:")
print(f" Status Code: {response.status_code}")
print(f" Response Time: {response.elapsed.total_seconds():.2f}s")
# Parse response
try:
response_data = response.json()
print(f"\n4. Response Data:")
print(json.dumps(response_data, indent=2))
# Check if successful
if response.status_code == 200 and response_data.get('success'):
print(f"\n✓ SUCCESS: Report synchronized successfully!")
print(f" Synced Count: {response_data.get('synced_count', 0)}")
print(f" Server Timestamp: {response_data.get('server_timestamp')}")
return True
else:
print(f"\n✗ FAILED: {response_data.get('error', 'Unknown error')}")
print(f" Details: {response_data.get('details', 'No details')}")
return False
except json.JSONDecodeError:
print(f"\n✗ FAILED: Could not parse JSON response")
print(f" Response Body: {response.text[:500]}")
return False
except requests.exceptions.Timeout:
print(f"\n✗ FAILED: Request timed out after 60 seconds")
return False
except requests.exceptions.ConnectionError:
print(f"\n✗ FAILED: Could not connect to server")
print(f" Make sure the server is running at {API_URL}")
return False
except Exception as e:
print(f"\n✗ FAILED: Unexpected error occurred")
print(f" Error: {str(e)}")
return False
def test_idempotency():
"""Test that duplicate sync_id is handled correctly"""
print("\n" + "=" * 80)
print("Testing Idempotency (Duplicate Sync ID)")
print("=" * 80)
# Create test data
sync_data = create_test_sync_data()
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {API_TOKEN}"
}
print("\n1. Sending first sync request...")
try:
response1 = requests.post(API_URL, json=sync_data, headers=headers, timeout=60)
print(f" Status: {response1.status_code}")
if response1.status_code == 200:
print(" ✓ First sync successful")
print("\n2. Sending duplicate sync request (same sync_id)...")
response2 = requests.post(API_URL, json=sync_data, headers=headers, timeout=60)
print(f" Status: {response2.status_code}")
if response2.status_code == 200:
data2 = response2.json()
if data2.get('success') and 'already' in data2.get('message', '').lower():
print(" ✓ Duplicate detected and handled correctly")
print(f" Message: {data2.get('message')}")
return True
else:
print(" ✗ Duplicate not detected properly")
return False
else:
print(" ✗ Second request failed")
return False
else:
print(" ✗ First sync failed")
return False
except Exception as e:
print(f" ✗ Error: {str(e)}")
return False
def test_validation():
"""Test input validation"""
print("\n" + "=" * 80)
print("Testing Input Validation")
print("=" * 80)
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {API_TOKEN}"
}
# Test 1: Missing required field
print("\n1. Testing missing required field (sync_id)...")
invalid_data = create_test_sync_data()
del invalid_data['sync_id']
response = requests.post(API_URL, json=invalid_data, headers=headers, timeout=60)
if response.status_code == 400:
print(" ✓ Correctly rejected (400 Bad Request)")
data = response.json()
print(f" Error: {data.get('error')}")
else:
print(f" ✗ Expected 400, got {response.status_code}")
# Test 2: Invalid UUID format
print("\n2. Testing invalid UUID format...")
invalid_data = create_test_sync_data()
invalid_data['bets'][0]['uuid'] = 'invalid-uuid'
response = requests.post(API_URL, json=invalid_data, headers=headers, timeout=60)
if response.status_code == 400:
print(" ✓ Correctly rejected (400 Bad Request)")
data = response.json()
print(f" Error: {data.get('error')}")
else:
print(f" ✗ Expected 400, got {response.status_code}")
# Test 3: Invalid datetime format
print("\n3. Testing invalid datetime format...")
invalid_data = create_test_sync_data()
invalid_data['sync_timestamp'] = 'invalid-datetime'
response = requests.post(API_URL, json=invalid_data, headers=headers, timeout=60)
if response.status_code == 400:
print(" ✓ Correctly rejected (400 Bad Request)")
data = response.json()
print(f" Error: {data.get('error')}")
else:
print(f" ✗ Expected 400, got {response.status_code}")
def test_authentication():
"""Test authentication requirements"""
print("\n" + "=" * 80)
print("Testing Authentication")
print("=" * 80)
sync_data = create_test_sync_data()
# Test 1: No authentication
print("\n1. Testing without authentication...")
headers = {"Content-Type": "application/json"}
response = requests.post(API_URL, json=sync_data, headers=headers, timeout=60)
if response.status_code == 401:
print(" ✓ Correctly rejected (401 Unauthorized)")
data = response.json()
print(f" Error: {data.get('error')}")
else:
print(f" ✗ Expected 401, got {response.status_code}")
# Test 2: Invalid token
print("\n2. Testing with invalid token...")
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer invalid-token-12345"
}
response = requests.post(API_URL, json=sync_data, headers=headers, timeout=60)
if response.status_code == 401:
print(" ✓ Correctly rejected (401 Unauthorized)")
data = response.json()
print(f" Error: {data.get('error')}")
else:
print(f" ✗ Expected 401, got {response.status_code}")
def main():
"""Run all tests"""
print("\n" + "=" * 80)
print("REPORTS SYNCHRONIZATION API TEST SUITE")
print("=" * 80)
print(f"\nAPI URL: {API_URL}")
print(f"API Token: {API_TOKEN[:20]}..." if len(API_TOKEN) > 20 else f"API Token: {API_TOKEN}")
print(f"Test Time: {datetime.utcnow().isoformat()}")
# Run tests
results = []
# Test authentication first
results.append(("Authentication", test_authentication()))
# Test validation
results.append(("Validation", test_validation()))
# Test main sync endpoint
results.append(("Sync Endpoint", test_sync_endpoint()))
# Test idempotency
results.append(("Idempotency", test_idempotency()))
# Print summary
print("\n" + "=" * 80)
print("TEST SUMMARY")
print("=" * 80)
for test_name, passed in results:
status = "✓ PASSED" if passed else "✗ FAILED"
print(f"{test_name:.<50} {status}")
total_tests = len(results)
passed_tests = sum(1 for _, passed in results if passed)
print(f"\nTotal: {passed_tests}/{total_tests} tests passed")
if passed_tests == total_tests:
print("\n🎉 All tests passed!")
return 0
else:
print(f"\n⚠️ {total_tests - passed_tests} test(s) failed")
return 1
if __name__ == "__main__":
import sys
sys.exit(main())
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment