This document describes the implementation of the reports sync API endpoint and web interface according to the Reports Synchronization Protocol Documentation.
## Changes Made
### 1. Database Model Updates
#### ReportSync Model (`app/models.py`)
-**Added**: `cap_compensation_balance` field to track the CAP redistribution balance at the time of sync
-**Type**: `NUMERIC(15, 2)` with default value 0.00
-**Purpose**: Stores the accumulated shortfall from cap compensation system for historical tracking
#### Updated `to_dict()` method
- Now includes `cap_compensation_balance` in JSON serialization
### 2. API Endpoint Updates (`app/api/routes.py`)
#### `/api/reports/sync` Endpoint
**New Features:**
1.**Cap Compensation Balance Handling**
- Accepts `cap_compensation_balance` field in request payload
- Stores the value in the ReportSync record
- Tracks historical CAP balance over time
2.**Differentiation Between New and Updated Records**
-**Bets**:
- Checks for existing bet UUIDs
- Updates existing bets if fields have changed (total_amount, paid, paid_out)
- Tracks `bets_new` and `bets_duplicate` counts
- Skips unchanged duplicate bets
-**Extraction Stats**:
- Checks for existing stats by match_id + client_id
- Updates existing stats with new data
- Tracks `stats_new` and `stats_updated` counts
3.**Enhanced Logging**
-`ReportSyncLog` now tracks:
-`bets_new`: Number of new bets added
-`bets_duplicate`: Number of duplicate bets (updated or skipped)
-`stats_new`: Number of new extraction stats
-`stats_updated`: Number of updated extraction stats
- Joins ReportSync with ClientActivity and APIToken
- Provides client_id and token_name for dropdown
- Filters by user's tokens for non-admin users
3.**Enhanced Filtering**
- Applies date range filters to query
- Supports client_id filtering
- Maintains pagination and sorting
## Database Migration
### Automatic Migration System
The application uses an automatic migration system that runs all pending migrations at application launch. The migration to add `cap_compensation_balance` is already integrated into this system.
**Migration Class**: `Migration_011_AddCapCompensationBalance` in [`app/database/migrations.py`](app/database/migrations.py:777)
**What it does:**
1. Checks if `report_syncs` table exists
2. Checks if `cap_compensation_balance` column already exists
3. Adds `cap_compensation_balance DECIMAL(15,2) NOT NULL DEFAULT 0.00` if not present
The reports synchronization protocol sends betting and extraction statistics data from the client to the server. The system uses **incremental synchronization** - only new or updated records are sent after the initial full sync.
**IMPORTANT**: The system now syncs ALL reports (not just today's data) and includes cap compensation balance information.
---
## API Endpoint
-**URL**: `/api/reports/sync`
-**Method**: `POST`
-**Content-Type**: `application/json`
-**Authentication**: Bearer token (if configured)
-**Interval**: 10 minutes (configurable)
---
## Request Payload Structure
```json
{
"sync_id":"sync_20260201_214327_abc12345",
"client_id":"client_unique_identifier",
"sync_timestamp":"2026-02-01T21:43:27.249Z",
"date_range":"all",
"start_date":"2026-01-01T00:00:00",
"end_date":"2026-02-01T21:43:27.249Z",
"bets":[...],
"extraction_stats":[...],
"cap_compensation_balance":5000.0,
"summary":{...},
"is_incremental":true,
"sync_type":"incremental"
}
```
---
## Request Fields
### Metadata Fields
| Field | Type | Description |
|-------|------|-------------|
| `sync_id` | String | Unique identifier for this sync operation |
| `client_id` | String | Unique client identifier (machine ID or rustdesk_id) |
| `sync_timestamp` | ISO 8601 DateTime | When the sync was initiated |
| `date_range` | String | Date range for sync: "all", "today", "yesterday", "week" |
| `start_date` | ISO 8601 DateTime | Start of date range |
| `end_date` | ISO 8601 DateTime | End of date range |
| `is_incremental` | Boolean | True if this is an incremental sync (only new/changed data) |
| `sync_type` | String | "full" for first sync, "incremental" for subsequent syncs |
### Cap Compensation Balance
| Field | Type | Description |
|-------|------|-------------|
| `cap_compensation_balance` | Float | Accumulated shortfall from cap compensation system |
**Note**: This field represents the current balance of cap compensation adjustments. It's the `accumulated_shortfall` value from the `PersistentRedistributionAdjustmentModel` table, which tracks adjustments across all extractions.
3. Oldest completed items are removed when queue is full
4. Failed items are re-queued when server becomes available
---
## Implementation Notes for Server Developers
### 1. Handling Incremental Syncs
- Check `sync_type` field to determine if full or incremental
- For incremental syncs, only process new/updated records
- Use `sync_id` for tracking and deduplication
### 2. Cap Compensation Balance
- The `cap_compensation_balance` field represents the current accumulated shortfall
- This value should be stored and used for reconciliation
- It tracks adjustments across all extractions
### 3. Data Validation
- Validate all required fields are present
- Check UUIDs are unique
- Verify match IDs exist in your database
- Validate datetime formats
### 4. Error Handling
- Return appropriate HTTP status codes
- Provide clear error messages
- Log sync failures for debugging
### 5. Performance Considerations
- Process large payloads in batches if needed
- Use database transactions for data integrity
- Implement idempotent operations for retry safety
### 6. Security
- Validate Bearer token authentication
- Verify client_id matches expected clients
- Rate limit sync requests if necessary
---
## Example Full Request
```json
{
"sync_id":"sync_20260201_214327_abc12345",
"client_id":"machine_hostname_1234567890",
"sync_timestamp":"2026-02-01T21:43:27.249Z",
"date_range":"all",
"start_date":"2026-01-01T00:00:00",
"end_date":"2026-02-01T21:43:27.249Z",
"bets":[
{
"uuid":"bet-uuid-12345",
"fixture_id":"fixture-20260201",
"bet_datetime":"2026-02-01T10:30:00",
"paid":true,
"paid_out":false,
"total_amount":5000.0,
"bet_count":2,
"details":[
{
"match_id":123,
"match_number":1,
"outcome":"WIN1",
"amount":3000.0,
"win_amount":0.0,
"result":"pending"
},
{
"match_id":124,
"match_number":2,
"outcome":"DRAW",
"amount":2000.0,
"win_amount":0.0,
"result":"pending"
}
]
}
],
"extraction_stats":[
{
"match_id":123,
"fixture_id":"fixture-20260201",
"match_datetime":"2026-02-01T12:00:00",
"total_bets":50,
"total_amount_collected":100000.0,
"total_redistributed":95000.0,
"actual_result":"WIN1",
"extraction_result":"WIN1",
"cap_applied":true,
"cap_percentage":5.0,
"under_bets":20,
"under_amount":40000.0,
"over_bets":30,
"over_amount":60000.0,
"result_breakdown":{
"WIN1":{"bets":10,"amount":20000.0},
"DRAW":{"bets":5,"amount":10000.0},
"WIN2":{"bets":35,"amount":70000.0}
}
}
],
"cap_compensation_balance":5000.0,
"summary":{
"total_payin":100000.0,
"total_payout":95000.0,
"net_profit":5000.0,
"total_bets":50,
"total_matches":1
},
"is_incremental":true,
"sync_type":"incremental"
}
```
---
## Summary
The reports sync protocol provides:
✅ Complete data for report recreation
✅ Incremental sync (new/updated records only)
✅ Tracking of synced entities
✅ Retry mechanism for failed syncs
✅ Syncs ALL reports (not just today)
✅ Includes cap compensation balance
✅ Robust queue management
✅ Exponential backoff for retries
The server can use this data to recreate all reports, track cap compensation adjustments, and maintain accurate financial records across all historical data.
