Fix: Add requires_full_sync flag for empty database recovery

- Add requires_full_sync flag to /api/reports/sync response when no records exist for client across ALL report tables (bets, extraction_stats, match_reports)
- Add requires_full_sync flag to /api/reports/last-sync response when no records exist for client
- Update REPORTS_SYNC_PROTOCOL_DOCUMENTATION.md with full sync required scenario
- Update REPORTS_SYNC_API_SPECIFICATION.txt with requires_full_sync field documentation
- Create REPORTS_SYNC_EMPTY_DATABASE_FIX.md with comprehensive fix documentation

This fix addresses the critical gap where the server couldn't signal when it required a full sync after database reset, ensuring automatic recovery without manual intervention.

REPORTS_SYNC_API_SPECIFICATION.txt

@@ -296,6 +296,7 @@ Content-Type: application/json
   "success": true,
   "synced_count": 45,
   "message": "Report data synchronized successfully",
+  "requires_full_sync": false,
   "server_timestamp": "2026-02-01T08:26:20.123456"
 }
 
@@ -313,6 +314,13 @@ message (string, required)
   - Human-readable success message
   - Description of sync operation
 
+requires_full_sync (boolean, required)
+  - IMPORTANT: Indicates whether the server requires a full sync
+  - Set to true when the server has NO records for this client across ALL report tables (bets, extraction_stats, match_reports)
+  - Client should perform a full sync (send all historical data) when this is true
+  - This handles scenarios where the server database was reset but client has local tracking
+  - If ANY record exists for the client in ANY table, this flag will be false
+
 server_timestamp (string, required)
   - ISO 8601 format timestamp on server
   - When the server processed the sync

REPORTS_SYNC_EMPTY_DATABASE_FIX.md

+# Reports Sync Empty Database Recovery Fix
+
+## Problem Description
+
+The reports synchronization system had a critical gap in handling scenarios where the server database is empty (e.g., after a database reset) but the client has local tracking records indicating it has synced before.
+
+### Scenario
+
+1. **Client State**: Client has many existing matches in its local database and a local tracking record (`ReportsSyncTrackingModel`) indicating it has synced before
+2. **Server State**: Server database has been reset and is empty
+3. **Client Action**: Client correctly makes a "lazy sync" request before posting the reports sync request
+4. **Problem**: Client only sends an incremental request with no matches (because it thinks it has already synced)
+5. **Result**: Server accepts the incremental sync but never receives the historical data
+
+### Root Cause
+
+The sync protocol was designed with the assumption that:
+- **First Sync (Full Sync)**: No previous sync record exists → All bets and extraction stats are sent
+- **Subsequent Syncs (Incremental)**: Only records updated since `last_synced_at` are sent
+
+However, this logic was **client-side only**. The server had no mechanism to:
+1. Detect when its database is empty for a specific client
+2. Signal to the client that a full sync is required
+3. Override the client's incremental sync decision
+
+## Solution
+
+Added a `requires_full_sync` flag to the server's response that signals when the server database is empty and requires a full sync.
+
+### Implementation Changes
+
+#### 1. Server Response Enhancement (`/api/reports/sync`)
+
+**File**: `app/api/routes.py`
+
+**Change**: Added `requires_full_sync` flag to the sync response
+
+```python
+# Check if ANY records exist for the client across ALL report tables
+total_bets_for_client = Bet.query.filter_by(client_id=data['client_id']).count()
+total_stats_for_client = ExtractionStats.query.filter_by(client_id=data['client_id']).count()
+total_match_reports_for_client = MatchReport.query.filter_by(client_id=data['client_id']).count()
+
+# If no records exist in any table, this is effectively a first sync and requires full sync
+requires_full_sync = (total_bets_for_client == 0 and total_stats_for_client == 0 and total_match_reports_for_client == 0)
+
+return jsonify({
+    'success': True,
+    'synced_count': bets_count + stats_count,
+    'message': 'Report data synchronized successfully',
+    'requires_full_sync': requires_full_sync,  # NEW FLAG
+    'server_timestamp': datetime.utcnow().isoformat()
+}), 200
+```
+
+**Logic**:
+- Check if ANY records exist for the client across ALL report tables (bets, extraction_stats, match_reports)
+- If NO records exist in any table (`total_bets_for_client == 0 and total_stats_for_client == 0 and total_match_reports_for_client == 0`), set `requires_full_sync: true`
+- This indicates the server database was empty for this client and the client should perform a full sync
+
+#### 2. Last Sync Endpoint Enhancement (`/api/reports/last-sync`)
+
+**File**: `app/api/routes.py`
+
+**Change**: Added `requires_full_sync` flag when no sync records exist
+
+```python
+if not last_sync:
+    # Check if ANY records exist for this client across all report tables
+    total_bets_for_client = Bet.query.filter_by(client_id=client_id).count()
+    total_stats_for_client = ExtractionStats.query.filter_by(client_id=client_id).count()
+    total_match_reports_for_client = MatchReport.query.filter_by(client_id=client_id).count()
+    
+    # If no records exist in any table, signal that full sync is required
+    requires_full_sync = (total_bets_for_client == 0 and total_stats_for_client == 0 and total_match_reports_for_client == 0)
+    
+    return jsonify({
+        'success': True,
+        'message': 'No sync records found for this client',
+        'client_id': client_id,
+        'last_sync_id': None,
+        'last_sync_timestamp': None,
+        'last_sync_type': None,
+        'total_syncs': 0,
+        'requires_full_sync': requires_full_sync,  # NEW FLAG
+        'server_timestamp': datetime.utcnow().isoformat()
+    }), 200
+```
+
+**Logic**:
+- When querying the last sync and no sync records are found, check if ANY records exist for the client across ALL report tables (bets, extraction_stats, match_reports)
+- If NO records exist in any table, set `requires_full_sync: true`
+- This allows the client to proactively check if a full sync is needed before sending data
+
+### Client-Side Implementation (Required)
+
+The client must be updated to handle the `requires_full_sync` flag:
+
+```python
+# After receiving sync response
+if response.get('requires_full_sync'):
+    logger.info("Server requires full sync - performing full sync")
+    # Perform full sync with all historical data
+    full_sync_data = prepare_full_sync_data()
+    response = send_sync_request(full_sync_data)
+```
+
+Or check before syncing:
+
+```python
+# Check if server requires full sync
+last_sync_response = get_last_sync_info()
+if last_sync_response.get('requires_full_sync'):
+    logger.info("Server database is empty - performing full sync")
+    # Perform full sync with all historical data
+    full_sync_data = prepare_full_sync_data()
+    response = send_sync_request(full_sync_data)
+else:
+    # Perform incremental sync
+    incremental_data = prepare_incremental_sync_data()
+    response = send_sync_request(incremental_data)
+```
+
+## Updated Documentation
+
+### 1. Protocol Documentation (`REPORTS_SYNC_PROTOCOL_DOCUMENTATION.md`)
+
+Added:
+- Response fields table with `requires_full_sync` field
+- "Full Sync Required Scenario" section explaining the behavior
+- Example scenario showing how the fix works
+
+### 2. API Specification (`REPORTS_SYNC_API_SPECIFICATION.txt`)
+
+Added:
+- `requires_full_sync` field to success response format
+- Detailed description of when and why this flag is set
+
+## Testing Recommendations
+
+### Test Case 1: Empty Server Database
+
+1. Reset server database for a specific client (delete all bets, extraction_stats, match_reports for that client)
+2. Client has local tracking showing it synced 100 records yesterday
+3. Client sends incremental sync with 0 new records
+4. Server checks all report tables and finds 0 records for this client
+5. Server responds with `requires_full_sync: true`
+6. Client performs full sync and sends all 100 records
+7. Server now has complete data
+
+### Test Case 2: Normal Incremental Sync
+
+1. Server has existing data from previous syncs
+2. Client sends incremental sync with 5 new records
+3. Server responds with `requires_full_sync: false`
+4. Client continues with normal incremental sync behavior
+
+### Test Case 3: Last Sync Query
+
+1. Reset server database for a specific client (delete all bets, extraction_stats, match_reports for that client)
+2. Client queries `/api/reports/last-sync?client_id=xxx`
+3. Server checks all report tables and finds 0 records for this client
+4. Server responds with `requires_full_sync: true`
+5. Client performs full sync
+
+## Benefits
+
+1. **Automatic Recovery**: Server can signal when it needs a full sync without manual intervention
+2. **Data Integrity**: Ensures server receives complete data even after database resets by checking across ALL report tables
+3. **Backward Compatible**: Existing clients that don't check the flag will still work (they just won't benefit from the optimization)
+4. **Clear Protocol**: Explicit signal in the API response makes the behavior clear and documented
+
+## Migration Notes
+
+### For Existing Deployments
+
+1. **Server Update**: Deploy the updated server code with the `requires_full_sync` flag
+2. **Client Update**: Update client code to check and handle the `requires_full_sync` flag
+3. **Testing**: Test the scenario where server database is reset but client has local tracking
+
+### For New Deployments
+
+1. Implement both server and client with the `requires_full_sync` flag support
+2. Follow the updated protocol documentation
+3. Include test cases for empty database recovery
+
+## Summary
+
+The fix addresses a critical gap in the sync protocol by allowing the server to signal when it requires a full sync. This ensures data integrity in scenarios where the server database is empty for a specific client but the client has local tracking records, such as after a database reset or disaster recovery.
+
+The `requires_full_sync` flag provides a clear, explicit mechanism for the server to communicate its state to the client, enabling automatic recovery without manual intervention. The server checks across ALL report tables (bets, extraction_stats, match_reports) to determine if a full sync is required, ensuring accurate detection of empty database state.
\ No newline at end of file

REPORTS_SYNC_PROTOCOL_DOCUMENTATION.md

@@ -225,10 +225,46 @@ The client maintains tracking records for:
 {
   "success": true,
   "synced_count": 25,
-  "message": "Successfully synced 25 items"
+  "message": "Successfully synced 25 items",
+  "requires_full_sync": false
 }
 ```
 
+### Response Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `success` | Boolean | Whether the sync operation succeeded |
+| `synced_count` | Integer | Number of items successfully synced |
+| `message` | String | Human-readable success message |
+| `requires_full_sync` | Boolean | **IMPORTANT**: If true, client should perform a full sync (send all historical data). This is set to true when the server database is empty for this client. |
+| `server_timestamp` | ISO 8601 DateTime | When the server processed the sync |
+
+### Full Sync Required Scenario
+
+When the server database has NO records for a specific client across ALL report tables (bets, extraction_stats, match_reports), the server will set `requires_full_sync: true` in the response. This happens when:
+
+- The server database was reset and is now empty for this client
+- This is the first time this client has ever synced with the server
+- All previous records for this client were deleted
+
+The client should:
+
+1. Check the `requires_full_sync` flag in the response
+2. If true, perform a full sync with `sync_type: "full"` and send all historical data
+3. This ensures the server receives all records even if the client's local tracking indicates it has synced before
+
+**Example scenario:**
+- Client has local tracking showing it synced 100 records yesterday
+- Server database was reset and now has 0 records for this client (no bets, no extraction_stats, no match_reports)
+- Client sends incremental sync with 0 new records
+- Server checks all report tables and finds no records for this client
+- Server responds with `requires_full_sync: true`
+- Client performs full sync and sends all 100 records
+- Server now has complete data
+
+**Important**: The server checks across ALL report tables (bets, extraction_stats, match_reports) to determine if a full sync is required. If ANY record exists for the client in ANY table, `requires_full_sync` will be false.
+
 ### Error Response
 
 ```json

app/api/routes.py

@@ -1405,10 +1405,20 @@ def api_reports_sync():
         
         logger.info(f"Report sync {data['sync_id']} completed successfully from client {data['client_id']}: {bets_count} bets, {stats_count} stats ({stats_new} new, {stats_updated} updated)")
         
+        # Check if this was the first sync for this client (server database was empty)
+        # Check if ANY records exist for this client across all report tables
+        total_bets_for_client = Bet.query.filter_by(client_id=data['client_id']).count()
+        total_stats_for_client = ExtractionStats.query.filter_by(client_id=data['client_id']).count()
+        total_match_reports_for_client = MatchReport.query.filter_by(client_id=data['client_id']).count()
+        
+        # If no records exist in any table, this is effectively a first sync and requires full sync
+        requires_full_sync = (total_bets_for_client == 0 and total_stats_for_client == 0 and total_match_reports_for_client == 0)
+        
         return jsonify({
             'success': True,
             'synced_count': bets_count + stats_count,
             'message': 'Report data synchronized successfully',
+            'requires_full_sync': requires_full_sync,
             'server_timestamp': datetime.utcnow().isoformat()
         }), 200
         
@@ -1483,6 +1493,14 @@ def api_get_last_sync():
             .first()
         
         if not last_sync:
+            # Check if ANY records exist for this client across all report tables
+            total_bets_for_client = Bet.query.filter_by(client_id=client_id).count()
+            total_stats_for_client = ExtractionStats.query.filter_by(client_id=client_id).count()
+            total_match_reports_for_client = MatchReport.query.filter_by(client_id=client_id).count()
+            
+            # If no records exist in any table, signal that full sync is required
+            requires_full_sync = (total_bets_for_client == 0 and total_stats_for_client == 0 and total_match_reports_for_client == 0)
+            
             return jsonify({
                 'success': True,
                 'message': 'No sync records found for this client',
@@ -1491,6 +1509,7 @@ def api_get_last_sync():
                 'last_sync_timestamp': None,
                 'last_sync_type': None,
                 'total_syncs': 0,
+                'requires_full_sync': requires_full_sync,  # Signal to client that full sync is needed
                 'server_timestamp': datetime.utcnow().isoformat()
             }), 200