# Reports Synchronization Protocol Documentation

## Overview
The reports synchronization protocol sends betting and extraction statistics data from the client to the server. The system uses **incremental synchronization** - only new or updated records are sent after the initial full sync.

**IMPORTANT**: The system now syncs ALL reports (not just today's data) and includes cap compensation balance information.

---

## API Endpoint

- **URL**: `/api/reports/sync`
- **Method**: `POST`
- **Content-Type**: `application/json`
- **Authentication**: Bearer token (if configured)
- **Interval**: 10 minutes (configurable)

---

## Request Payload Structure

```json
{
  "sync_id": "sync_20260201_214327_abc12345",
  "client_id": "client_unique_identifier",
  "sync_timestamp": "2026-02-01T21:43:27.249Z",
  "date_range": "all",
  "start_date": "2026-01-01T00:00:00",
  "end_date": "2026-02-01T21:43:27.249Z",
  "bets": [...],
  "extraction_stats": [...],
  "cap_compensation_balance": 5000.0,
  "summary": {...},
  "is_incremental": true,
  "sync_type": "incremental"
}
```

---

## Request Fields

### Metadata Fields

| Field | Type | Description |
|-------|------|-------------|
| `sync_id` | String | Unique identifier for this sync operation |
| `client_id` | String | Unique client identifier (machine ID or rustdesk_id) |
| `sync_timestamp` | ISO 8601 DateTime | When the sync was initiated |
| `date_range` | String | Date range for sync: "all", "today", "yesterday", "week" |
| `start_date` | ISO 8601 DateTime | Start of date range |
| `end_date` | ISO 8601 DateTime | End of date range |
| `is_incremental` | Boolean | True if this is an incremental sync (only new/changed data) |
| `sync_type` | String | "full" for first sync, "incremental" for subsequent syncs |

### Cap Compensation Balance

| Field | Type | Description |
|-------|------|-------------|
| `cap_compensation_balance` | Float | Accumulated shortfall from cap compensation system |

**Note**: This field represents the current balance of cap compensation adjustments. It's the `accumulated_shortfall` value from the `PersistentRedistributionAdjustmentModel` table, which tracks adjustments across all extractions.

---

## Bet Data Structure

Each bet in the `bets` array contains:

```json
{
  "uuid": "bet-uuid-here",
  "fixture_id": "fixture-123",
  "bet_datetime": "2026-02-01T10:30:00",
  "paid": true,
  "paid_out": false,
  "total_amount": 5000.0,
  "bet_count": 3,
  "details": [
    {
      "match_id": 123,
      "match_number": "MATCH001",
      "outcome": "WIN1",
      "amount": 2000.0,
      "win_amount": 0.0,
      "result": "pending"
    }
  ]
}
```

### Bet Fields

| Field | Type | Description |
|-------|------|-------------|
| `uuid` | String | Unique bet identifier |
| `fixture_id` | String | Fixture identifier from matches table |
| `bet_datetime` | ISO 8601 DateTime | When the bet was placed |
| `paid` | Boolean | Whether payment was received |
| `paid_out` | Boolean | Whether winnings were paid out |
| `total_amount` | Float | Sum of all bet detail amounts |
| `bet_count` | Integer | Number of bet details |
| `details` | Array | Array of bet detail objects |

### Bet Detail Fields

| Field | Type | Description |
|-------|------|-------------|
| `match_id` | Integer | Match ID from matches table |
| `match_number` | Integer | Match number for display |
| `outcome` | String | Bet outcome/prediction (e.g., "WIN1", "DRAW", "X") |
| `amount` | Float | Bet amount |
| `win_amount` | Float | Winning amount (0.0 if not won) |
| `result` | String | Result status: "win", "lost", "pending", "cancelled" |

**Important**: Only bets with non-cancelled details are included in the sync.

---

## Extraction Stats Structure

Each stat in the `extraction_stats` array contains:

```json
{
  "match_id": 123,
  "fixture_id": "fixture-123",
  "match_datetime": "2026-02-01T12:00:00",
  "total_bets": 50,
  "total_amount_collected": 100000.0,
  "total_redistributed": 95000.0,
  "actual_result": "WIN1",
  "extraction_result": "WIN1",
  "cap_applied": true,
  "cap_percentage": 5.0,
  "under_bets": 20,
  "under_amount": 40000.0,
  "over_bets": 30,
  "over_amount": 60000.0,
  "result_breakdown": {
    "WIN1": {"bets": 10, "amount": 20000.0},
    "DRAW": {"bets": 5, "amount": 10000.0},
    "WIN2": {"bets": 35, "amount": 70000.0}
  }
}
```

### Extraction Stats Fields

| Field | Type | Description |
|-------|------|-------------|
| `match_id` | Integer | Match ID from matches table |
| `fixture_id` | String | Fixture identifier |
| `match_datetime` | ISO 8601 DateTime | When the match was completed |
| `total_bets` | Integer | Total number of bets on this match |
| `total_amount_collected` | Float | Total amount collected from all bets |
| `total_redistributed` | Float | Total amount redistributed to winners |
| `actual_result` | String | The actual match result |
| `extraction_result` | String | Result from extraction system (if different) |
| `cap_applied` | Boolean | Whether redistribution CAP was applied |
| `cap_percentage` | Float | CAP percentage used (if applied) |
| `under_bets` | Integer | Number of UNDER bets |
| `under_amount` | Float | Total amount bet on UNDER |
| `over_bets` | Integer | Number of OVER bets |
| `over_amount` | Float | Total amount bet on OVER |
| `result_breakdown` | JSON | Detailed breakdown by result option |

---

## Summary Structure

```json
{
  "total_payin": 100000.0,
  "total_payout": 95000.0,
  "net_profit": 5000.0,
  "total_bets": 50,
  "total_matches": 10
}
```

### Summary Fields

| Field | Type | Description |
|-------|------|-------------|
| `total_payin` | Float | Total amount collected from bets |
| `total_payout` | Float | Total amount redistributed |
| `net_profit` | Float | Net profit (payin - payout) |
| `total_bets` | Integer | Total number of bets |
| `total_matches` | Integer | Total number of matches |

---

## Incremental Synchronization Logic

The system uses `ReportsSyncTrackingModel` to track what has been synced:

### First Sync (Full Sync)
- No previous sync record exists
- All bets and extraction stats are sent
- `sync_type: "full"`
- `date_range: "all"` (sends all historical data)

### Subsequent Syncs (Incremental)
- Only records updated since `last_synced_at` are sent
- For each bet: checks if `bet.updated_at > tracking.last_synced_at`
- For each stat: checks if `stat.updated_at > tracking.last_synced_at`
- `sync_type: "incremental"`
- `date_range: "all"` (but only includes new/changed records)

### Tracking Records

The client maintains tracking records for:
- **Sync operations**: `entity_type='sync'`, `entity_id='latest'`
- **Individual bets**: `entity_type='bet'`, `entity_id=bet.uuid`
- **Extraction stats**: `entity_type='extraction_stat'`, `entity_id=match_id`

---

## Server Response Format

### Success Response

```json
{
  "success": true,
  "synced_count": 25,
  "message": "Successfully synced 25 items"
}
```

### Error Response

```json
{
  "success": false,
  "error": "Error message here"
}
```

---

## Data Completeness for Report Recreation

The protocol sends all necessary data to recreate the same reports on the server:

### Complete Information Includes:
1. **Bet Information**: Complete bet details including amounts, outcomes, results
2. **Match Information**: Match IDs and numbers linked to bet details
3. **Extraction Statistics**: Complete extraction data including caps, amounts, results
4. **Cap Compensation Balance**: Current accumulated shortfall for cap compensation
5. **Timestamps**: All datetime fields for accurate reporting
6. **Financial Data**: Payin, payout, redistribution amounts

### Server Can Use This Data To:
- Calculate daily/weekly/monthly summaries
- Generate match-by-match reports
- Track winning/losing bets
- Calculate profit/loss
- Apply the same extraction logic
- Track cap compensation adjustments
- Reconcile accumulated shortfall across all extractions

---

## Retry and Queue System

The client includes a robust retry mechanism:

### Queue Management
- **Queue Model**: `ReportsSyncQueueModel`
- **Max Queue Size**: 1000 items
- **Max Retries**: 5 attempts
- **Backoff Strategy**: Exponential backoff (60s * 2^retry_count)

### Queue Status
- `pending`: Waiting to be synced
- `syncing`: Currently being synced
- `completed`: Successfully synced
- `failed`: Failed after max retries

### Retry Logic
1. Failed syncs are queued for retry
2. Exponential backoff between retries
3. Oldest completed items are removed when queue is full
4. Failed items are re-queued when server becomes available

---

## Implementation Notes for Server Developers

### 1. Handling Incremental Syncs
- Check `sync_type` field to determine if full or incremental
- For incremental syncs, only process new/updated records
- Use `sync_id` for tracking and deduplication

### 2. Cap Compensation Balance
- The `cap_compensation_balance` field represents the current accumulated shortfall
- This value should be stored and used for reconciliation
- It tracks adjustments across all extractions

### 3. Data Validation
- Validate all required fields are present
- Check UUIDs are unique
- Verify match IDs exist in your database
- Validate datetime formats

### 4. Error Handling
- Return appropriate HTTP status codes
- Provide clear error messages
- Log sync failures for debugging

### 5. Performance Considerations
- Process large payloads in batches if needed
- Use database transactions for data integrity
- Implement idempotent operations for retry safety

### 6. Security
- Validate Bearer token authentication
- Verify client_id matches expected clients
- Rate limit sync requests if necessary

---

## Example Full Request

```json
{
  "sync_id": "sync_20260201_214327_abc12345",
  "client_id": "machine_hostname_1234567890",
  "sync_timestamp": "2026-02-01T21:43:27.249Z",
  "date_range": "all",
  "start_date": "2026-01-01T00:00:00",
  "end_date": "2026-02-01T21:43:27.249Z",
  "bets": [
    {
      "uuid": "bet-uuid-12345",
      "fixture_id": "fixture-20260201",
      "bet_datetime": "2026-02-01T10:30:00",
      "paid": true,
      "paid_out": false,
      "total_amount": 5000.0,
      "bet_count": 2,
      "details": [
        {
          "match_id": 123,
          "match_number": 1,
          "outcome": "WIN1",
          "amount": 3000.0,
          "win_amount": 0.0,
          "result": "pending"
        },
        {
          "match_id": 124,
          "match_number": 2,
          "outcome": "DRAW",
          "amount": 2000.0,
          "win_amount": 0.0,
          "result": "pending"
        }
      ]
    }
  ],
  "extraction_stats": [
    {
      "match_id": 123,
      "fixture_id": "fixture-20260201",
      "match_datetime": "2026-02-01T12:00:00",
      "total_bets": 50,
      "total_amount_collected": 100000.0,
      "total_redistributed": 95000.0,
      "actual_result": "WIN1",
      "extraction_result": "WIN1",
      "cap_applied": true,
      "cap_percentage": 5.0,
      "under_bets": 20,
      "under_amount": 40000.0,
      "over_bets": 30,
      "over_amount": 60000.0,
      "result_breakdown": {
        "WIN1": {"bets": 10, "amount": 20000.0},
        "DRAW": {"bets": 5, "amount": 10000.0},
        "WIN2": {"bets": 35, "amount": 70000.0}
      }
    }
  ],
  "cap_compensation_balance": 5000.0,
  "summary": {
    "total_payin": 100000.0,
    "total_payout": 95000.0,
    "net_profit": 5000.0,
    "total_bets": 50,
    "total_matches": 1
  },
  "is_incremental": true,
  "sync_type": "incremental"
}
```

---

## Summary

The reports sync protocol provides:

✅ Complete data for report recreation  
✅ Incremental sync (new/updated records only)  
✅ Tracking of synced entities  
✅ Retry mechanism for failed syncs  
✅ Syncs ALL reports (not just today)  
✅ Includes cap compensation balance  
✅ Robust queue management  
✅ Exponential backoff for retries  

The server can use this data to recreate all reports, track cap compensation adjustments, and maintain accurate financial records across all historical data.