This design covers four related deliverables in the existing reporting system:
1. Align daily reports with actual settled-bet accounting
2. Fix and verify Excel daily report download
3. Explain how remote zipped debug-log upload would be implemented
4. Automatically repair historical report/statistics data in the database where past values are inconsistent
## Goal
Make local daily reports match the intended accounting model used by settled bets and synced reporting, eliminate internal inconsistencies between on-screen daily reports and Excel exports, and provide a safe strategy for historical data correction.
## Current Findings
### Local daily summary
The daily summary endpoint in `mbetterclient/web_dashboard/routes.py` computes:
- payin from `BetDetailModel.amount` for non-cancelled bet details tied to bets placed in the selected time window
- payout from `BetDetailModel.win_amount` for winning settled bet details in the same time window
This is the correct accounting basis for a daily financial report.
### Excel daily summary
The Excel download route in `mbetterclient/web_dashboard/routes.py` computes:
- payin from bet details in the selected time window
- payout from `ExtractionStatsModel.total_redistributed` based on `match_datetime`
That means payin and payout come from different source populations and possibly different time boundaries. This can produce false losses or other inconsistencies.
### Match reports
The match reports route uses `is_bet_detail_winning(detail, match, session)` while referencing `match` before it is loaded in the loop that aggregates payouts. That can make payout classification unstable and may also contribute to Excel/report failures when similar logic is reused.
### Synced payload
The reports sync payload includes actual bet detail data and extraction statistics, plus `cap_compensation_balance`. This indicates the system already distinguishes between:
- actual payout/accounting
- redistribution balancing/telemetry
The local reporting path should preserve that separation.
## Recommended Approach
### Approach 1 - Recommended
Use settled bet details as the source of truth for financial reporting.
- Daily report UI summary uses settled bet details for payin and payout
- Excel daily summary uses the exact same payout basis as the daily summary API
- Match reports load the match before checking `is_bet_detail_winning`
- Historical `ExtractionStatsModel` financial fields are repaired from authoritative bet detail records
- Redistribution-balance tracking remains separate and is not mixed into daily profit/loss
Pros:
- Aligns local reports with actual money paid and received
- Makes UI and Excel consistent
- Safest for historical correction
Cons:
- Requires a historical repair routine or migration
### Approach 2
Fix only UI/API/Excel calculations and leave historical `ExtractionStatsModel` values untouched.
Pros:
- Smaller code change
Cons:
- Existing historical rows remain misleading
- Other views that rely on extraction stats can still diverge
### Approach 3
Introduce a shared reporting service abstraction for all report paths before fixing the bug.
Pros:
- Strong long-term structure
Cons:
- Larger refactor than necessary for this bugfix
## Chosen Design
Proceed with Approach 1.
## Implementation Design
### 1. Shared accounting basis for daily reports
Daily report financial totals should be defined as:
-`total_payin`: sum of non-cancelled `BetDetailModel.amount` for bet details whose parent `BetModel.bet_datetime` falls in the selected window
-`total_payout`: sum of `BetDetailModel.win_amount` for winning bet details in the same selected window
-`net_profit`: `total_payin - total_payout`
-`total_bets`: count of non-cancelled bet details in the selected window
-`total_matches`: number of unique `match_id` values represented by those bet details
This definition should be used consistently in:
-`/api/reports/daily-summary`
-`/api/reports/download-excel`
- any future local daily financial report export
### 2. Match report correction
The match reports aggregation path should load the `MatchModel` for each `match_id` before evaluating winning status for payout calculation. This ensures payout classification uses the same authoritative logic already used elsewhere in the dashboard.
### 3. Excel download verification and repair
The Excel route should be tested as a real file-producing endpoint.
Validation requirements:
- route returns HTTP 200 for valid requests
- content type is the XLSX MIME type
- returned file can be opened as a workbook
- summary sheet values match the daily summary API values for the same data window
- match sheet payout values match the corrected match report logic
### 4. Historical data repair
Automatic repair should target financial fields in `ExtractionStatsModel`.
For each match with existing extraction stats:
- recompute `total_bets` from non-cancelled bet details for that `match_id`
- recompute `total_amount_collected` as sum of non-cancelled detail amounts
- recompute `total_redistributed` as sum of `win_amount` for bet details with winning result
- preserve fields that reflect extraction outcome metadata unless separate evidence shows they are wrong:
-`actual_result`
-`extraction_result`
-`match_datetime`
-`result_breakdown` unless explicitly regenerated from repaired values
Preferred delivery mechanism:
- a one-time migration, following the existing migration pattern in `mbetterclient/database/migrations.py`
Safety behavior:
- if a match has multiple extraction stats rows, repair each row based on the current authoritative bet-detail state for that match and log duplicates for follow-up
- log how many rows changed and by how much
- do not automatically rewrite the global persistent redistribution adjustment in this pass unless the investigation proves it must also be backfilled
### 5. Remote zipped debug-log upload - explanation only
If implemented later, the feature should work like this:
- discover the active log directory from app settings / user data path
- copy current log files into a temp directory to avoid partial reads from active writers
- optionally redact secrets or generate a cleaned export set
- create a ZIP archive containing logs plus a small manifest JSON with version, platform, timestamp, and identifiers safe to transmit
- upload via HTTPS POST to a dedicated authenticated endpoint using either `multipart/form-data` or raw `application/zip`
- apply size limits, retries, and explicit opt-in or admin trigger
- keep this separate from normal report sync traffic
## Error Handling
- If Excel workbook generation fails, return a clear JSON error and log the exception with route context
- If the historical repair migration encounters malformed rows, skip and log them rather than aborting the entire migration where possible
- If match lookup fails during report generation, treat that match as unavailable for classification and log a warning; do not silently fabricate payout values
