- Implement BackgroundPing class that runs in separate thread
- Send ping messages every configurable interval during long operations
- Fix worker to use background ping during frame processing
- Ensure ping works even during blocking model inference operations
- Add proper cleanup of ping thread in finally block

- Fix job data display in jobs page to show user prompt from prompts array
- Add user prompt step to analyze and training job requests for display purposes
- Change default prompt from 'describe this image' to 'describe this media content'
- Complete multi-step prompt system for analyze and training workflows
- Fix socket communication for large messages exceeding 4096 bytes
- Update all components to handle new prompt composition logic
- Fix job data display in history page to show user prompt

All changes follow the AI.PROMPT rules for database migrations and API updates.

- Fix SocketCommunicator.receive_message() to handle messages larger than 4096 bytes
- Read complete JSON messages by accumulating data until newline delimiter
- Prevent JSON decode errors when responses exceed socket buffer size
- Complete implementation of multi-step prompts for analyze and training
- Training page now sends video and audio prompts with split_media always included
- Worker_training handles multi-step prompts with video processing and audio placeholder
- Analyze page passes analyze_audio option to worker_analyze

All changes follow the AI.PROMPT rules for database migrations and API updates.

- Add new prompts database table for system prompt management
- Create admin prompts page with analyze and training sections
- Add prompts management UI with separate areas for different functionalities
- Update analyze page with audio analysis and video splitting checkboxes
- Modify analyze API to handle new prompt composition logic
- Update worker_analyze to handle multi-step prompts (video + audio)
- Fix stats reload blinking in analyze page right panel by updating only changing sections
- Add prompts link to admin menu navigation

All changes follow the AI.PROMPT rules for database migrations and API updates.

- Add nvidia-smi as intermediate fallback before PyTorch in GPU stats collection
- Fallback order: pynvml -> nvidia-smi -> PyTorch
- Applied to api.py, backend.py, and cluster_client.py GPU stats functions
- nvidia-smi provides accurate memory usage and utilization data
- Fix SocketCommunicator.receive_message() timeout parameter error
- Added optional timeout parameter to receive_message method
- Fixes 'unexpected keyword argument timeout' error in api_stats and backend functions

- Modified history route in web.py to support pagination parameters
- Added pagination controls to history.html template
- Added JavaScript functions for page navigation and page size changes
- Pagination shows current page info and total job count
- Page size selector allows 10, 20, 50, or 100 jobs per page
- Navigation includes Previous/Next buttons and numbered page buttons

- Remove 'Progress shown below' from job submission success message
- Add progress bar display in analyze page sidebar (same as jobs page)
- Add spinner animation for processing jobs
- Auto-redirect to results page when job completes (2 second delay)
- Update JavaScript to fetch and display real-time progress updates
- Add CSS styles for progress bar and spinner animations

- Replaced all print statements in backend.py with timestamped log_message calls
- Includes worker registration, job forwarding, progress updates, ping handling, and startup/shutdown messages
- All backend logging now uses consistent timestamped format

- Corrected indentation of logging_utils import to be inside the try block
- Fixed malformed try/except structure that was causing SyntaxError

- Added **kwargs to log_message() function signature to accept arguments like 'flush=True'
- Maintains compatibility with original print() calls that may have had additional arguments

- Corrected malformed import statement that had logging_utils import inside database import parentheses
- Separated the imports properly to fix SyntaxError

- Modified jobs route to include completed jobs from last 24 hours
- Updated api/job_status_updates to include completed jobs so frontend gets notified when jobs complete
- Updated jobs page title and description to reflect it shows completed jobs
- Completed jobs already show 'View Result' link via existing JavaScript logic

This ensures users can see their recently completed jobs on the jobs page and easily access results.

- Created logging_utils.py with log_message() function that prefixes all output with date, time, and process name
- Replaced all print() statements with log_message() calls across the entire codebase
- Fixed circular import issues by separating logging from utils.py
- All console output now follows format: [YYYY-MM-DD HH:MM:SS] [process_name] message

This improves debugging and monitoring by providing clear timestamps and process identification for all log messages.

- Fixed backend get_progress handler to use queue ID instead of cluster job ID
- Removed duplicate get_progress handler that was causing confusion
- Progress is now correctly retrieved from the processing_queue table using the queue ID

This resolves the issue where job progress was always showing 'Processing...' instead of actual progress status and advancement bar.

- Add logic to set master weight to 0 when first client connects (unless explicitly set via command line)
- This ensures proper load balancing between local and remote workers

Fix job ID mismatch in progress updates - use full job_id instead of job_id_int for database lookups

Implement progress tracking for jobs: add progress updates in worker, backend handlers, and database updates

Fix CLI debug argument to default to False instead of reading from database, preventing unwanted debug inheritance

Fix debug reset - only consider CLI args and env vars, ignore config files to prevent unwanted debug persistence