- 08 Oct, 2025 40 commits
-
-
Stefy Lanza (nextime / spora ) authored
- Fixed process type mapping in queue manager ('analyze' -> 'analysis', 'train' -> 'training') - Implemented actual job sending in cluster master assign_job_to_worker() - Modified cluster client to forward jobs to local backend and monitor results - Added result polling mechanism for cluster jobs - Jobs should now execute on connected cluster workers instead of remaining queued The issue was that jobs were being assigned but never sent to workers. Now: 1. Queue manager selects worker using VRAM-aware logic 2. Cluster master assigns job and sends it via websocket 3. Cluster client receives job and forwards to local backend 4. Cluster client polls backend for results and sends back to master 5. Results are properly returned to web interface
-
Stefy Lanza (nextime / spora ) authored
- Updated queue manager to use select_worker_for_job() and assign_job_to_worker() instead of unimplemented assign_job_with_model() - Now properly implements VRAM-aware worker selection based on model requirements - Jobs will be assigned to distributed workers when available with sufficient VRAM - Falls back to local processing when no suitable distributed worker is found - Added proper error handling and logging for job assignment process
-
Stefy Lanza (nextime / spora ) authored
- Fixed 'list.append() takes exactly one argument (0 given)' error in update_queue_status - Removed empty params.append() calls for timestamp fields that use CURRENT_TIMESTAMP directly - Queue processing should now work correctly without errors
-
Stefy Lanza (nextime / spora ) authored
- Add delete_queue_item function to database.py with ownership validation - Add delete_job method to QueueManager class - Add /job/<id>/delete endpoint in web.py with user authentication - Update history.html template to show delete button for queued jobs - Only allow users to delete their own jobs or admins to delete any job - Add confirmation dialog for job deletion
-
Stefy Lanza (nextime / spora ) authored
- Make assign_job_with_model synchronous and return None to trigger local fallback - Remove asyncio from queue processing to avoid threading issues - Jobs now properly execute locally when no distributed workers are available - Maintain async file transfer infrastructure for future distributed worker support
-
Stefy Lanza (nextime / spora ) authored
- Convert assign_job_with_model and assign_job_to_worker to synchronous methods - Remove asyncio dependencies from queue processing - Simplify model transfer to avoid async websocket calls for now - Fix syntax errors in cluster_master.py
-
Stefy Lanza (nextime / spora ) authored
- Add VRAM requirement estimation for models - Implement intelligent worker selection based on VRAM, weight, and load - Add support for concurrent job execution with VRAM tracking - Integrate RunPod pod creation and management as fallback - Implement model loading/unloading logic to avoid redundant transfers - Add file transfer handling for shared storage and websocket fallback - Update queue system to use advanced cluster scheduling - Add job_id column to processing_queue table - Update web interface to submit jobs through queue system - Fix function name references in cluster master
-
Stefy Lanza (nextime / spora ) authored
Implement advanced job scheduling system with VRAM-aware worker selection, job queuing, RunPod integration, model management, and real-time notifications - Add VRAM requirement determination for models - Implement intelligent worker selection based on VRAM availability and weights - Add job queuing when no workers available with automatic retry - Integrate RunPod pod creation and management for scaling - Implement model loading/unloading with reference counting - Add file transfer support for remote workers (shared storage + websocket) - Enable concurrent job processing with VRAM tracking - Create job results page with detailed output display - Add real-time job completion notifications with result links - Update history page with live progress updates and result links - Fix async handling in cluster master websocket communication - Add database schema updates for job tracking
-
Stefy Lanza (nextime / spora ) authored
- When only 1 video-capable model exists, include hidden input field - Form still submits the correct model_path value - Maintains proper form functionality while hiding UI when not needed
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
- Model selection now only shows when there are 2+ video-capable models - When only 1 model is available, no model selection is shown (uses it automatically) - Removed the fallback text input that was showing inappropriately
-
Stefy Lanza (nextime / spora ) authored
- Show model dropdown only when multiple video-capable models are available - Filter models by 'video to text' capabilities and availability status - Admins see all models, regular users see only available ones - Fall back to text input when only one or no models are available
-
Stefy Lanza (nextime / spora ) authored
- Added detect_model_capabilities function that analyzes HF model tags and pipeline - Auto-detects capabilities for Hugging Face models when adding via admin or --model - Special handling for known models like Qwen VL, LLaVA, Whisper, etc. - Updated UI with hints about auto-detection
-
Stefy Lanza (nextime / spora ) authored
- Added capabilities TEXT column to models table - Updated admin interface to show and edit model capabilities - Default model has 'video to text, image to text' capabilities - Added Capabilities column to models table
-
Stefy Lanza (nextime / spora ) authored
- Added function to retrieve only available models for frontend/API use
-
Stefy Lanza (nextime / spora ) authored
- Added available BOOLEAN column to models table - Default model is marked as available - Models specified via --model are marked as available - Updated admin interface to show and edit availability status - Added Status column to models table showing Available/Hidden
-
Stefy Lanza (nextime / spora ) authored
- Added UNIQUE constraint on path column in models table - Added cleanup query to remove duplicate models, keeping only one per path
-
Stefy Lanza (nextime / spora ) authored
- Modified init_db to check for existing default model before inserting - Prevents multiple insertions of the same model
-
Stefy Lanza (nextime / spora ) authored
- Added ensure_model_exists function to database.py - Modified vidai.py to call ensure_model_exists when --model is specified - Automatically detects Hugging Face vs local models
-
Stefy Lanza (nextime / spora ) authored
- Added models table to database - Created admin routes for model CRUD operations - Added models.html template - Added models link to admin dropdown - Implemented Hugging Face model download functionality - Added download_huggingface_model function in utils.py
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
-
Stefy Lanza (nextime / spora ) authored
Fix timezone issue by explicitly treating SQLite timestamps as UTC before converting to Unix timestamp
-
Stefy Lanza (nextime / spora ) authored
Fix last seen timestamp parsing to handle both ISO and SQLite timestamp formats for proper timezone display
-