Add cross-platform persistent directories and authenticated ZIP downloads (v1.2.1)

## Cross-Platform PyInstaller Support
- Implemented persistent directory system for Windows (%APPDATA%), macOS (~/Library/Application Support), and Linux (/opt/MBetter)
- Added PyInstaller environment detection and automatic directory path optimization
- Created comprehensive directory utilities (app/utils/directories.py) with fallback mechanisms
- Enhanced configuration system to use persistent mbetterd.conf instead of .env for PyInstaller deployments

## Configuration System Enhancement
- Added configuration file migration system (migrate_config.py) for .env to mbetterd.conf transition
- Enhanced config.py with persistent config file loading and priority system (env vars > config file > defaults)
- Automatic config file discovery in persistent directories with fallback to local .env

## Authenticated ZIP Downloads
- Added new /api/download/zip/<match_id> endpoint with hybrid JWT/API token authentication
- Enhanced /api/updates endpoint to use authenticated download URLs instead of web routes
- Implemented access control ensuring users can only download their own match files (or admin access)
- Updated API routes to use persistent ZIP_UPLOADS_DIR for file serving

## Upload System Enhancement
- Modified file upload handlers to use persistent directories (ZIP_UPLOADS_DIR, FIXTURES_DIR, TEMP_DIR)
- Enhanced upload file paths to use appropriate persistent directories based on file type
- Updated main routes for ZIP downloads to use persistent directory paths
- Fixed PyInstaller temporary directory issues ensuring files persist between application restarts

## Documentation Updates
- Updated README.md with PyInstaller features, cross-platform compatibility, and authenticated downloads
- Enhanced CHANGELOG.md with comprehensive v1.2.1 release notes
- Added migration utility documentation and deployment instructions
- Updated version numbers and platform compatibility information

## Technical Improvements
- Enhanced error handling and logging throughout persistent directory operations
- Added comprehensive fallback mechanisms for various deployment scenarios
- Improved path handling with proper cross-platform directory structures
- Security enhancements for API file downloads with proper authentication validation

CHANGELOG.md

@@ -5,6 +5,59 @@ All notable changes to the Fixture Manager daemon project will be documented in
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.1] - 2025-08-21
+
+### Added
+- **Cross-platform persistent directory system** for PyInstaller compatibility
+  - Windows support using `%APPDATA%/MBetter` directory structure
+  - macOS support using `~/Library/Application Support/MBetter` directory
+  - Linux support with primary `/opt/MBetter` and fallback `~/.local/share/MBetter`
+  - Automatic PyInstaller environment detection and optimization
+- **Configuration file migration system**
+  - Automatic `.env` to `mbetterd.conf` migration for PyInstaller deployments
+  - `migrate_config.py` utility script for production environment transitions
+  - Persistent configuration storage in platform-appropriate locations
+  - Fallback support for legacy `.env` files during development
+- **Authenticated ZIP download API endpoint**
+  - New `/api/download/zip/<match_id>` endpoint with hybrid authentication
+  - Supports both JWT and API token authentication with automatic fallback
+  - Access control ensuring users can only download their own match files (or admin access)
+  - Secure file serving from persistent ZIP upload directories
+
+### Enhanced
+- **Upload directory persistence** - All uploads now stored outside PyInstaller temp directories
+  - ZIP files stored in persistent ZIP_UPLOADS_DIR
+  - Fixture files stored in persistent FIXTURES_DIR
+  - Temporary files managed in persistent TEMP_DIR
+- **Configuration system improvements**
+  - Enhanced config loading with priority: environment variables > config file > defaults
+  - Automatic configuration file discovery in persistent directories
+  - Comprehensive error handling with fallback mechanisms
+- **API updates endpoint improvements**
+  - ZIP download URLs now use authenticated API endpoint instead of web routes
+  - Enhanced security for programmatic access to ZIP files
+  - Consistent authentication across all API endpoints
+
+### Fixed
+- **PyInstaller temporary directory issues** - Files now persist between application restarts
+- **Cross-platform path handling** - Proper directory structures for all supported platforms
+- **Configuration loading reliability** - Robust fallback mechanisms for various deployment scenarios
+
+### Technical Details
+- **Directory Structure**: Platform-specific persistent directories with proper fallbacks
+- **Config Migration**: Seamless transition from development .env to production mbetterd.conf
+- **PyInstaller Detection**: Automatic environment detection and directory path adjustment
+- **Authentication**: Unified authentication system across web and API endpoints
+- **File Storage**: Persistent upload storage with cross-platform compatibility
+
+### Security
+- **Enhanced API authentication** for file downloads
+- **Access control validation** for ZIP file access
+- **Secure configuration file handling** with proper permissions
+- **Path traversal protection** in persistent directory operations
+
+---
+
 ## [1.2.0] - 2025-08-21
 
 ### Added

README.md

@@ -85,12 +85,13 @@ sudo ./install.sh
    ```bash
    cp .env.example .env
    # Edit .env with your configuration
+   # Note: For PyInstaller deployments, configuration will migrate to mbetterd.conf automatically
    ```
 
 ## Configuration
 
-### Environment Variables
-The system uses environment variables for configuration. Key settings include:
+### Configuration File (mbetterd.conf)
+The system automatically migrates from `.env` to `mbetterd.conf` stored in persistent directories for PyInstaller compatibility. Configuration settings include:
 
 ```bash
 # Database Configuration
@@ -275,10 +276,18 @@ curl -X POST "http://your-server/api/updates" \
 - **Incremental Updates**: Use `from` parameter for efficient data synchronization
 - **Flexible Methods**: Supports both GET (query params) and POST (JSON body)
 - **Configurable Limits**: Respects system setting for maximum fixtures returned
-- **ZIP Downloads**: Includes direct download URLs for completed match files
+- **Authenticated ZIP Downloads**: Secure direct download URLs with token authentication
 - **Hybrid Authentication**: Works with both JWT and API tokens automatically
 - **Smart Fallback**: Gracefully handles existing data without active timestamps
 
+#### Download ZIP Files (Authenticated)
+```bash
+# Download ZIP file for specific match (requires authentication)
+curl -X GET "http://your-server/api/download/zip/123" \
+  -H "Authorization: Bearer YOUR_API_TOKEN" \
+  -o "match_123.zip"
+```
+
 ## File Format Requirements
 
 ### Fixture Files (CSV/XLSX)
@@ -544,7 +553,7 @@ curl -X DELETE http://your-server/profile/tokens/123/delete \
 
 ## Building Single Executable
 
-The project can be packaged as a single executable file for easy distribution:
+The project can be packaged as a single executable file for easy distribution with **cross-platform persistent directories**:
 
 ### Quick Build
 ```bash
@@ -567,9 +576,18 @@ The executable will be created in the `dist/` directory and includes:
 - Database utilities and models
 - Web dashboard and API
 - Configuration templates
+- **Cross-platform persistent directory support**
 
 **Executable Size**: ~80-120MB
 **No Python Installation Required** on target systems
+**Cross-Platform Compatibility**: Windows, macOS, and Linux
+
+### PyInstaller Features
+- **Persistent Data Storage**: Files persist between application restarts
+- **Cross-Platform Directories**: Uses OS-appropriate locations (AppData, Library, /opt)
+- **Configuration Migration**: Automatic .env to mbetterd.conf migration
+- **Upload Directory Persistence**: ZIP files and fixtures stored outside temp directories
+- **Platform Detection**: Automatic PyInstaller environment detection
 
 See [BUILD.md](BUILD.md) for detailed build instructions and troubleshooting.
 
@@ -677,11 +695,20 @@ curl -H "Authorization: Bearer $API_TOKEN" \
 
 ---
 
-**Version**: 1.2.0
+**Version**: 1.2.1
 **Last Updated**: 2025-08-21
-**Minimum Requirements**: Python 3.8+, MySQL 5.7+, Linux Kernel 3.10+
-
-### Recent Updates (v1.2.0)
+**Minimum Requirements**: Python 3.8+, MySQL 5.7+, Linux/Windows/macOS
+
+### Recent Updates (v1.2.1) - PyInstaller Enhancement
+- ✅ **Cross-Platform Persistent Directories**: Windows (%APPDATA%), macOS (~/Library/Application Support), Linux (/opt/MBetter)
+- ✅ **Configuration Migration**: Automatic .env to mbetterd.conf migration for PyInstaller deployments
+- ✅ **Authenticated ZIP Downloads**: Secure API endpoint for ZIP file downloads with token authentication
+- ✅ **PyInstaller Detection**: Automatic detection and optimization for PyInstaller environments
+- ✅ **Persistent Upload Storage**: Uploads stored outside PyInstaller temp directories
+- ✅ **Migration Utility**: migrate_config.py script for environment transition
+- ✅ **Platform-Specific Paths**: OS-appropriate directory structures for all platforms
+
+### Updates (v1.2.0) - API Enhancement
 - ✅ **New `/api/updates` Endpoint**: Incremental fixture synchronization with timestamp-based filtering
 - ✅ **Hybrid Authentication**: JWT and API token support with automatic fallback
 - ✅ **Fixture Active Time Tracking**: Automatic timestamp management for fixture activation

app/api/routes.py

@@ -675,11 +675,11 @@ def api_get_updates():
                     from flask import url_for, current_app
                     import os
                     
-                    # Construct download link
-                    # Assuming zip files are stored in uploads directory
-                    zip_path = os.path.join(current_app.config.get('UPLOAD_FOLDER', 'uploads'), match.zip_filename)
+                    # Construct download link using authenticated API endpoint
+                    # Use the persistent ZIP uploads directory
+                    zip_path = os.path.join(current_app.config.get('ZIP_UPLOADS_DIR', current_app.config.get('UPLOAD_FOLDER', 'uploads')), match.zip_filename)
                     if os.path.exists(zip_path):
-                        match_dict['zip_download_url'] = url_for('main.download_zip',
+                        match_dict['zip_download_url'] = url_for('api.api_download_zip',
                                                                match_id=match.id,
                                                                _external=True)
                     else:
@@ -709,4 +709,76 @@ def api_get_updates():
         
     except Exception as e:
         logger.error(f"API updates error: {str(e)}")
-        return jsonify({'error': 'Failed to retrieve updates'}), 500
\ No newline at end of file
+        return jsonify({'error': 'Failed to retrieve updates'}), 500
+
+@bp.route('/download/zip/<int:match_id>', methods=['GET'])
+def api_download_zip(match_id):
+    """Download ZIP file for specific match - supports both JWT and API tokens"""
+    try:
+        from app.models import Match, User
+        from flask_jwt_extended import jwt_required, get_jwt_identity, verify_jwt_in_request
+        from app.auth.jwt_utils import validate_api_token, extract_token_from_request
+        from flask import send_file
+        import os
+        
+        user = None
+        auth_method = None
+        
+        # Try JWT authentication first (short-lived session tokens)
+        try:
+            verify_jwt_in_request()
+            user_id = get_jwt_identity()
+            user = User.query.get(user_id)
+            auth_method = "JWT"
+            logger.info(f"ZIP download via JWT token by user {user.username if user else 'unknown'}")
+        except Exception as jwt_error:
+            logger.debug(f"JWT authentication failed: {str(jwt_error)}")
+            
+            # If JWT fails, try API token authentication (long-lived tokens)
+            try:
+                token = extract_token_from_request()
+                if not token:
+                    return jsonify({
+                        'error': 'Authentication required',
+                        'message': 'Either JWT token or API token required'
+                    }), 401
+                
+                user, api_token = validate_api_token(token)
+                auth_method = f"API Token ({api_token.name if api_token else 'JWT'})"
+                logger.info(f"ZIP download via API token by user {user.username}")
+            except Exception as api_error:
+                logger.debug(f"API token authentication failed: {str(api_error)}")
+                return jsonify({
+                    'error': 'Authentication failed',
+                    'message': 'Invalid JWT token or API token'
+                }), 401
+        
+        if not user or not user.is_active:
+            return jsonify({'error': 'User not found or inactive'}), 404
+        
+        # Get match and verify access
+        match = Match.query.get(match_id)
+        if not match:
+            return jsonify({'error': 'Match not found'}), 404
+        
+        # Check if user has access to this match (admin or creator)
+        if not user.is_admin and match.created_by != user.id:
+            return jsonify({'error': 'Access denied - you can only download ZIP files for your own matches'}), 403
+        
+        # Check if ZIP file exists
+        if not match.zip_filename or match.zip_upload_status != 'completed':
+            return jsonify({'error': 'ZIP file not available for this match'}), 404
+        
+        # Construct file path using persistent ZIP directory
+        zip_path = os.path.join(current_app.config.get('ZIP_UPLOADS_DIR', current_app.config.get('UPLOAD_FOLDER', 'uploads')), match.zip_filename)
+        
+        if not os.path.exists(zip_path):
+            return jsonify({'error': 'ZIP file not found on disk'}), 404
+        
+        logger.info(f"ZIP file download successful via {auth_method} by user {user.username} for match {match_id}")
+        
+        return send_file(zip_path, as_attachment=True, download_name=match.zip_filename)
+        
+    except Exception as e:
+        logger.error(f"API ZIP download error: {str(e)}")
+        return jsonify({'error': 'ZIP download failed'}), 500
\ No newline at end of file

app/main/routes.py

@@ -1363,7 +1363,7 @@ def download_zip(match_id):
             abort(404)
         
         # Construct file path
-        zip_path = os.path.join(current_app.config.get('UPLOAD_FOLDER', 'uploads'), match.zip_filename)
+        zip_path = os.path.join(current_app.config.get('ZIP_UPLOADS_DIR', current_app.config.get('UPLOAD_FOLDER', 'uploads')), match.zip_filename)
         
         if not os.path.exists(zip_path):
             flash('ZIP file not found on disk', 'error')

app/upload/file_handler.py

@@ -29,7 +29,11 @@ class FileUploadHandler:
         """Ensure handler is initialized with Flask app config"""
         if not self._initialized:
             from flask import current_app
-            self.upload_folder = current_app.config.get('UPLOAD_FOLDER')
+            # Use the persistent ZIP uploads directory for all uploads
+            self.upload_folder = current_app.config.get('ZIP_UPLOADS_DIR')
+            if self.upload_folder is None:
+                # Fallback to legacy UPLOAD_FOLDER if ZIP_UPLOADS_DIR not set
+                self.upload_folder = current_app.config.get('UPLOAD_FOLDER')
             self.chunk_size = current_app.config.get('CHUNK_SIZE', 8192)
             self.max_concurrent_uploads = current_app.config.get('MAX_CONCURRENT_UPLOADS', 5)
             if self.executor is None:
@@ -219,21 +223,32 @@ class FileUploadHandler:
                 # Extract file extension from original filename
                 _, ext = os.path.splitext(sanitized_name)
                 filename = sha1_hash + (ext or '.zip')
+                
+                # Use ZIP uploads directory
+                from flask import current_app
+                upload_dir = current_app.config.get('ZIP_UPLOADS_DIR', self.upload_folder)
             else:
                 # For other files, use timestamp-based naming
                 timestamp = datetime.utcnow().strftime('%Y%m%d_%H%M%S')
                 filename = "{}_{}".format(timestamp, sanitized_name)
+                
+                # Use general uploads directory or fixtures directory
+                from flask import current_app
+                if file_type == 'fixture':
+                    upload_dir = current_app.config.get('FIXTURES_DIR', self.upload_folder)
+                else:
+                    upload_dir = self.upload_folder
             
             # Create upload directory if it doesn't exist
-            if self.upload_folder is None:
-                logger.error("Upload folder is None - cannot create directory")
-                return None, "Upload folder not configured"
+            if upload_dir is None:
+                logger.error("Upload directory is None - cannot create directory")
+                return None, "Upload directory not configured"
             
             try:
-                os.makedirs(self.upload_folder, exist_ok=True)
-                file_path = os.path.join(self.upload_folder, filename)
+                os.makedirs(upload_dir, exist_ok=True)
+                file_path = os.path.join(upload_dir, filename)
             except OSError as e:
-                logger.error(f"Failed to create upload directory {self.upload_folder}: {str(e)}")
+                logger.error(f"Failed to create upload directory {upload_dir}: {str(e)}")
                 return None, f"Failed to create upload directory: {str(e)}"
             
             # Get file size and MIME type

app/utils/directories.py

+"""
+Cross-platform persistent directory utilities for PyInstaller compatibility.
+
+This module provides functions to create and manage persistent directories
+that survive PyInstaller temporary directory cleanup.
+"""
+import os
+import sys
+import platform
+from pathlib import Path
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+def get_platform_persistent_dir(app_name="MBetter"):
+    """
+    Get the appropriate persistent directory for the current platform.
+    
+    Args:
+        app_name (str): Application name for directory naming
+        
+    Returns:
+        Path: Platform-appropriate persistent directory path
+    """
+    system = platform.system().lower()
+    
+    if system == "windows":
+        # Windows: Use APPDATA\MBetter
+        base_dir = os.environ.get('APPDATA')
+        if not base_dir:
+            # Fallback to user profile
+            base_dir = os.path.expanduser('~')
+            return Path(base_dir) / app_name
+        return Path(base_dir) / app_name
+        
+    elif system == "darwin":  # macOS
+        # macOS: Use ~/Library/Application Support/MBetter
+        return Path.home() / "Library" / "Application Support" / app_name
+        
+    else:  # Linux and other Unix-like systems
+        # Linux: Check if we have write permission to /opt, otherwise use user directory
+        opt_path = Path("/opt") / app_name
+        try:
+            # Test if we can write to /opt/MBetter
+            opt_path.mkdir(parents=True, exist_ok=True)
+            test_file = opt_path / ".write_test"
+            test_file.touch()
+            test_file.unlink()
+            return opt_path
+        except (PermissionError, OSError):
+            # Fallback to user directory
+            return Path.home() / ".local" / "share" / app_name
+
+
+def get_uploads_directory(custom_path=None):
+    """
+    Get the uploads directory, with optional custom path override.
+    
+    Args:
+        custom_path (str, optional): Custom path override
+        
+    Returns:
+        Path: Uploads directory path
+    """
+    if custom_path:
+        return Path(custom_path)
+    
+    # Check environment variable first
+    env_path = os.environ.get('MBETTER_UPLOADS_DIR')
+    if env_path:
+        return Path(env_path)
+    
+    # Use platform-appropriate persistent directory
+    base_dir = get_platform_persistent_dir()
+    return base_dir / "uploads"
+
+
+def get_logs_directory(custom_path=None):
+    """
+    Get the logs directory, with optional custom path override.
+    
+    Args:
+        custom_path (str, optional): Custom path override
+        
+    Returns:
+        Path: Logs directory path
+    """
+    if custom_path:
+        return Path(custom_path)
+    
+    # Check environment variable first
+    env_path = os.environ.get('MBETTER_LOGS_DIR')
+    if env_path:
+        return Path(env_path)
+    
+    # Use platform-appropriate persistent directory
+    base_dir = get_platform_persistent_dir()
+    return base_dir / "logs"
+
+
+def get_data_directory(custom_path=None):
+    """
+    Get the data directory for database and other persistent data.
+    
+    Args:
+        custom_path (str, optional): Custom path override
+        
+    Returns:
+        Path: Data directory path
+    """
+    if custom_path:
+        return Path(custom_path)
+    
+    # Check environment variable first
+    env_path = os.environ.get('MBETTER_DATA_DIR')
+    if env_path:
+        return Path(env_path)
+    
+    # Use platform-appropriate persistent directory
+    base_dir = get_platform_persistent_dir()
+    return base_dir / "data"
+
+
+def ensure_directory_exists(directory_path, description="directory"):
+    """
+    Ensure a directory exists, creating it if necessary.
+    
+    Args:
+        directory_path (Path or str): Directory path to create
+        description (str): Description for logging purposes
+        
+    Returns:
+        Path: The created/existing directory path
+        
+    Raises:
+        OSError: If directory cannot be created
+    """
+    path = Path(directory_path)
+    
+    try:
+        path.mkdir(parents=True, exist_ok=True)
+        logger.info(f"Ensured {description} exists: {path}")
+        return path
+    except OSError as e:
+        logger.error(f"Failed to create {description} at {path}: {e}")
+        raise OSError(f"Cannot create {description} at {path}: {e}")
+
+
+def is_pyinstaller():
+    """
+    Check if the application is running from a PyInstaller bundle.
+    
+    Returns:
+        bool: True if running from PyInstaller, False otherwise
+    """
+    return getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS')
+
+
+def get_application_root():
+    """
+    Get the application root directory.
+    
+    For PyInstaller executables, this returns the directory containing the executable.
+    For regular Python scripts, this returns the script's directory.
+    
+    Returns:
+        Path: Application root directory
+    """
+    if is_pyinstaller():
+        # PyInstaller: Use the directory containing the executable
+        return Path(sys.executable).parent
+    else:
+        # Regular Python: Use the directory containing this file's parent directory
+        return Path(__file__).parent.parent.parent
+
+
+def setup_persistent_directories(config_class=None):
+    """
+    Set up all persistent directories needed by the application.
+    
+    Args:
+        config_class: Configuration class to update (optional)
+        
+    Returns:
+        dict: Dictionary containing all created directory paths
+    """
+    directories = {}
+    
+    try:
+        # Create all necessary directories
+        directories['uploads'] = ensure_directory_exists(
+            get_uploads_directory(),
+            "uploads directory"
+        )
+        
+        directories['logs'] = ensure_directory_exists(
+            get_logs_directory(),
+            "logs directory"
+        )
+        
+        directories['data'] = ensure_directory_exists(
+            get_data_directory(),
+            "data directory"
+        )
+        
+        # Create subdirectories for uploads
+        directories['fixtures'] = ensure_directory_exists(
+            directories['uploads'] / "fixtures",
+            "fixtures upload directory"
+        )
+        
+        directories['zips'] = ensure_directory_exists(
+            directories['uploads'] / "zips",
+            "zip files upload directory"
+        )
+        
+        directories['temp'] = ensure_directory_exists(
+            directories['uploads'] / "temp",
+            "temporary files directory"
+        )
+        
+        logger.info("Successfully set up all persistent directories")
+        logger.info(f"Application running from PyInstaller: {is_pyinstaller()}")
+        logger.info(f"Base persistent directory: {get_platform_persistent_dir()}")
+        
+        return directories
+        
+    except Exception as e:
+        logger.error(f"Failed to setup persistent directories: {e}")
+        raise
+
+
+def cleanup_temp_directory(temp_dir=None):
+    """
+    Clean up temporary files in the temp directory.
+    
+    Args:
+        temp_dir (Path, optional): Temp directory to clean. Defaults to get_uploads_directory() / "temp"
+    """
+    if temp_dir is None:
+        temp_dir = get_uploads_directory() / "temp"
+    
+    temp_path = Path(temp_dir)
+    if not temp_path.exists():
+        return
+    
+    try:
+        import shutil
+        import time
+        
+        # Remove files older than 24 hours
+        current_time = time.time()
+        cleanup_count = 0
+        
+        for file_path in temp_path.rglob('*'):
+            if file_path.is_file():
+                try:
+                    file_age = current_time - file_path.stat().st_mtime
+                    if file_age > 86400:  # 24 hours in seconds
+                        file_path.unlink()
+                        cleanup_count += 1
+                except OSError:
+                    pass  # Skip files we can't delete
+        
+        if cleanup_count > 0:
+            logger.info(f"Cleaned up {cleanup_count} old temporary files")
+            
+    except Exception as e:
+        logger.warning(f"Failed to cleanup temp directory: {e}")
+
+
+# Environment variable names for configuration
+ENV_VARS = {
+    'uploads': 'MBETTER_UPLOADS_DIR',
+    'logs': 'MBETTER_LOGS_DIR', 
+    'data': 'MBETTER_DATA_DIR'
+}
+
+# Default directory structure
+DEFAULT_STRUCTURE = {
+    'uploads': ['fixtures', 'zips', 'temp'],
+    'logs': [],
+    'data': []
+}
\ No newline at end of file

config.py

 import os
-from dotenv import load_dotenv
+from pathlib import Path
 
-load_dotenv()
+def load_persistent_config():
+    """Load configuration from mbetterd.conf in persistent directories"""
+    config_values = {}
+    
+    # Import persistent directory utilities
+    try:
+        from app.utils.directories import get_persistent_directories
+        directories = get_persistent_directories()
+        config_path = os.path.join(directories['config'], 'mbetterd.conf')
+        
+        if os.path.exists(config_path):
+            print(f"Loading configuration from persistent directory: {config_path}")
+            with open(config_path, 'r') as f:
+                for line in f:
+                    line = line.strip()
+                    if line and not line.startswith('#') and '=' in line:
+                        key, value = line.split('=', 1)
+                        config_values[key.strip()] = value.strip()
+        else:
+            print(f"No persistent config found at {config_path}, checking for local .env")
+    except Exception as e:
+        print(f"Failed to load persistent config: {str(e)}")
+    
+    # Fallback to local .env file
+    if not config_values:
+        local_env_path = os.path.join(os.getcwd(), '.env')
+        if os.path.exists(local_env_path):
+            print(f"Loading configuration from local .env file: {local_env_path}")
+            with open(local_env_path, 'r') as f:
+                for line in f:
+                    line = line.strip()
+                    if line and not line.startswith('#') and '=' in line:
+                        key, value = line.split('=', 1)
+                        config_values[key.strip()] = value.strip()
+    
+    # Set environment variables from config file
+    for key, value in config_values.items():
+        if key not in os.environ:
+            os.environ[key] = value
+    
+    return config_values
+
+def get_config_value(key: str, default: str = '') -> str:
+    """Get configuration value with priority: environment > config file > default"""
+    return os.environ.get(key, default)
+
+# Load configuration
+_CONFIG_VALUES = load_persistent_config()
+
+# Import persistent directory utilities
+try:
+    from app.utils.directories import (
+        get_uploads_directory,
+        get_logs_directory,
+        get_data_directory,
+        setup_persistent_directories,
+        is_pyinstaller
+    )
+    PERSISTENT_DIRS_AVAILABLE = True
+except ImportError:
+    # Fallback if utilities not available
+    PERSISTENT_DIRS_AVAILABLE = False
 
 class Config:
-    """Base configuration class"""
-    SECRET_KEY = os.environ.get('SECRET_KEY') or 'dev-secret-key-change-in-production'
+    """Base configuration class with cross-platform persistent directory support"""
+    SECRET_KEY = get_config_value('SECRET_KEY', 'dev-secret-key-change-in-production')
     
     # Database Configuration
-    MYSQL_HOST = os.environ.get('MYSQL_HOST') or 'localhost'
-    MYSQL_PORT = int(os.environ.get('MYSQL_PORT') or 3306)
-    MYSQL_USER = os.environ.get('MYSQL_USER') or 'root'
-    MYSQL_PASSWORD = os.environ.get('MYSQL_PASSWORD') or ''
-    MYSQL_DATABASE = os.environ.get('MYSQL_DATABASE') or 'fixture_manager'
+    MYSQL_HOST = get_config_value('MYSQL_HOST', 'localhost')
+    MYSQL_PORT = int(get_config_value('MYSQL_PORT', '3306'))
+    MYSQL_USER = get_config_value('MYSQL_USER', 'root')
+    MYSQL_PASSWORD = get_config_value('MYSQL_PASSWORD', '')
+    MYSQL_DATABASE = get_config_value('MYSQL_DATABASE', 'fixture_manager')
     
     SQLALCHEMY_DATABASE_URI = f"mysql+pymysql://{MYSQL_USER}:{MYSQL_PASSWORD}@{MYSQL_HOST}:{MYSQL_PORT}/{MYSQL_DATABASE}"
     SQLALCHEMY_TRACK_MODIFICATIONS = False
@@ -23,48 +84,125 @@ class Config:
         'max_overflow': 0
     }
     
-    # File Upload Configuration
-    UPLOAD_FOLDER = os.environ.get('UPLOAD_FOLDER') or os.path.join(os.path.dirname(os.path.abspath(__file__)), 'uploads')
-    MAX_CONTENT_LENGTH = int(os.environ.get('MAX_CONTENT_LENGTH') or 2 * 1024 * 1024 * 1024)  # 2GB for large ZIP files
+    # Initialize persistent directories
+    @classmethod
+    def _get_persistent_directories(cls):
+        """Get persistent directories, with fallback to current directory"""
+        if PERSISTENT_DIRS_AVAILABLE:
+            try:
+                uploads_dir = get_uploads_directory()
+                logs_dir = get_logs_directory()
+                data_dir = get_data_directory()
+                return {
+                    'uploads': str(uploads_dir),
+                    'logs': str(logs_dir),
+                    'data': str(data_dir)
+                }
+            except Exception as e:
+                import logging
+                logging.warning(f"Failed to get persistent directories: {e}")
+        
+        # Fallback to current directory structure
+        base_dir = os.path.dirname(os.path.abspath(__file__))
+        return {
+            'uploads': os.path.join(base_dir, 'uploads'),
+            'logs': os.path.join(base_dir, 'logs'),
+            'data': os.path.join(base_dir, 'data')
+        }
+    
+    # File Upload Configuration - Use persistent directories
+    _persistent_dirs = _get_persistent_directories()
+    UPLOAD_FOLDER = get_config_value('UPLOAD_FOLDER', _persistent_dirs['uploads'])
+    MAX_CONTENT_LENGTH = int(get_config_value('MAX_CONTENT_LENGTH', str(2 * 1024 * 1024 * 1024)))  # 2GB for large ZIP files
     ALLOWED_FIXTURE_EXTENSIONS = {'csv', 'xlsx', 'xls'}
     ALLOWED_ZIP_EXTENSIONS = {'zip', '7z', 'rar'}  # Support more archive formats
     
+    # Upload subdirectories
+    FIXTURES_UPLOAD_FOLDER = os.path.join(UPLOAD_FOLDER, 'fixtures')
+    ZIP_UPLOAD_FOLDER = os.path.join(UPLOAD_FOLDER, 'zips')
+    TEMP_UPLOAD_FOLDER = os.path.join(UPLOAD_FOLDER, 'temp')
+    
     # Large File Upload Configuration
-    LARGE_FILE_THRESHOLD = int(os.environ.get('LARGE_FILE_THRESHOLD') or 100 * 1024 * 1024)  # 100MB
-    STREAMING_UPLOAD_ENABLED = os.environ.get('STREAMING_UPLOAD_ENABLED', 'True').lower() == 'true'
-    UPLOAD_TIMEOUT = int(os.environ.get('UPLOAD_TIMEOUT') or 3600)  # 1 hour timeout for large files
+    LARGE_FILE_THRESHOLD = int(get_config_value('LARGE_FILE_THRESHOLD', str(100 * 1024 * 1024)))  # 100MB
+    STREAMING_UPLOAD_ENABLED = get_config_value('STREAMING_UPLOAD_ENABLED', 'True').lower() == 'true'
+    UPLOAD_TIMEOUT = int(get_config_value('UPLOAD_TIMEOUT', '3600'))  # 1 hour timeout for large files
     
     # Security Configuration
-    JWT_SECRET_KEY = os.environ.get('JWT_SECRET_KEY') or SECRET_KEY
-    JWT_ACCESS_TOKEN_EXPIRES = int(os.environ.get('JWT_ACCESS_TOKEN_EXPIRES') or 3600)  # 1 hour
-    BCRYPT_LOG_ROUNDS = int(os.environ.get('BCRYPT_LOG_ROUNDS') or 12)
+    JWT_SECRET_KEY = get_config_value('JWT_SECRET_KEY', SECRET_KEY)
+    JWT_ACCESS_TOKEN_EXPIRES = int(get_config_value('JWT_ACCESS_TOKEN_EXPIRES', '3600'))  # 1 hour
+    BCRYPT_LOG_ROUNDS = int(get_config_value('BCRYPT_LOG_ROUNDS', '12'))
     
-    # Daemon Configuration
-    DAEMON_PID_FILE = os.environ.get('DAEMON_PID_FILE') or '/var/run/fixture-daemon.pid'
-    DAEMON_LOG_FILE = os.environ.get('DAEMON_LOG_FILE') or '/var/log/fixture-daemon.log'
-    DAEMON_WORKING_DIR = os.environ.get('DAEMON_WORKING_DIR') or '/var/lib/fixture-daemon'
+    # Daemon Configuration - Use persistent directories
+    DAEMON_PID_FILE = get_config_value('DAEMON_PID_FILE', os.path.join(_persistent_dirs['data'], 'fixture-daemon.pid'))
+    DAEMON_LOG_FILE = get_config_value('DAEMON_LOG_FILE', os.path.join(_persistent_dirs['logs'], 'fixture-daemon.log'))
+    DAEMON_WORKING_DIR = get_config_value('DAEMON_WORKING_DIR', _persistent_dirs['data'])
     
     # Web Server Configuration
-    HOST = os.environ.get('HOST') or '0.0.0.0'
-    PORT = int(os.environ.get('PORT') or 5000)
-    DEBUG = os.environ.get('DEBUG', 'False').lower() == 'true'
+    HOST = get_config_value('HOST', '0.0.0.0')
+    PORT = int(get_config_value('PORT', '5000'))
+    DEBUG = get_config_value('DEBUG', 'False').lower() == 'true'
     
     # Logging Configuration
-    LOG_LEVEL = os.environ.get('LOG_LEVEL') or 'INFO'
+    LOG_LEVEL = get_config_value('LOG_LEVEL', 'INFO')
     LOG_FORMAT = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
     
     # File Processing Configuration
-    CHUNK_SIZE = int(os.environ.get('CHUNK_SIZE') or 8192)  # 8KB chunks for file processing
-    MAX_CONCURRENT_UPLOADS = int(os.environ.get('MAX_CONCURRENT_UPLOADS') or 5)
+    CHUNK_SIZE = int(get_config_value('CHUNK_SIZE', '8192'))  # 8KB chunks for file processing
+    MAX_CONCURRENT_UPLOADS = int(get_config_value('MAX_CONCURRENT_UPLOADS', '5'))
+    
+    # PyInstaller Detection
+    IS_PYINSTALLER = is_pyinstaller() if PERSISTENT_DIRS_AVAILABLE else False
     
     @staticmethod
     def init_app(app):
-        """Initialize application with configuration"""
-        # Create necessary directories
-        os.makedirs(Config.UPLOAD_FOLDER, exist_ok=True)
-        os.makedirs(os.path.dirname(Config.DAEMON_PID_FILE), exist_ok=True)
-        os.makedirs(os.path.dirname(Config.DAEMON_LOG_FILE), exist_ok=True)
-        os.makedirs(Config.DAEMON_WORKING_DIR, exist_ok=True)
+        """Initialize application with configuration and set up persistent directories"""
+        import logging
+        logger = logging.getLogger(__name__)
+        
+        try:
+            if PERSISTENT_DIRS_AVAILABLE:
+                # Use the persistent directory system
+                directories = setup_persistent_directories()
+                logger.info(f"Set up persistent directories: {directories}")
+                
+                # Update app config with actual directory paths
+                app.config['UPLOAD_FOLDER'] = str(directories['uploads'])
+                app.config['FIXTURES_UPLOAD_FOLDER'] = str(directories['fixtures'])
+                app.config['ZIP_UPLOAD_FOLDER'] = str(directories['zips'])
+                app.config['TEMP_UPLOAD_FOLDER'] = str(directories['temp'])
+                
+                # Log PyInstaller detection
+                if is_pyinstaller():
+                    logger.info("Running from PyInstaller executable - using persistent directories")
+                else:
+                    logger.info("Running from Python script - using persistent directories")
+            else:
+                # Fallback to creating directories manually
+                os.makedirs(Config.UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.FIXTURES_UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.ZIP_UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.TEMP_UPLOAD_FOLDER, exist_ok=True)
+                logger.warning("Persistent directory utilities not available - using fallback directory creation")
+            
+            # Create other necessary directories
+            os.makedirs(os.path.dirname(Config.DAEMON_PID_FILE), exist_ok=True)
+            os.makedirs(os.path.dirname(Config.DAEMON_LOG_FILE), exist_ok=True)
+            os.makedirs(Config.DAEMON_WORKING_DIR, exist_ok=True)
+            
+        except Exception as e:
+            logger.error(f"Failed to initialize directories: {e}")
+            # Fallback to basic directory creation
+            try:
+                os.makedirs(Config.UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.FIXTURES_UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.ZIP_UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(Config.TEMP_UPLOAD_FOLDER, exist_ok=True)
+                os.makedirs(os.path.dirname(Config.DAEMON_PID_FILE), exist_ok=True)
+                os.makedirs(os.path.dirname(Config.DAEMON_LOG_FILE), exist_ok=True)
+                os.makedirs(Config.DAEMON_WORKING_DIR, exist_ok=True)
+            except Exception as fallback_error:
+                logger.error(f"Critical: Failed to create basic directories: {fallback_error}")
+                raise
 
 class DevelopmentConfig(Config):
     """Development configuration"""
@@ -72,7 +210,7 @@ class DevelopmentConfig(Config):
     SQLALCHEMY_ECHO = True
 
 class ProductionConfig(Config):
-    """Production configuration"""
+    """Production configuration with enhanced persistent directory support"""
     DEBUG = False
     SQLALCHEMY_ECHO = False
     
@@ -85,14 +223,27 @@ class ProductionConfig(Config):
         from logging.handlers import RotatingFileHandler
         
         if not app.debug:
+            # Ensure log directory exists
+            log_file = app.config.get('DAEMON_LOG_FILE', cls.DAEMON_LOG_FILE)
+            log_dir = os.path.dirname(log_file)
+            os.makedirs(log_dir, exist_ok=True)
+            
             file_handler = RotatingFileHandler(
-                cls.DAEMON_LOG_FILE, 
-                maxBytes=10240000, 
+                log_file,
+                maxBytes=10240000,
                 backupCount=10
             )
             file_handler.setFormatter(logging.Formatter(cls.LOG_FORMAT))
             file_handler.setLevel(logging.INFO)
             app.logger.addHandler(file_handler)
+            
+            # Log production startup info
+            logger = logging.getLogger(__name__)
+            logger.info("Production configuration initialized")
+            if PERSISTENT_DIRS_AVAILABLE:
+                logger.info(f"Using persistent directories - PyInstaller: {is_pyinstaller()}")
+                logger.info(f"Upload folder: {app.config.get('UPLOAD_FOLDER', cls.UPLOAD_FOLDER)}")
+                logger.info(f"Log file: {log_file}")
 
 class TestingConfig(Config):
     """Testing configuration"""

migrate_config.py

+#!/usr/bin/env python3
+"""
+Migration script to move .env to persistent directories as mbetterd.conf
+"""
+import os
+import shutil
+from pathlib import Path
+
+def migrate_config():
+    """Migrate .env file to persistent directories as mbetterd.conf"""
+    
+    # Import persistent directory utilities
+    try:
+        from app.utils.directories import get_persistent_directories, setup_persistent_directories
+        directories = setup_persistent_directories()
+        config_dir = directories['config']
+        
+        print(f"Using persistent config directory: {config_dir}")
+        
+        # Check if .env file exists
+        env_file = Path('.env')
+        if not env_file.exists():
+            print("No .env file found to migrate")
+            return False
+        
+        # Create mbetterd.conf path
+        conf_file = Path(config_dir) / 'mbetterd.conf'
+        
+        # Create config directory if it doesn't exist
+        os.makedirs(config_dir, exist_ok=True)
+        
+        # Copy .env to mbetterd.conf
+        print(f"Copying {env_file} to {conf_file}")
+        shutil.copy2(env_file, conf_file)
+        
+        # Add header to conf file
+        with open(conf_file, 'r') as f:
+            content = f.read()
+        
+        header = """# MBetter Daemon Configuration
+# This file contains configuration settings for the MBetter daemon
+# Format: KEY=VALUE (one per line)
+# Lines starting with # are comments
+
+"""
+        
+        with open(conf_file, 'w') as f:
+            f.write(header + content)
+        
+        print(f"Successfully migrated configuration to {conf_file}")
+        print(f"Original .env file preserved at {env_file}")
+        
+        return True
+        
+    except Exception as e:
+        print(f"Error migrating configuration: {str(e)}")
+        return False
+
+if __name__ == '__main__':
+    success = migrate_config()
+    if success:
+        print("\nConfiguration migration completed successfully!")
+        print("The application will now use mbetterd.conf from persistent directories.")
+        print("You can safely remove the .env file if desired.")
+    else:
+        print("\nConfiguration migration failed!")
+        print("The application will continue to use the .env file.")
\ No newline at end of file