• Stefy Lanza (nextime / spora )'s avatar
    Implement smart memory management with 3-tier offloading · be8bac00
    Stefy Lanza (nextime / spora ) authored
    Add _get_gpu_memory_map() to configure optimal memory strategy:
    - GPU: 95% of available VRAM (leaves 5% for CUDA overhead)
    - CPU: Up to user-specified limit (--ram) or auto-detected
    - Disk: Only as last resort when GPU+CPU are full
    
    Update --ram help text to clarify it's the CPU offloading limit.
    
    This provides better performance by prioritizing GPU, then CPU,
    and only using slow disk offloading when absolutely necessary.
    be8bac00
Name
Last commit
Last update
__pycache__ Loading commit data...
LICENSE.md Loading commit data...
README.md Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...