- 28 Feb, 2026 4 commits
-
-
Stefy Lanza (nextime / spora ) authored
- Comment out procname in requirements-nvidia.txt - Comment out procname in requirements-vulkan.txt - Add note about requiring libproc2-dev for procname
-
Stefy Lanza (nextime / spora ) authored
- build.sh: Update package list to include glslc, glslang-tools, glslang-dev - README.md: Update installation instructions with correct package names - Add better guidance for finding glslc in non-standard locations
-
Stefy Lanza (nextime / spora ) authored
- Update build.sh to check for glslc before attempting build - Update README with correct package names (glslang-tools/glslang) - Add troubleshooting for missing glslc error
-
Stefy Lanza (nextime / spora ) authored
- Add build.sh script with nvidia/vulkan arguments (default: nvidia) - Create backend abstraction: ModelBackend base class - Implement NvidiaBackend using HuggingFace Transformers - Implement VulkanBackend using llama-cpp-python with GGUF models - Add separate requirements files for nvidia and vulkan backends - Add --backend argument with auto/nvidia/vulkan options - Add Vulkan-specific options: --n-gpu-layers, --n-ctx - Make procname import optional - Update README with comprehensive Vulkan usage instructions - Add Vulkan troubleshooting section - Add GGUF model recommendations The application now supports: - NVIDIA GPUs via PyTorch/Transformers (HuggingFace models) - AMD GPUs via llama-cpp-python/Vulkan (GGUF models)
-
- 27 Feb, 2026 2 commits
-
-
Stefy Lanza (nextime / spora ) authored
- Add InvalidLogitsProcessor to replace NaN and Inf values with finite numbers - Add _validate_generation_params() to clamp temperature and top_p to valid ranges - Add try-except blocks with fallback to greedy decoding on numerical errors - Add error handling in streaming responses to prevent crashes - Fix temperature=0 handling to use greedy decoding instead of sampling
-
Stefy Lanza (nextime / spora ) authored
- Add main server script with FastAPI and memory-aware model loading - Add requirements.txt with dependencies and platform-specific PyTorch options - Add comprehensive README.md with installation, usage, and troubleshooting - Add LICENSE.md with GPLv3 license
-