• Stefy Lanza (nextime / spora )'s avatar
    Add OOM handling during generation to prevent crashes · 33a7e421
    Stefy Lanza (nextime / spora ) authored
    - Wrap generate() with try-except to catch CUDA OOM errors
    - On OOM: clear CUDA cache, retry with half tokens, return graceful error if still failing
    - Wrap generate_stream() thread with error handling using shared variable
    - Yield error messages to client instead of crashing the process
    - Allows server to continue running after generation OOM
    33a7e421
Name
Last commit
Last update
.gitignore Loading commit data...
LICENSE.md Loading commit data...
README.md Loading commit data...
build.sh Loading commit data...
coder Loading commit data...
coderai Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
requirements.txt~ Loading commit data...