• Stefy Lanza (nextime / spora )'s avatar
    Fix NaN/inf probability tensor error during generation · ae1d0e38
    Stefy Lanza (nextime / spora ) authored
    - Add InvalidLogitsProcessor to replace NaN and Inf values with finite numbers
    - Add _validate_generation_params() to clamp temperature and top_p to valid ranges
    - Add try-except blocks with fallback to greedy decoding on numerical errors
    - Add error handling in streaming responses to prevent crashes
    - Fix temperature=0 handling to use greedy decoding instead of sampling
    ae1d0e38
Name
Last commit
Last update
__pycache__ Loading commit data...
LICENSE.md Loading commit data...
README.md Loading commit data...
coderai Loading commit data...
requirements.txt Loading commit data...