• Stefy Lanza (nextime / spora )'s avatar
    Fix NaN/inf probability tensor error during generation · ae1d0e38
    Stefy Lanza (nextime / spora ) authored
    - Add InvalidLogitsProcessor to replace NaN and Inf values with finite numbers
    - Add _validate_generation_params() to clamp temperature and top_p to valid ranges
    - Add try-except blocks with fallback to greedy decoding on numerical errors
    - Add error handling in streaming responses to prevent crashes
    - Fix temperature=0 handling to use greedy decoding instead of sampling
    ae1d0e38
This project manages its dependencies using pip. Learn more
requirements.txt 1.91 KB