• Stefy Lanza (nextime / spora )'s avatar
    Add fallback for models that don't support load_in_4bit quantization · e7e2c626
    Stefy Lanza (nextime / spora ) authored
    Modify _try_load_model() to catch TypeError when quantization arguments
    are not supported by the model class. When this happens, the method now:
    1. Warns the user about unsupported quantization
    2. Retries loading the model without quantization arguments
    3. Returns the model successfully if loading works
    
    This fixes issues with models like Qwen3.5 that don't support
    bitsandbytes quantization.
    e7e2c626
Name
Last commit
Last update
.gitignore Loading commit data...
LICENSE.md Loading commit data...
README.md Loading commit data...
build.sh Loading commit data...
coder Loading commit data...
coderai Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
requirements.txt~ Loading commit data...