-
Stefy Lanza (nextime / spora ) authored
Z-Image-Turbo-unsloth-bnb-4bit (and any pre-quantized bnb/fp8/nf4/gptq/awq checkpoint, or a runtime-quantized model) dequantizes to a HALF compute dtype and its transformer uses FlashAttention, which only supports fp16/bf16. The per-model image loader defaults precision to f32, so such a model loaded in float32 and crashed with "FlashAttention only support fp16 and bf16 data type" (image/character generation → 400/500), besides wasting VRAM. When precision is left at the f32 default AND the model is quantized (name contains bnb/4bit/8bit/fp8/nf4/gptq/awq, or config sets load_in_4bit/8bit/ component_quantization), load in bf16 instead. Non-quantized models keep the f32 default. --no-ram already forced fp16, so it's unaffected. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
b3014a3d
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| frontproxy | ||
| models | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| main.py | ||
| platform_paths.py | ||
| system_app.py |