Skip to content
Commit 5afc48a2 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

an f32 default filled the card, and a decode OOM was called a load failure



Two image models "failed to load" on a 24 GB card mid-run. Neither had failed
to load: both loaded, and both completed every denoising step before dying on
the VAE decode.

pornmaster-pro-pony (SDXL, no precision on its entry) loaded at f32 —
"Using precision: f32 (torch.float32)" — finished 30/30 steps, then died on
"Tried to allocate 1008.00 MiB ... 704.69 MiB is free", with 22.12 GiB of a
23.56 GiB card held by its own weights. The same architecture at half width
measures 7.49 GB. Krea-2-Edit, correctly quantized (GGUF Q4_K_M transformer +
4-bit text encoder, bf16), finished 8/8 steps and died the same way on 190 MiB.

The bf16-for-an-undeclared-precision default was unreachable: ImageConfig
supplied precision "f32" into every per-model config, so by the time the loader
asked, nothing was ever undeclared. The default is now "auto" and resolves from
the card — a half precision at 32 GB or less, f32 above it, since a card big
enough to hold both loses nothing by upcasting. bf16 rather than fp16 for the
half type: same two bytes, but fp16 tops out near 65504 and a flow-matching DiT
runs past it into NaN and a black image (Z-Image-Turbo, 2026-10-25). An
explicit precision, on the card or the model, is still honoured.

The reporting was the costlier half. The generate call sat inside the load's
`except Exception`, so a decode OOM was filed as a load error, fell through to
the sd.cpp backend, and answered "Failed to load image model 'X'. Errors:
diffusers: CUDA out of memory". Two separate readers concluded from that wording
that VRAM eviction was leaking and that the models never loaded; neither was
true, and the real causes went unexamined for it. The same wording had already
mislabelled a seeded-request failure once — see models.utils.generator_device.
Generation now runs outside that handler and reports itself: 503 and "the model
loaded and ran, so this is not a load failure" for an OOM, naming the decode.

Also honour per-model `vae_tiling` on the image path. Slicing splits a BATCH
and so does nothing for n=1 — which is the case that OOMed; tiling is what
bounds the peak of one large decode.

Co-Authored-By: default avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
parent 523761cf
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment