-
Stefy Lanza (nextime / spora ) authored
A checkpoint is a UNet, a VAE and its text encoders; which task it performs is a property of the pipeline class wrapped around them, not of the weights. Both tasks already worked after 0.4.93, but each endpoint loaded its OWN pipeline under its own key -- so a client that creates a character with a model and then edits it with the same one held an 18 GB SDXL fine-tune twice, and the second load evicted the first. Alternating between the two thrashes the card continuously. Reading the two loaders side by side turned up the other half of it: _load_img2img_pipeline ignored the model's configuration altogether -- dtype hardcoded from global args, no offload, no quantization, no VAE override. Only _load_diffusers_pipeline honours models.json. A 4-bit model was therefore served at 4 bits for generation and bf16 for editing, out of two copies. So the model is loaded once, through the configured loader, under the manager's own key -- its VRAM accounting and eviction now see one model -- and each task gets a view built with AutoPipelineFor*.from_pipe around the components already resident. /v1/images/generations gets the same treatment in reverse, so a model an edit loaded can create a character without reloading. The views are cached on the base pipeline, not as extra models-dict entries. Deliberately: they share the base's tensors and so must die exactly when it does, and a second models-dict key per task is what let a cache key reach from_pretrained in 0.4.93. The img2img:/inpaint: keys are gone from the module entirely. Task detection asks diffusers' own tables rather than a signature, because a ControlNet text-to-image pipeline requires an `image` -- the control hint, not a source -- and a signature check would file it as img2img. Tested. Two behaviours worth knowing: the views share the UNet, so a LoRA or IP-Adapter applied through one task is visible to the others (same weights, by design); and SD, SDXL, Flux and Z-Image all have entries for all three tasks, so this is not an SD-only arrangement. I had assumed Z-Image was unmapped and wrote a test asserting it; the test caught that and both it and a code comment are corrected. 28 tests here, 187 across the image files; full suite 4549 passed with the same 15 pre-existing failures as 0.4.91 (diff empty). Co-Authored-By:
Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
79b3b8cd