• Stefy Lanza (nextime / spora )'s avatar
    serve text-to-image and image-to-image from one copy of the weights (v0.4.94) · 79b3b8cd
    Stefy Lanza (nextime / spora ) authored
    A checkpoint is a UNet, a VAE and its text encoders; which task it performs is
    a property of the pipeline class wrapped around them, not of the weights. Both
    tasks already worked after 0.4.93, but each endpoint loaded its OWN pipeline
    under its own key -- so a client that creates a character with a model and then
    edits it with the same one held an 18 GB SDXL fine-tune twice, and the second
    load evicted the first. Alternating between the two thrashes the card
    continuously.
    
    Reading the two loaders side by side turned up the other half of it:
    _load_img2img_pipeline ignored the model's configuration altogether -- dtype
    hardcoded from global args, no offload, no quantization, no VAE override. Only
    _load_diffusers_pipeline honours models.json. A 4-bit model was therefore
    served at 4 bits for generation and bf16 for editing, out of two copies.
    
    So the model is loaded once, through the configured loader, under the manager's
    own key -- its VRAM accounting and eviction now see one model -- and each task
    gets a view built with AutoPipelineFor*.from_pipe around the components already
    resident. /v1/images/generations gets the same treatment in reverse, so a model
    an edit loaded can create a character without reloading.
    
    The views are cached on the base pipeline, not as extra models-dict entries.
    Deliberately: they share the base's tensors and so must die exactly when it
    does, and a second models-dict key per task is what let a cache key reach
    from_pretrained in 0.4.93. The img2img:/inpaint: keys are gone from the module
    entirely.
    
    Task detection asks diffusers' own tables rather than a signature, because a
    ControlNet text-to-image pipeline requires an `image` -- the control hint, not
    a source -- and a signature check would file it as img2img. Tested.
    
    Two behaviours worth knowing: the views share the UNet, so a LoRA or
    IP-Adapter applied through one task is visible to the others (same weights, by
    design); and SD, SDXL, Flux and Z-Image all have entries for all three tasks,
    so this is not an SD-only arrangement. I had assumed Z-Image was unmapped and
    wrote a test asserting it; the test caught that and both it and a code comment
    are corrected.
    
    28 tests here, 187 across the image files; full suite 4549 passed with the same
    15 pre-existing failures as 0.4.91 (diff empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    79b3b8cd
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...