• Stefy Lanza (nextime / spora )'s avatar
    edit an image from an instruction, and load a GGUF component at all (v0.4.96) · 02eaca42
    Stefy Lanza (nextime / spora ) authored
    Requested for a client that edits photos by instruction -- "make the dress
    red", "add a necklace" -- which plain img2img cannot do at any strength,
    because `strength` is the fraction of the source it destroys before
    re-imagining it. Serving Qwen-Image-Edit / FLUX.1-Kontext class models turned
    up two latent bugs on the way.
    
    The instruction path. /v1/images/edits now shapes its call to what the
    pipeline accepts, from the signature, rather than branching on the model:
    
      QwenImageEditPlus / QwenImageEdit / FluxKontext   image ✓  strength ✗
      SDXL-Img2Img / QwenImageImg2Img / FluxImg2Img     image ✓  strength ✓
    
    Taking an `image` with no `strength` IS the distinction, and it generalises,
    so no list of class names has to be maintained. `strength` is dropped where it
    does not exist -- a caller cannot know which kind of model it is addressing,
    and these pipelines carry no **kwargs to swallow it, so passing it through
    would be a hard TypeError. `guidance_scale` becomes `true_cfg_scale`, the knob
    they actually read. Steps default to 4 at cfg 1.0 for them instead of
    inheriting img2img's 30 at 7.5, which would waste a minute an image and
    overcook the result. Up to three images may be given (`image` plus `images`)
    for a composed edit, conditioned at one shape; more is a 400.
    
    `instruction_edit` is reported in /v1/models so a client can hide the strength
    slider and treat the prompt as an instruction. Verified to classify the four
    Qwen/Kontext/FireRed ids and to leave Pony, Z-Image and Illustrious as img2img.
    
    component_models is new: a map from a component to ANOTHER repo, loaded and
    handed to the pipeline as a prebuilt object. Several Qwen-Image-Edit
    derivatives ship a transformer and nothing else -- the base supplies the VAE,
    the text encoder and the scheduler -- and there was previously no way to
    express that, so they could not be served at all. Its entry in
    component_quantization still applies, so a transformer too large for the card
    is quantized as it loads.
    
    Two latent bugs, both of which anyone using GGUF image components would have
    hit:
    
      * `gguf` was never a declared dependency. build_gguf_pipeline_components has
        existed through several releases and could not have worked: diffusers needs
        gguf>=0.10.0 for GGUFQuantizationConfig and only says so mid-load, as
        "Unable to load weights from checkpoint file". Now in requirements.txt,
        which build.sh installs.
      * from_single_file was called with no `config`, so it INFERRED the
        architecture -- and its fallback is Stable Diffusion 1.5. A Qwen-Image,
        Flux or Wan transformer therefore failed with "stable-diffusion-v1-5 does
        not appear to have a file named config.json". It now passes the base
        pipeline's own component subfolder, which was already at hand.
    
    Measured on an RTX 3090 after the fixes: the Q4_K_M transformer binds as
    QwenImageTransformer2DModel (13.23B params, 85s) and the assembled
    QwenImageEditPlusPipeline is 22.68 GiB resident with a 4-bit text encoder --
    which fits 24 GB but leaves 1.32 GiB and OOMs on 390 MiB of activations. That
    is a deployment setting, not a code one, and it is why the figure is measured
    here rather than asserted.
    
    Full suite 4558 passed with the same 15 pre-existing failures as 0.4.91 (diff
    empty).
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    02eaca42
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...