take the size the caller asked for, and the words they trained (v0.4.90)

Three things the image endpoints accepted and then quietly did something else
with.

size. A size that did not parse fell through to the default, so "1024" or
"1k x 1k" generated a square nobody asked for and the response looked like a
success. A side that was not a multiple of 8 went to the pipeline as-is, where
the VAE cannot represent it. Nothing capped it either, so 4096x4096 was
accepted and came back as a CUDA OOM -- which reads like a busy GPU rather
than a size the caller chose. Now: a 400 quoting what was sent, a 400 for a
zero dimension (snapping it up would answer an uninitialised client variable
with an 8x8 image), snapping to the grid with a log line, and a 400 naming
image.max_side / image.max_pixels when it is over the ceiling. "auto" is
accepted as in the OpenAI API, and the dead `width != width` NaN guard on a
value that was already an int is gone. The archive records the size actually
generated alongside the one requested.

/v1/images/inpaint no longer defaults to 1024x1024, which squashed every
non-square photo and pulled the mask out of alignment with it -- worse than
the same default on /v1/images/edits, where it was fixed in 0.4.86. Omitted
now keeps the source's shape.

Textual inversions. easynegative, bad-hands-5 and the rest are vectors living
in CLIP's embedding table under a word of their own, and until the file is
loaded that word is an ordinary unknown token -- so a large part of the
ecosystem's negative prompting asked for nothing at all. They load from the
request (embeddings: [...]), from the model's config (textual_inversion), or
by filename from image.embeddings_dir, which is how A1111 works and what a
prompt copied off a model card assumes. Two details decide whether this works
at all: an SDXL-format embedding holds a clip_l and a clip_g tensor that must
go into the matching tokenizer/encoder pair, and a multi-vector embedding
occupies several tokens (tok, tok_1, tok_2...) that the prompt has to be
expanded into -- without it an eight-vector embedding contributes one vector
and appears inert. A name the caller gave that will not load is a 400; a bad
file in the directory is a line in the log, because one file should not break
every request.

/v1/images/outfit goes through the same encoder now, so the weight grammar and
clip_skip reach it too. Its per-frame video path encodes the prompt once for
the clip instead of once per frame.

And a 400 is no longer reported as a failed backend. On the first request for
a model that is not loaded yet, the generate call sits inside the try that
falls back from diffusers to sd.cpp, so a rejected request was caught as a
load failure and answered with a report about which backends failed. The three
edit endpoints did the same through their catch-alls. All four re-raise
HTTPException untouched.

Found by the tests while writing them: the embeddings directory was searched
by the renamed trigger word rather than the filename, so "easynegative:myneg"
never resolved, and "768X512" did not parse.

64 new tests; full suite 4475 passed with the same 15 pre-existing failures as
0.4.89 (diff empty).
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
parent 777406dc
......@@ -838,9 +838,11 @@ persisted per `config_id`, so the configs don't overwrite each other.
| `POST /v1/images/from3d` | 3D model → rendered 2D image |
| `GET /v1/images/progress` | Current generation progress (step/total) |
Generations, edits and inpaint take `negative_prompt`, `clip_skip` and the
`(tag:1.3)` / `[tag]` / `BREAK` prompt grammar, and encode prompts of any
length (CLIP's 77-token limit does not apply) — see
Generations, edits and inpaint take `negative_prompt`, `clip_skip`,
`embeddings` (textual inversions, so `easynegative` and friends work in a
prompt) and the `(tag:1.3)` / `[tag]` / `BREAK` prompt grammar, and encode
prompts of any length — CLIP's 77-token limit does not apply. `size` is
validated rather than silently replaced. See
[docs/image-prompts.md](docs/image-prompts.md).
### Video
......
......@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here.
__version__ = "0.4.89"
__version__ = "0.4.90"
__tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
# Configure the CUDA caching allocator BEFORE torch is imported anywhere.
......
......@@ -2931,7 +2931,7 @@ async def api_model_configure(request: Request, username: str = Depends(require_
# the A1111 weight grammar is parsed. Per model because the
# booru-tag fine-tunes need clip_skip 2 whatever the caller
# sends, and an OpenAI-API client has no field to send it in.
"clip_skip", "prompt_weighting",
"clip_skip", "prompt_weighting", "textual_inversion",
"component_quantization", "output_crf", "force_vram_update",
"balanced_gpu_percent", "acceleration",
"cache_type_k", "cache_type_v", "kv_offload", "n_batch", "n_ubatch", "n_seq_max",
......
......@@ -1553,6 +1553,12 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
</div>
<span class="form-hint">On, the A1111 grammar is parsed: <code>(tag:1.3)</code> emphasises, <code>[tag]</code> attenuates, bare <code>(tag)</code> is 1.1, nesting multiplies, <code>\(</code> is a literal bracket, and <code>BREAK</code> starts a new 75-token window. Off, brackets reach the model as text. Either way the prompt is encoded in as many windows as it needs, so nothing is cut at 77 tokens.</span>
<div class="form-row" style="margin-top:.75rem">
<label class="form-label">Textual inversions <span class="muted">(comma-separated; always loaded for this model)</span></label>
<input type="text" id="cfg-textual-inversion" class="form-input" placeholder="easynegative, bad-hands-5, my-emb.safetensors:myneg">
<span class="form-hint">Learned prompt tokens, so their trigger words work in any prompt sent to this model. A bare name is looked up in the server's embeddings directory (<code>image.embeddings_dir</code> — everything in it is loaded anyway); a path or a HuggingFace repo id also works, and <code>ref:token</code> renames the trigger word. A request's own <code>embeddings</code> list is loaded on top of these.</span>
</div>
<!-- video output quality (CRF) — only relevant for video models -->
<div class="form-row" style="margin-top:.75rem;max-width:340px">
<label class="form-label">Video output quality — CRF
......@@ -4305,6 +4311,8 @@ function openCfgModal(idx, cfgIdx){
document.getElementById('cfg-vae-tiling').checked = !!s.vae_tiling;
document.getElementById('cfg-clip-skip').value = (s.clip_skip != null ? s.clip_skip : '');
document.getElementById('cfg-prompt-weighting').checked = (s.prompt_weighting !== false);
document.getElementById('cfg-textual-inversion').value =
(Array.isArray(s.textual_inversion) ? s.textual_inversion.join(', ') : (s.textual_inversion || ''));
document.getElementById('cfg-output-crf').value = (s.output_crf != null ? s.output_crf : '');
document.getElementById('cfg-balanced-gpu-pct').value = (s.balanced_gpu_percent != null ? s.balanced_gpu_percent : '');
_populateCompSelects();
......@@ -5410,6 +5418,8 @@ async function saveModelConfig(){
clip_skip: (document.getElementById('cfg-clip-skip').value.trim() === ''
? null : parseInt(document.getElementById('cfg-clip-skip').value, 10)),
prompt_weighting: document.getElementById('cfg-prompt-weighting').checked,
textual_inversion: document.getElementById('cfg-textual-inversion').value
.split(',').map(v => v.trim()).filter(v => v !== ''),
};
try{
const r = await fetch(ROOT_PATH + '/admin/api/model-configure',{
......
This diff is collapsed.
......@@ -310,6 +310,13 @@ class ImageConfig:
# Parse the A1111 prompt grammar -- (tag:1.3), [tag], BREAK. Off keeps
# brackets literal; the prompt is windowed past 77 tokens either way.
prompt_weighting: bool = True
# Request-size ceiling. A size above it is a 400 naming the limit, which
# is a better answer than the CUDA OOM an absurd size used to produce.
max_side: int = 4096
max_pixels: int = 4096 * 1024 # 4.2 MP -- 2048x2048, 2560x1440, …
# A flat directory of textual-inversion embeddings, loaded by filename so
# a prompt copied off a model card finds its trigger word. Empty = none.
embeddings_dir: str = ""
@dataclass
......@@ -2137,7 +2144,10 @@ class ConfigManager:
"vae_tiling": self.config.image.vae_tiling,
"clip_on_cpu": self.config.image.clip_on_cpu,
"clip_skip": self.config.image.clip_skip,
"prompt_weighting": self.config.image.prompt_weighting
"prompt_weighting": self.config.image.prompt_weighting,
"max_side": self.config.image.max_side,
"max_pixels": self.config.image.max_pixels,
"embeddings_dir": self.config.image.embeddings_dir
},
"archive": {
"enabled": self.config.archive.enabled,
......
......@@ -1227,6 +1227,9 @@ def main():
global_args.clip_on_cpu = config.image.clip_on_cpu
global_args.image_clip_skip = config.image.clip_skip
global_args.image_prompt_weighting = config.image.prompt_weighting
global_args.image_max_side = config.image.max_side
global_args.image_max_pixels = config.image.max_pixels
global_args.image_embeddings_dir = config.image.embeddings_dir or None
global_args.system_prompt = config.system_prompt
global_args.tools_closer_prompt = config.tools_closer_prompt
global_args.grammar_guided_gen = config.grammar_guided
......
......@@ -256,6 +256,30 @@ def _is_sdxl(pipeline) -> bool:
return getattr(pipeline, "text_encoder_2", None) is not None
def _expand_learned_tokens(pipeline, text: str, tokenizer) -> str:
"""``text`` with multi-vector textual inversions written out in full.
A textual inversion trained on several vectors is registered as ``tok``,
``tok_1``, ``tok_2``..., and a prompt only ever says ``tok``. Diffusers
expands it in ``maybe_convert_prompt`` on its way into the tokenizer, so
anything encoding a prompt itself has to do the same -- otherwise an
embedding like ``easynegative`` (eight vectors) contributes exactly one of
them and the prompt looks like it did nothing.
It replaces only registered trigger words, so the weight grammar passes
through untouched and this is safe to run before parsing.
"""
if not text:
return text
convert = getattr(pipeline, "maybe_convert_prompt", None)
if convert is None:
return text
try:
return convert(text, tokenizer)
except Exception:
return text
def _hidden(out, text_encoder, clip_skip: Optional[int], sdxl: bool):
"""The hidden state a diffusers pipeline would have used.
......@@ -331,8 +355,13 @@ def encode_weighted_prompt(pipeline, prompt: str, negative_prompt: str = "",
tokenized = []
n_windows = 1
for tokenizer, _ in pairs:
pos = tokenize_weighted(tokenizer, prompt or "", plain=plain)
neg = tokenize_weighted(tokenizer, negative_prompt or "", plain=plain)
pos = tokenize_weighted(
tokenizer, _expand_learned_tokens(pipeline, prompt, tokenizer),
plain=plain)
neg = tokenize_weighted(
tokenizer,
_expand_learned_tokens(pipeline, negative_prompt, tokenizer),
plain=plain)
tokenized.append((tokenizer, pos, neg))
n_windows = max(n_windows, len(pos[0]), len(neg[0]))
......
# CoderAI - OpenAI-compatible API server
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
"""Learned prompt tokens: ``easynegative`` and the rest of the embeddings.
A textual inversion is a handful of vectors trained to sit in CLIP's embedding
table under a word of their own. Writing that word in a prompt is how half the
Stable-Diffusion ecosystem's negative prompts are built -- ``easynegative``,
``bad-hands-5``, ``FastNegativeV2`` -- and without the embedding loaded the
word is just an unknown token, so the prompt silently asks for nothing.
Three ways to get one loaded, in the order a request resolves them:
* ``embeddings: ["easynegative"]`` on the request, for an ad-hoc one.
* ``textual_inversion`` on the model's config, for the ones a model always
wants.
* every file in ``image.embeddings_dir``, discovered by filename -- which is
how A1111 works, and what a prompt copied from a model card assumes.
A reference is a bare name (matched against the directory), a path to a
``.safetensors`` / ``.pt`` / ``.bin`` file, or a HuggingFace repo id. Appending
``:token`` renames the trigger word: ``my-emb.safetensors:myneg``.
Two details decide whether this actually works:
* **SDXL keeps two text encoders**, so an SDXL-format embedding holds a
``clip_l`` and a ``clip_g`` tensor that have to go into the matching
tokenizer/encoder pair. Loading only the first gives the token an embedding
in one encoder and noise in the other.
* **A multi-vector embedding occupies several tokens**, registered as
``tok``, ``tok_1``, ``tok_2``... Diffusers expands the prompt through
``maybe_convert_prompt`` before tokenizing; anything encoding a prompt
itself has to do the same or only the first vector is ever used. That is
handled in :mod:`codai.models.prompt_weighting`.
"""
import logging
import os
from typing import Dict, Iterable, List, Optional, Tuple
_log = logging.getLogger(__name__)
#: What an embedding file looks like on disk.
SUFFIXES = (".safetensors", ".pt", ".bin", ".ckpt")
#: Marks the tokens already loaded into a pipeline, so a second request does
#: not try to register the same trigger word again (diffusers raises on that).
_LOADED_ATTR = "_coderai_textual_inversions"
def _split_ref(ref: str) -> Tuple[str, Optional[str]]:
"""``"file.safetensors:token"`` -> (``"file.safetensors"``, ``"token"``).
A Windows drive letter or an ``hf://`` style prefix would be mangled by a
naive split, so only a colon in the last path segment counts.
"""
ref = str(ref).strip()
head, sep, tail = ref.rpartition(":")
if sep and tail and "/" not in tail and os.sep not in tail and len(head) > 1:
return head, tail
return ref, None
def _token_for(ref: str, explicit: Optional[str]) -> str:
"""The trigger word: the one asked for, else the filename, as A1111 does."""
if explicit:
return explicit
base = os.path.basename(ref.rstrip("/"))
for suffix in SUFFIXES:
if base.endswith(suffix):
return base[: -len(suffix)]
return base
def discover(directory: Optional[str]) -> Dict[str, str]:
"""``{token: path}`` for every embedding file in ``directory``.
Not recursive and not sorted by anything meaningful: a flat directory of
trigger words is the convention these files arrive in.
"""
out: Dict[str, str] = {}
if not directory:
return out
directory = os.path.expanduser(directory)
try:
names = os.listdir(directory)
except OSError as exc:
_log.warning("embeddings_dir %s cannot be read: %s", directory, exc)
return out
for name in sorted(names):
if not name.endswith(SUFFIXES):
continue
path = os.path.join(directory, name)
if os.path.isfile(path):
out[_token_for(name, None)] = path
return out
def resolve(ref: str, directory: Optional[str] = None) -> Tuple[str, str]:
"""``(source, token)`` for one reference, or raise ``FileNotFoundError``.
``source`` is a path that exists, or a HuggingFace repo id left for
diffusers to fetch. A bare name is looked up in ``directory`` first,
because that is what a prompt copied off a model card means by it.
"""
raw, explicit = _split_ref(ref)
token = _token_for(raw, explicit)
if os.path.isfile(os.path.expanduser(raw)):
return os.path.expanduser(raw), token
if directory:
# Looked up by the name of the FILE, not by the trigger word: with
# "easynegative:myneg" the word is myneg and the file is still
# easynegative.safetensors.
name = _token_for(raw, None)
found = discover(directory)
if name in found:
return found[name], token
# a name given with its extension, or a file sitting beside the others
candidate = os.path.join(os.path.expanduser(directory),
os.path.basename(raw))
if os.path.isfile(candidate):
return candidate, token
if "/" in raw and not raw.startswith((".", "/", "~")):
return raw, token # a repo id; diffusers resolves and caches it
raise FileNotFoundError(
f"textual inversion {ref!r} not found: no such file, nothing by that "
f"name in the embeddings directory"
+ (f" ({directory})" if directory else " (image.embeddings_dir is not set)")
+ ", and it is not a HuggingFace repo id")
def loaded_tokens(pipeline) -> set:
"""Trigger words already registered on this pipeline."""
return set(getattr(pipeline, _LOADED_ATTR, None) or ())
def _load_one(pipeline, source: str, token: str) -> None:
"""Register ``token`` on the pipeline, both encoders included on SDXL."""
sdxl = getattr(pipeline, "text_encoder_2", None) is not None
state = None
if sdxl and os.path.isfile(source) and source.endswith(".safetensors"):
try:
from safetensors.torch import load_file
state = load_file(source)
except Exception as exc:
_log.debug("cannot inspect %s as safetensors (%s)", source, exc)
if state is not None and "clip_g" in state and "clip_l" in state:
# The SDXL layout: one tensor per text encoder, and they are not
# interchangeable -- 768 wide for clip_l, 1280 for clip_g.
pipeline.load_textual_inversion(
state["clip_l"], token=token,
text_encoder=pipeline.text_encoder, tokenizer=pipeline.tokenizer)
pipeline.load_textual_inversion(
state["clip_g"], token=token,
text_encoder=pipeline.text_encoder_2, tokenizer=pipeline.tokenizer_2)
else:
pipeline.load_textual_inversion(source, token=token)
def apply(pipeline, refs: Iterable[str] = (), directory: Optional[str] = None,
strict: bool = False) -> List[str]:
"""Load ``refs`` (plus everything in ``directory``) onto ``pipeline``.
Returns the trigger words now usable in a prompt. Already-loaded ones are
skipped, so this is cheap to call on every request -- which it is, because
a pipeline outlives the request that loaded it and a later request must
find the same tokens there.
``strict`` raises on a reference that cannot be resolved or loaded: that is
for the ones a caller named explicitly, where quietly rendering without the
embedding is the failure mode worth avoiding. Directory sweeps are never
strict -- one unreadable file should not break every request.
"""
if not hasattr(pipeline, "load_textual_inversion"):
if strict and list(refs):
raise RuntimeError("this pipeline cannot load textual inversions")
return []
already = loaded_tokens(pipeline)
# (source, token, asked for by name) -- the last flag decides whether a
# failure is this request's problem or just a note in the log.
wanted: List[Tuple[str, str, bool]] = []
for ref in refs or ():
if not str(ref).strip():
continue
source, token = resolve(str(ref), directory) # may raise, by design
wanted.append((source, token, True))
named = {token for _, token, _ in wanted}
for token, path in discover(directory).items():
if token not in named:
wanted.append((path, token, False))
added = []
for source, token, explicit in wanted:
if token in already:
continue
try:
_load_one(pipeline, source, token)
except Exception as exc:
if strict and explicit:
raise RuntimeError(
f"textual inversion {token!r} could not be loaded: {exc}") from exc
_log.warning("textual inversion %r (%s) not loaded: %s",
token, source, exc)
continue
already.add(token)
added.append(token)
setattr(pipeline, _LOADED_ATTR, already)
if added:
_log.info("Textual inversions loaded: %s", ", ".join(sorted(added)))
return sorted(already)
......@@ -57,6 +57,12 @@ class ImageGenerationRequest(BaseModel):
"Take the text embedding this many CLIP layers from the end instead of the last. "
"The booru-tag SDXL fine-tune families (Pony, Illustrious, NoobAI) are trained at 2. "
"Omitted uses the model's configured default, then the pipeline's own."))
embeddings: Optional[List[str]] = Field(None, description=(
"Textual-inversion embeddings to load before generating, so their trigger "
"words work in the prompt (e.g. [\"easynegative\"]). A bare name is looked "
"up in image.embeddings_dir; a path or a HuggingFace repo id also works, and "
"'ref:token' renames the trigger word. Everything in embeddings_dir, plus the "
"model's own textual_inversion list, is loaded whether this is set or not."))
prompt_weighting: Optional[bool] = Field(None, description=(
"Parse the A1111 prompt grammar: '(tag:1.3)' to emphasise, '[tag]' to attenuate, "
"'(tag)' for 1.1, nesting multiplies, '\\\\(' escapes, and BREAK starts a new "
......
......@@ -28,7 +28,7 @@ reference; these are the deep dives.
| [ocr.md](ocr.md) | The `/v1/ocr` document-transcription subsystem |
| [chunking.md](chunking.md) | `POST /v1/chunk`: splitting a document for a retrieval index, with the structure decided per request by a template |
| [expressive-tts.md](expressive-tts.md) | Expressive text-to-speech |
| [image-prompts.md](image-prompts.md) | Image prompts: the `(tag:1.3)` / `[tag]` / `BREAK` grammar, prompts past CLIP's 77 tokens, `clip_skip`, negative prompts |
| [image-prompts.md](image-prompts.md) | Image prompts: the `(tag:1.3)` / `[tag]` / `BREAK` grammar, prompts past CLIP's 77 tokens, `clip_skip`, negative prompts, textual-inversion embeddings, size validation |
| [zimage-lora-training.md](zimage-lora-training.md) | LoRA training for image models |
| [dtype-auto-selection.md](dtype-auto-selection.md) | How model dtype is chosen automatically |
......
......@@ -49,6 +49,7 @@ take:
| `negative_prompt` | What to avoid. The counterpart of the quality tags (`score_4, score_5, score_6, worst quality`) and as load-bearing as the prompt itself. |
| `clip_skip` | Read the embedding this many CLIP layers from the end. Omitted uses the model's configured value, then the pipeline's own default. |
| `prompt_weighting` | `false` keeps brackets literal, for a caller sending prose. The windowing stays on either way, so turning it off never costs tokens. |
| `embeddings` | Textual inversions to load, so their trigger words work in the prompt — `["easynegative"]`. |
```json
{
......@@ -80,6 +81,62 @@ through. Until v0.4.89 the negative prompt was not: the call hardcoded an empty
string, so for every GGUF image model whatever the caller asked to avoid was
discarded between the API and the engine.
## Size
`size` is `"WIDTHxHEIGHT"` in pixels, or `"auto"` for the model's default.
A size that does not parse is a `400` saying so, rather than a silent
fall-back: `"1024"` used to generate a square nobody asked for, and the
response looked like a success. A side that is not a multiple of 8 is snapped
to the nearest one and logged, because the VAE downsamples by 8 and cannot
represent an odd side. A size above the install's ceiling is a `400` naming the
limit — `4096x4096` used to be accepted and come back as a CUDA OOM, which
reads like a busy GPU rather than a size the caller chose.
The ceiling is `image.max_side` (default 4096) and `image.max_pixels` (default
4.2 MP, which covers 2048×2048 and 2560×1440). Raise both if the card can take
it.
On `/v1/images/edits` and `/v1/images/inpaint`, omitting `size` keeps the
source photo's own aspect at about 1 MP on a 64-pixel grid. Both used to
default to a square, which squashed every non-square photo — and on inpaint it
pulled the mask out of alignment with the image as well.
## Learned prompt tokens (textual inversion)
A textual inversion is a few vectors trained to sit in CLIP's embedding table
under a word of their own. Writing that word is how much of the ecosystem's
negative prompting is built — `easynegative`, `bad-hands-5`, `FastNegativeV2`
— and until the file is loaded the word is an ordinary unknown token, so the
prompt quietly asks for nothing.
Three ways to get one loaded, all of which can be combined:
* `embeddings: ["easynegative"]` on the request, for an ad-hoc one.
* `textual_inversion` on the model's config (admin → Models → Prompt
Handling), for the ones a model always wants.
* every file in `image.embeddings_dir`, discovered by filename — which is how
A1111 works, and what a prompt copied off a model card assumes.
A reference is a bare name (looked up in the directory), a path to a
`.safetensors` / `.pt` / `.bin`, or a HuggingFace repo id. `ref:token` renames
the trigger word: `my-emb.safetensors:myneg`.
A name the caller gave that cannot be loaded is a `400` — rendering without it
would look like a success. A file in the directory that fails to load is a line
in the log, because one bad file should not break every request.
Two details that decide whether this works at all, both handled:
* **SDXL keeps two text encoders**, so an SDXL-format embedding holds a
`clip_l` and a `clip_g` tensor that must go into the matching
tokenizer/encoder pair. Loading only the first gives the trigger word an
embedding in one encoder and nothing in the other.
* **A multi-vector embedding occupies several tokens**, registered as `tok`,
`tok_1`, `tok_2`… A prompt only ever writes `tok`, so it is expanded before
tokenizing. Without that, an eight-vector embedding contributes one vector
and appears to do nothing.
## What is not covered
Flux, SD3 and Z-Image carry a T5 encoder with a much longer limit and
......@@ -88,10 +145,6 @@ path — the weight grammar is a CLIP-era convention and a T5 reads it as text,
which is what those models expect. Negative prompts work there as the pipeline
supports them.
Textual-inversion embeddings (`easynegative` and friends) are not loaded by
name yet; a LoRA is the supported way to add learned tokens
(see [zimage-lora-training.md](zimage-lora-training.md)).
## Implementation
`codai/models/prompt_weighting.py` holds the parser, the windowing and the
......@@ -99,5 +152,12 @@ encoder, and documents how `clip_skip` maps to a hidden-state index — SDXL
takes the penultimate layer by default and counts from there, SD 1.x takes the
final output and needs the layer norm applied by hand. Getting that mapping
wrong shifts every image, so it mirrors diffusers exactly.
`codai/models/textual_inversion.py` resolves and loads the embeddings, and
tracks what a pipeline already carries — a pipeline outlives the request that
loaded it, and diffusers raises on registering the same trigger word twice.
`tests/test_prompt_weighting_and_clip_skip.py` pins the grammar table, the
window arithmetic, both layer conventions and the three defects above.
window arithmetic and both layer conventions;
`tests/test_image_size_and_embeddings.py` pins the size rules and the
embedding resolution, including the SDXL two-encoder split and the
multi-vector expansion.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment