fit the caption to the frame, install the font that draws it, and stop reading...

fit the caption to the frame, install the font that draws it, and stop reading an unset field as a free GPU (v0.5.6)

Four defects behind one report: vertical renders came back with caption lines
running off both edges, in the wrong typeface, and speech-to-text answered
500/503 while a text model held the card.

THE FONT WAS NEVER IN THE IMAGE. Dockerfile.oci and Dockerfile.update install
fonts-montserrat; Dockerfile.oci-venv -- the one build_oci_image.sh actually
builds -- installed no fonts at all, so no image built the normal way has ever
had the family the compose API documents and defaults to. libass does not fail
on a missing family, it substitutes: DejaVu is considerably wider, which is how
a missing package became a layout bug with nothing but a log line to say so.
All three Dockerfiles now install the fonts AND fontconfig (Dockerfile.oci had
the fonts but not fc-match, so it could not even tell whether a family
existed).

And because an image rebuild is a poor way to add a brand font, fonts can now
be supplied per installation: files dropped in <config>/fonts -- or wherever
CODERAI_FONT_DIR points -- are registered at startup by writing a fontconfig
snippet into the user config. The snippet matters more than it looks: captions
are burned by libass inside an ffmpeg CHILD process, which does its own
fontconfig lookup, so a font this process can find is not necessarily a font
the renderer can use. The config volume outlives both a restart and a new
image, which is the point.

CAPTION SIZE CAME FROM THE FRAME HEIGHT. On a vertical video the height is the
LONG side: at 1080x1920 the font was sized 0.052 * 1920 ~ 100px to be drawn
across a 1080px frame. It is now sized from the short side, which leaves 16:9
byte-identical (there the short side IS the height) and gives 9:16 the same
text-to-width ratio as 16:9. Line breaking upstream works on a word budget and
cannot know how wide the words are, and WrapStyle is 2 -- no automatic wrapping
-- so a line that still overflows was simply drawn past the edge. Each event's
widest line is now measured against the real font file, and one that does not
fit gets its own smaller size through an inline \fs override: the long line
shrinks, not the whole video. Measured with the actual face rather than
estimated per character, because the families involved differ in width by
exactly the margin that decides the question. Clients scaling font_scale down
by hand for 9:16 and 4:5 can stop.

AN UNSET FIELD READ AS A FREE GPU. used_vram_gb=0 was honoured as authoritative
-- yet zero is this estimator's own "could not determine" sentinel, and it is
what the admin UI stores for a field the operator never filled in. A 27B
Q4_K_P with used_vram_gb=0 and measured_vram_gb=18.0 therefore estimated 0.00
GB against 8852 MiB of free VRAM. Nothing was evicted, every layer was aimed at
the GPU, the load failed and retried down a ladder -- all, 52, 39, 26 layers,
re-reading the weights at each rung -- for 11m50s, and settled with 39 of 66
layers on the CPU. The request then generated for another 19 minutes, 31m04s in
all, holding GPU ownership the whole time: every other model queued behind it,
which is what made speech-to-text answer 500/503. The swap gate was working
correctly; it swapped the moment the owner finished.

A real measurement is better information than an unfilled field, so it now
wins. An operator who means "no weights on the GPU" has n_gpu_layers=0, which
is handled separately and still reserves nothing.

Worth recording for the next time this looks like a quantization problem: it
was not. Those weights are 4-bit on disk and the KV cache was already
q4_0/q4_0 with flash attention -- about 2.2 GB for n_ctx=32000. Both were at
their most aggressive setting while the model sat on the CPU.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.5.5"
+__version__ = "0.5.6"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/app.py

@@ -126,6 +126,18 @@ async def lifespan(app: FastAPI):
     except Exception as exc:
         print(f"[threads] could not size the default pool: {exc}", flush=True)
 
+    # Operator-supplied caption fonts, before any render can ask for one. This
+    # writes a fontconfig snippet into the user config, so the ffmpeg/libass
+    # SUBPROCESS that burns captions resolves the same families this process
+    # does -- a font found only here would still be substituted there.
+    try:
+        from codai.compose.media import register_operator_fonts
+        _fdirs = register_operator_fonts()
+        if _fdirs:
+            boot_phase(f"fonts: registered {', '.join(_fdirs)}")
+    except Exception as exc:
+        print(f"[fonts] could not register operator fonts: {exc}", flush=True)
+
     # And then watch whether that loop is actually free. A synchronous call made
     # from a coroutine stops every request in the process, which from outside
     # looks like an overloaded upstream rather than a server that is not

codai/compose/captions.py

@@ -230,6 +230,82 @@ def _ass_time(t: float) -> str:
     return f"{h:d}:{m:02d}:{s:05.2f}"
 
 
+_FACE_CACHE: dict = {}
+
+
+def _loaded_font(font: str, bold: bool, size: int):
+    """A PIL face for (family, weight, size), parsed once.
+
+    Measuring is per EVENT -- a few hundred times in a long video -- and
+    ``truetype()`` re-reads and re-parses the file on every call.
+    """
+    key = (font or "", bool(bold), int(size))
+    if key in _FACE_CACHE:
+        return _FACE_CACHE[key]
+    face = None
+    try:
+        from PIL import ImageFont
+        from codai.compose.media import font_file
+        path = font_file(font, bold=bold)
+        if path:
+            face = ImageFont.truetype(path, size)
+    except Exception:
+        face = None
+    _FACE_CACHE[key] = face
+    return face
+
+
+def _text_width(text: str, size: int, font: str, bold: bool, spacing: float) -> Optional[float]:
+    """Rendered width of one line in pixels, or None if it cannot be measured.
+
+    Measured with the real font file rather than estimated from the character
+    count: the whole point is to catch a line that does not fit, and glyph
+    advances differ enough between families (Montserrat vs its DejaVu
+    substitute) that a per-character guess is wrong in exactly the case that
+    matters. ASS ``Spacing`` is extra advance per character, so it is added
+    here the way libass adds it.
+    """
+    if not text:
+        return 0.0
+    try:
+        fnt = _loaded_font(font, bold, size)
+        if fnt is None:
+            return None
+        return float(fnt.getlength(text)) + float(spacing or 0.0) * max(0, len(text) - 1)
+    except Exception:
+        return None
+
+
+def _fit_size(rows, size: int, usable: float, font: str, bold: bool,
+              spacing: float, upper: bool) -> int:
+    """The largest size <= ``size`` at which every row fits in ``usable`` px.
+
+    Line breaking happens upstream, on a word budget, which cannot know how
+    wide the words actually are -- a line of five long words in a wide font
+    overflows a frame that five short ones fit. WrapStyle is 2 (no automatic
+    wrapping), so an overflowing line is simply drawn past the frame edge
+    instead of being folded. Shrinking that one event is the least surprising
+    repair: the caption stays on one line, where the caller put it.
+    """
+    if usable <= 0 or not rows:
+        return size
+    widest = 0.0
+    for row in rows:
+        text = " ".join(row)
+        if upper:
+            text = text.upper()
+        w = _text_width(text, size, font, bold, spacing)
+        if w is None:
+            return size                      # cannot measure -> do not guess
+        widest = max(widest, w)
+    if widest <= usable:
+        return size
+    # Advance scales linearly with size, so one ratio gets there; floor() and a
+    # 1px margin keep a rounding error from leaving it a hair too wide.
+    scaled = int(size * usable / widest)
+    return max(12, min(size, scaled))
+
+
 def _ass_escape(text: str) -> str:
     return (text or "").replace("\\", "\\\\").replace("{", "\\{").replace("}", "\\}")
 
@@ -245,11 +321,18 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts
     at 1080x1920 and 720x1280; margins keep every line inside the safe area."""
     p = PRESETS.get((preset or "karaoke").lower(), PRESETS["karaoke"])
     upper = bool(uppercase) or p["upper"]
-    base = height * 0.052 * p["scale"] * float(font_scale or 1.0)
+    # Sized from the SHORT side of the frame, not its height. Caption text is
+    # limited by the width it has to fit across, and on a vertical video the
+    # height is the long side: at 1080x1920, height*0.052 gave a 100px font in
+    # a 1080px-wide frame, so lines ran off both edges. min() makes a landscape
+    # frame size exactly as before (there the short side IS the height) and
+    # brings 9:16 back to the same text-to-width ratio as 16:9.
+    base = min(int(width), int(height)) * 0.052 * p["scale"] * float(font_scale or 1.0)
     size = max(12, int(round(base)))
     align = _ALIGN.get((position or "center").lower(), 5)
     margin_v = int(round(height * 0.10))
     margin_h = int(round(width * 0.06))
+    usable_w = int(width) - 2 * margin_h
     primary = _ass_colour(colour)
     hi = _ass_colour(highlight)
     outline = _ass_colour(outline_colour)
@@ -282,11 +365,17 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts
         # A scene with a presenter bubble at the bottom pushes its captions up:
         # ASS takes a per-event MarginV, so only those events move.
         mv = int(ev.get("margin_v") or 0)
+        # Does this event's widest line actually fit? If not, it gets its own
+        # smaller size through an inline \fs override -- per event, so one long
+        # line never shrinks the whole video.
+        ev_size = _fit_size(rows, size, usable_w, font or "DejaVu Sans",
+                            bool(p["bold"]), p["spacing"], upper)
+        fs = f"{{\\fs{ev_size}}}" if ev_size != size else ""
         if not p["karaoke"]:
             text = "\\N".join(" ".join(_ass_escape(w.upper() if upper else w) for w in row)
                               for row in rows)
             body.append(f"Dialogue: 0,{_ass_time(ev['start'])},{_ass_time(ev['end'])},"
-                        f"cap,,0,0,{mv},,{text}")
+                        f"cap,,0,0,{mv},,{fs}{text}")
             continue
         # Karaoke: one event per word, the whole line drawn each time with the
         # active word in the highlight colour. Simpler than \k timings and it
@@ -308,7 +397,7 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts
             if end <= start:
                 end = start + 0.04
             body.append(f"Dialogue: 0,{_ass_time(start)},{_ass_time(end)},"
-                        f"cap,,0,0,{mv},,{{\\c{primary}}}" + "\\N".join(parts))
+                        f"cap,,0,0,{mv},,{fs}{{\\c{primary}}}" + "\\N".join(parts))
     return "\n".join(head + body) + "\n"
 
 

codai/compose/media.py

@@ -424,6 +424,93 @@ def font_file(name: str = "", bold: bool = True) -> Optional[str]:
     return out
 
 
+_FONT_EXTS = (".ttf", ".otf", ".ttc", ".pfb", ".woff", ".woff2")
+_FONTS_REGISTERED = False
+
+
+def operator_font_dirs() -> list:
+    """Directories an operator may drop caption fonts into, highest priority first.
+
+    ``CODERAI_FONT_DIR`` lets a deployment point somewhere else entirely (a
+    shared asset mount, say); otherwise it is ``<config>/fonts``, which rides
+    the config volume and so survives both a restart and a new image.
+    """
+    out = []
+    env = os.environ.get("CODERAI_FONT_DIR")
+    if env:
+        out.extend(d for d in env.split(os.pathsep) if d.strip())
+    try:
+        from codai.platform_paths import default_fonts_dir
+        out.append(str(default_fonts_dir()))
+    except Exception:
+        pass
+    seen, uniq = set(), []
+    for d in out:
+        d = os.path.abspath(os.path.expanduser(d))
+        if d not in seen:
+            seen.add(d)
+            uniq.append(d)
+    return uniq
+
+
+def register_operator_fonts(force: bool = False) -> list:
+    """Make operator-supplied fonts visible to fontconfig, and so to libass.
+
+    Captions are burned by ffmpeg's libass, a SUBPROCESS that does its own
+    fontconfig lookup -- so it is not enough for this process to find the file.
+    The directory has to be in fontconfig's own configuration, and the one
+    place a non-root process can put it that a child also reads is the user
+    config: ``$XDG_CONFIG_HOME/fontconfig/conf.d``, inherited through the
+    environment. (Writing to /usr/local/share/fonts would be cleaner and is
+    not available: the container runs unprivileged.)
+
+    Returns the directories registered. Idempotent, and cheap enough to call at
+    every startup: it rewrites the snippet only when the content changes, since
+    fc-cache on a large font tree is not free.
+    """
+    global _FONTS_REGISTERED
+    if _FONTS_REGISTERED and not force:
+        return []
+    dirs = [d for d in operator_font_dirs() if os.path.isdir(d)]
+    have = [d for d in dirs
+            if any(f.lower().endswith(_FONT_EXTS) for f in os.listdir(d))]
+    _FONTS_REGISTERED = True
+    if not have:
+        return []
+    xdg = os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config")
+    conf_d = os.path.join(xdg, "fontconfig", "conf.d")
+    body = ("<?xml version=\"1.0\"?>\n"
+            "<!DOCTYPE fontconfig SYSTEM \"urn:fontconfig:fonts.dtd\">\n"
+            "<!-- written by coderai; operator caption fonts -->\n"
+            "<fontconfig>\n"
+            + "".join(f"  <dir>{d}</dir>\n" for d in have)
+            + "</fontconfig>\n")
+    conf = os.path.join(conf_d, "99-coderai-fonts.conf")
+    try:
+        os.makedirs(conf_d, exist_ok=True)
+        try:
+            unchanged = open(conf, encoding="utf-8").read() == body
+        except Exception:
+            unchanged = False
+        if not unchanged:
+            with open(conf, "w", encoding="utf-8") as fh:
+                fh.write(body)
+        fccache = shutil.which("fc-cache")
+        if fccache and not unchanged:
+            subprocess.run([fccache, "-f"] + have,
+                           capture_output=True, timeout=180)
+        _FONT_CACHE.clear()          # a new font changes what was resolved
+        try:                         # ...and what the caption measurer loaded
+            from codai.compose.captions import _FACE_CACHE
+            _FACE_CACHE.clear()
+        except Exception:
+            pass
+    except Exception as exc:
+        print(f"[fonts] could not register {have}: {exc}")
+        return []
+    return have
+
+
 def font_installed(name: str) -> bool:
     """Is this exact family available? (A caption font that is not gets a warning.)"""
     if not name:

codai/models/manager.py

@@ -4520,9 +4520,33 @@ class MultiModelManager:
         # Config-declared LoRA(s) are added on top — they load into VRAM beyond
         # the base weights. (Not added to forced/measured: forced is the user's
         # final number; a measurement already reflects any load-time adapters.)
-        if explicit is not None:
+        try:
+            _explicit_val = float(explicit) if explicit is not None else None
+        except (TypeError, ValueError):
+            _explicit_val = None
+        if _explicit_val is not None and _explicit_val > 0:
             return _dbg_est("used_vram_gb",
-                            _apply_factors(float(explicit)) + self._config_lora_vram_gb(cfg))
+                            _apply_factors(_explicit_val) + self._config_lora_vram_gb(cfg))
+        if _explicit_val is not None and _explicit_val <= 0 and (measured or 0) > 0:
+            # A configured ZERO is not a declaration that the model needs no
+            # VRAM -- zero is this function's own "could not determine" sentinel
+            # (see the docstring), and it is what the admin UI stores for a field
+            # the operator never filled in. Honouring it as authoritative told
+            # the planner a model was free, so nothing was evicted for it and it
+            # was loaded with every layer aimed at a GPU that did not have room.
+            #
+            # Measured on zeiss, 2026-10-26: a 27B Q4_K_P with used_vram_gb=0 and
+            # measured_vram_gb=18.0 estimated 0.00 GB against 8.6 GB free. The
+            # load failed and retried down a ladder -- all -> 52 -> 39 -> 26
+            # layers, each rung re-reading the weights -- for 11m50s, landed with
+            # 39 of 66 layers on the CPU, and then generated so slowly that the
+            # request outlived its client while still owning the GPU, queueing
+            # every other model behind it.
+            #
+            # A real measurement is strictly better information than an unset
+            # field, so it wins. An operator who truly wants no reservation has
+            # n_gpu_layers=0, which is handled above and means it.
+            return _dbg_est("measured-over-zero-config", float(measured))
 
         # No configured used_vram_gb → the real measurement is GROUND TRUTH.
         if measured:

codai/platform_paths.py

@@ -136,6 +136,20 @@ def default_loras_dir() -> Path:
     return ensure_dir(legacy_style_config_dir() / "loras")
 
 
+def default_fonts_dir() -> Path:
+    """Where an operator drops font files for captions and overlays.
+
+    Beside voices and characters, because it is the same kind of thing: content
+    the installation supplies, not something the image can know in advance. A
+    caption font is a hard requirement of a render -- libass substitutes
+    silently when a family is missing, and a wider substitute pushes text out
+    of frame -- so an operator must be able to add one WITHOUT rebuilding the
+    image. This directory is on the config volume, so what lands here survives
+    a restart and a new image.
+    """
+    return ensure_dir(legacy_style_config_dir() / "fonts")
+
+
 def default_whisper_server_path() -> str:
     if os.name == "nt":
         local = _windows_dir("LOCALAPPDATA", _home_dir() / "AppData" / "Local")

docs/video-compose.md

@@ -297,7 +297,28 @@ claimed.
   captions). Without it every compose request answers `501`.
 * **Fonts**: captions ask for a family by name. The images ship Montserrat,
   Inter, DejaVu and Noto (including CJK); a family that is not installed falls
-  back to the closest one and adds a warning, it does not fail.
+  back to the closest one and adds a warning, it does not fail. Treat that
+  warning as an error in disguise when the layout matters: the substitute is a
+  different width, and a wider one pushes lines towards the frame edge.
+
+  **Adding your own**: drop font files (`.ttf`, `.otf`, `.ttc`) into
+  `<config>/fonts` — `~/.coderai/fonts`, or `/config/coderai/fonts` in the
+  images — and restart. They are registered with fontconfig at startup, so the
+  ffmpeg/libass child that burns the captions resolves them too, and because
+  the directory lives on the config volume they survive a restart and a new
+  image. `CODERAI_FONT_DIR` points somewhere else (several paths, `:`-separated).
+  Check with `fc-match -f '%{family}\n' 'Your Family:style=Bold'` inside the
+  container: it must echo the family you asked for.
+
+* **Caption size** is derived from the SHORT side of the frame, times `0.052`,
+  times the preset's scale, times `captions.font_scale`. A 16:9 frame is
+  unchanged by this (there the short side is the height); a vertical frame is
+  sized as if it were that square, rather than from its 1920px height, which
+  used to put a ~100px font in a 1080px-wide frame. On top of that, any single
+  line still too wide for the text area (`width - 2 * 6%`) is measured against
+  the real font file and given its own smaller size, so one long line shrinks
+  instead of running off the edge. Clients that were scaling `font_scale` down
+  by hand for 9:16 or 4:5 should drop that factor.
 * **Where the files go**: the ordinary output directory (`--file-path`), served
   by `/v1/files/`, under normal retention. A composed video is typically read
   once by the client and copied to its own storage.

packaging/linux/Dockerfile.oci

@@ -178,6 +178,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
       libvulkan1 \
       lm-sensors \
       espeak \
+      fontconfig \
       fonts-montserrat \
       fonts-inter \
       fonts-noto-core \

packaging/linux/Dockerfile.oci-venv

@@ -116,10 +116,24 @@ ENV DEBIAN_FRONTEND=noninteractive \
 #   variable or set triton.knobs.build.impl.
 # which surfaces only as "vLLM exited (code 1) before becoming ready" — so every
 # vLLM engine, surya OCR included, silently fell back to another engine.
+# Caption fonts are a RUNTIME dependency of the compose renderer, not a nicety:
+# libass resolves a family name through fontconfig, and when the family is
+# missing it silently substitutes. DejaVu is much wider than Montserrat, so a
+# 9:16 render came back with caption lines running past the frame edges and
+# only a log line to say why. Dockerfile.oci and Dockerfile.update carried
+# these packages; THIS file -- the one build_oci_image.sh actually builds --
+# never did, so every image built the normal way shipped without them.
 RUN apt-get update && apt-get install -y --no-install-recommends \
       ca-certificates \
       curl \
       ffmpeg \
+      fontconfig \
+      fonts-montserrat \
+      fonts-inter \
+      fonts-noto-core \
+      fonts-noto-cjk \
+      fonts-liberation \
+      fonts-dejavu-core \
       gcc \
       libc6-dev \
       git \

packaging/linux/Dockerfile.update

@@ -70,6 +70,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends openssh-client
 # makes a non-Latin script render as glyphs instead of boxes. The base image
 # carried DejaVu only, so a request for "Montserrat" silently got DejaVu.
 RUN apt-get update && apt-get install -y --no-install-recommends \
+        fontconfig \
         fonts-montserrat fonts-inter fonts-noto-core fonts-noto-cjk fonts-liberation \
     && fc-cache -f >/dev/null 2>&1 \
     && rm -rf /var/lib/apt/lists/* || true