14c + 14d: wrap text instead of shrinking it, and let a clip's sound come from its own file

14c. A text overlay was one drawtext line sized from the frame height, so a
long text ran off a vertical frame, and a pre-wrapped text was drawn as a
single line because an explicit newline was never a break. `overlays[].fit`
takes none (the default, today's behaviour) | wrap | shrink, and a newline now
breaks whatever the fit. Wrapping emits one drawtext per line, placed as a
single left-aligned block measured from the widest line, rather than letting
each line centre itself ragged. `captions.fit` takes shrink (the default) |
wrap, folding a too-wide row's trailing words onto a new row instead of
shrinking that event -- on a vertical video shrinking can go below readable.
Word order is preserved, so the karaoke highlight still walks the words in the
order they are spoken.

Both fitters now MEASURE the result instead of scaling the size by the width
ratio. Advance is very nearly linear in size, which is why the shortcut looks
right, but hinting rounds each glyph's advance to whole pixels and the error
compounds over a long string: a 95-character line computed to fit at size 8
measured 440 px against a 300 px budget. A test caught it. _fit_size carried
the same assumption and shrink is its default, so it is fixed the same way.

14d. `visuals[].audio_src` names a separate audio-only file -- DASH sites serve
picture and sound as separate streams, so the best video of a clip often has no
audio track and a caller that cannot mux had to skip it. The picture path is
deliberately left alone: stage 3 reuses it as the visual's video, and an
audio-only file there would render a scene with no image. Lengths in
audio_mix="clip" come from whatever carries the sound.

Also: the dialogue tests no longer depend on a TTS model being installed. They
test how a dialogue scene is VALIDATED and say so -- their portraits are inline
"so the test does not depend on what is saved on this install" -- but three of
them failed standalone with 501 "narration needs a TTS model" without reaching
the validation they exercise. Only the inventory is faked, not the check.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
parent 6729e974
...@@ -111,6 +111,14 @@ class Visual(BaseModel): ...@@ -111,6 +111,14 @@ class Visual(BaseModel):
# have none, and for a visual that is not the one carrying the scene (only # have none, and for a visual that is not the one carrying the scene (only
# the first audible video in a scene is taken; see _scene_clip_audio). # the first audible video in a scene is taken; see _scene_clip_audio).
keep_audio: bool = False keep_audio: bool = False
# This visual's sound, when it lives in a SEPARATE file. YouTube and other
# DASH sites serve picture and sound as separate streams, so the best
# available video of a clip often has no audio track at all; a caller that
# does not run ffmpeg itself cannot mux them and would have to skip the
# clip. Given here, this file supplies the sound wherever `keep_audio`
# applies, cut with the same `trim_start`, and `src` is left to carry only
# the picture (it is not required to have an audio stream).
audio_src: Optional[str] = None
# How loud this clip sits when it is mixed UNDER the narration # How loud this clip sits when it is mixed UNDER the narration
# (``audio_mix: "both"``). A per-visual override of the scene's # (``audio_mix: "both"``). A per-visual override of the scene's
# ``clip_volume``. # ``clip_volume``.
...@@ -216,6 +224,12 @@ class CaptionSpec(BaseModel): ...@@ -216,6 +224,12 @@ class CaptionSpec(BaseModel):
max_words_per_line: int = 4 max_words_per_line: int = 4
max_lines: int = 2 max_lines: int = 2
uppercase: bool = False uppercase: bool = False
# What happens to a row wider than the frame. "shrink" (the default, and
# what callers already get) reduces that event's size until it fits;
# "wrap" keeps the size and folds the row's trailing words onto a new one,
# which may use more rows than max_lines. On a vertical video shrinking can
# take a caption below readable, so wrap is the better repair there.
fit: str = "shrink" # shrink | wrap
model_config = ConfigDict(extra="allow") model_config = ConfigDict(extra="allow")
...@@ -229,6 +243,12 @@ class Overlay(BaseModel): ...@@ -229,6 +243,12 @@ class Overlay(BaseModel):
width: float = 0.15 # image width as a fraction of the canvas width: float = 0.15 # image width as a fraction of the canvas
color: str = "#FFFFFF" color: str = "#FFFFFF"
font: Optional[str] = None font: Optional[str] = None
# What happens to a text wider than the frame. "none" (the default, and
# what callers already get) draws one line at the size asked for, off the
# edge if it is too long; "wrap" keeps the size and adds lines; "shrink"
# keeps one line and reduces the size. A newline in `text` always breaks,
# whatever this is set to.
fit: str = "none" # none | wrap | shrink
model_config = ConfigDict(extra="allow") model_config = ConfigDict(extra="allow")
...@@ -1134,9 +1154,28 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None: ...@@ -1134,9 +1154,28 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None:
try: try:
_cp = media.resolve(getattr(cv, "src", None) or "", work, _cp = media.resolve(getattr(cv, "src", None) or "", work,
f"s{i}-clipaudio") f"s{i}-clipaudio")
if not rnd.has_audio_stream(_cp): # `path` stays the PICTURE, because stage 3 reuses it as
raise media.MediaError("the clip has no audio stream") # this visual's video and an audio-only file there would
# render no image at all. The sound is tracked separately
# so `audio_src` can point somewhere else entirely.
_asrc = str(getattr(cv, "audio_src", None) or "").strip()
if _asrc:
_ap = media.resolve(_asrc, work, f"s{i}-clipaudio-a")
else:
_ap = _cp
if not rnd.has_audio_stream(_ap):
raise media.MediaError(
"the separate audio file has no audio stream" if _asrc
else "the clip has no audio stream")
clip_media[i] = {"path": _cp, "probe": media.probe(_cp), clip_media[i] = {"path": _cp, "probe": media.probe(_cp),
"audio_path": _ap,
# Timed from whatever carries the SOUND:
# with audio_mix="clip" the scene is as
# long as what you can hear, and a
# video-only stream may differ in length
# from its audio sibling.
"audio_probe": (media.probe(_ap) if _asrc
else None),
"src": getattr(cv, "src", None) or "", "src": getattr(cv, "src", None) or "",
"trim_start": max(0.0, float( "trim_start": max(0.0, float(
getattr(cv, "trim_start", 0.0) or 0.0)), getattr(cv, "trim_start", 0.0) or 0.0)),
...@@ -1230,7 +1269,11 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None: ...@@ -1230,7 +1269,11 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None:
# narration's length, truncates the content itself. An explicit # narration's length, truncates the content itself. An explicit
# `duration` still wins: a caller who names one means it. # `duration` still wins: a caller who names one means it.
if mix == "clip" and not sc.duration: if mix == "clip" and not sc.duration:
_cd = float((clip_media.get(i) or {}).get("probe", {}).get("duration") or 0.0) _cm = clip_media.get(i) or {}
# The SOUND decides the length here, and with a separate
# audio_src that is a different file from the picture.
_cdp = _cm.get("audio_probe") or _cm.get("probe") or {}
_cd = float(_cdp.get("duration") or 0.0)
_cd -= float((clip_media.get(i) or {}).get("trim_start") or 0.0) _cd -= float((clip_media.get(i) or {}).get("trim_start") or 0.0)
if _cd > 0: if _cd > 0:
dur = round(_cd, 3) dur = round(_cd, 3)
...@@ -1592,7 +1635,7 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None: ...@@ -1592,7 +1635,7 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None:
# shorter than its slot was looped there, so its sound is # shorter than its slot was looped there, so its sound is
# looped too or it would fall silent halfway through the shot # looped too or it would fall silent halfway through the shot
# it belongs to. # it belongs to.
_cdur = float((cm.get("probe") or {}).get("duration") or 0.0) _cdur = float((cm.get("audio_probe") or cm.get("probe") or {}).get("duration") or 0.0)
_trim = float(cm.get("trim_start") or 0.0) _trim = float(cm.get("trim_start") or 0.0)
_loop = bool(_cdur and (_cdur - _trim) < scene_dur[i] - 0.02) _loop = bool(_cdur and (_cdur - _trim) < scene_dur[i] - 0.02)
_vol = cm.get("volume") _vol = cm.get("volume")
...@@ -1601,7 +1644,8 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None: ...@@ -1601,7 +1644,8 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None:
getattr(sc, "clip_volume", 0.35) or 0.35) getattr(sc, "clip_volume", 0.35) or 0.35)
try: try:
ctrack = rnd.clip_audio( ctrack = rnd.clip_audio(
cm["path"], os.path.join(work, f"clipaud-{i}.wav"), cm.get("audio_path") or cm["path"],
os.path.join(work, f"clipaud-{i}.wav"),
duration=scene_dur[i], trim_start=_trim, duration=scene_dur[i], trim_start=_trim,
volume=float(_vol), loop=_loop, cancel=cancel, log=log) volume=float(_vol), loop=_loop, cancel=cancel, log=log)
clip_used = True clip_used = True
...@@ -1676,7 +1720,7 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None: ...@@ -1676,7 +1720,7 @@ def _run_job(job_id: str, req: ComposeRequest, loop, base_url: str) -> None:
font=caps.font, font_scale=caps.font_scale, colour=caps.color, font=caps.font, font_scale=caps.font_scale, colour=caps.color,
highlight=caps.highlight_color, outline_colour=caps.outline_color, highlight=caps.highlight_color, outline_colour=caps.outline_color,
position=caps.position, preset=caps.preset, position=caps.position, preset=caps.preset,
uppercase=caps.uppercase)) uppercase=caps.uppercase, fit=caps.fit))
overlays = [] overlays = []
for k, o in enumerate(req.overlays or []): for k, o in enumerate(req.overlays or []):
spec = o.model_dump() spec = o.model_dump()
......
...@@ -323,10 +323,75 @@ def _fit_size(rows, size: int, usable: float, font: str, bold: bool, ...@@ -323,10 +323,75 @@ def _fit_size(rows, size: int, usable: float, font: str, bold: bool,
widest = max(widest, w) widest = max(widest, w)
if widest <= usable: if widest <= usable:
return size return size
# Advance scales linearly with size, so one ratio gets there; floor() and a
# 1px margin keep a rounding error from leaving it a hair too wide. def _widest_at(sz: int):
scaled = int(size * usable / widest) w = 0.0
return max(12, min(size, scaled)) for row in rows:
t = " ".join(row)
if upper:
t = t.upper()
m = _text_width(t, sz, font, bold, spacing)
if m is None:
return None
w = max(w, m)
return w
# Found by measuring, not by scaling the size by the width ratio. Advance is
# very nearly linear in size, which is why that shortcut looks right, but
# hinting rounds each glyph's advance to whole pixels and the error
# compounds over a long line -- measured on a 95-character line, the size
# the ratio chose was still 47% over its budget. A shrunk caption that
# still overflows is the one outcome this function exists to prevent.
lo, hi, best = 12, int(size), None
while lo <= hi:
mid = (lo + hi) // 2
m = _widest_at(mid)
if m is None:
return size # cannot measure -> do not guess
if m <= usable:
best = mid
lo = mid + 1
else:
hi = mid - 1
return best if best is not None else 12
def _wrap_rows(rows, size: int, usable: float, font: str, bold: bool,
spacing: float, upper: bool):
"""Re-flow ``rows`` so every row fits ``usable`` px at ``size``.
The alternative repair is ``_fit_size``, which keeps the caller's rows and
shrinks the text. On a vertical video that can take a caption below the
size anyone wants to read, so ``captions.fit="wrap"`` moves a too-wide
row's trailing words onto a new row instead and leaves the size alone. The
event may end up with more rows than ``max_lines`` asked for: being one row
taller is a smaller harm than being unreadable, which is the whole point of
asking for wrap.
Word order is never changed, so the karaoke highlight still walks the words
in the order they are spoken.
"""
if usable <= 0 or not rows:
return rows
out = []
for row in rows:
cur = []
for word in row:
cand = cur + [word]
text = " ".join(cand)
if upper:
text = text.upper()
w = _text_width(text, size, font, bold, spacing)
if w is None:
return rows # cannot measure -> do not guess
if cur and w > usable:
out.append(cur)
cur = [word]
else:
cur = cand
if cur:
out.append(cur)
return out or rows
def _ass_escape(text: str) -> str: def _ass_escape(text: str) -> str:
...@@ -337,7 +402,7 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts ...@@ -337,7 +402,7 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts
font_scale: float = 1.0, colour: str = "#FFFFFF", font_scale: float = 1.0, colour: str = "#FFFFFF",
highlight: str = "#FFE14D", outline_colour: str = "#000000", highlight: str = "#FFE14D", outline_colour: str = "#000000",
position: str = "center", preset: str = "karaoke", position: str = "center", preset: str = "karaoke",
uppercase: bool = False) -> str: uppercase: bool = False, fit: str = "shrink") -> str:
"""The ASS subtitle file the renderer burns in. """The ASS subtitle file the renderer burns in.
Sizes are derived from the canvas height so the same request looks the same Sizes are derived from the canvas height so the same request looks the same
...@@ -388,11 +453,19 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts ...@@ -388,11 +453,19 @@ def build_ass(events: List[dict], *, width: int, height: int, font: str = "Monts
# A scene with a presenter bubble at the bottom pushes its captions up: # A scene with a presenter bubble at the bottom pushes its captions up:
# ASS takes a per-event MarginV, so only those events move. # ASS takes a per-event MarginV, so only those events move.
mv = int(ev.get("margin_v") or 0) mv = int(ev.get("margin_v") or 0)
# Does this event's widest line actually fit? If not, it gets its own # Does this event's widest line actually fit? If not, `fit` decides the
# smaller size through an inline \fs override -- per event, so one long # repair: "shrink" (the default, and what callers already get) gives
# line never shrinks the whole video. # this event its own smaller size through an inline \fs override, per
ev_size = _fit_size(rows, size, usable_w, font or "DejaVu Sans", # event so one long line never shrinks the whole video; "wrap" keeps the
bool(p["bold"]), p["spacing"], upper) # size and folds the row instead, which is what a vertical video wants
# when shrinking would take the text below readable.
if str(fit or "shrink").strip().lower() == "wrap":
rows = _wrap_rows(rows, size, usable_w, font or "DejaVu Sans",
bool(p["bold"]), p["spacing"], upper)
ev_size = size
else:
ev_size = _fit_size(rows, size, usable_w, font or "DejaVu Sans",
bool(p["bold"]), p["spacing"], upper)
fs = f"{{\\fs{ev_size}}}" if ev_size != size else "" fs = f"{{\\fs{ev_size}}}" if ev_size != size else ""
if not p["karaoke"]: if not p["karaoke"]:
text = "\\N".join(" ".join(_ass_escape(w.upper() if upper else w) for w in row) text = "\\N".join(" ".join(_ass_escape(w.upper() if upper else w) for w in row)
......
...@@ -808,6 +808,125 @@ def _esc_text(text: str) -> str: ...@@ -808,6 +808,125 @@ def _esc_text(text: str) -> str:
.replace("]", "\\]").replace(";", "\\;")) .replace("]", "\\]").replace(";", "\\;"))
#: Where a long word may be split when it alone is wider than a line. A URL is
#: the case that actually turns up in an overlay, and breaking it before a
#: separator keeps it readable instead of chopping it mid-token.
_BREAK_BEFORE = "?&/#=-_."
def _measure(text: str, size: int, font: str, bold: bool):
"""Width of ``text`` in px, or None when the font cannot be measured."""
try:
from codai.compose.captions import _text_width
return _text_width(text, size, font, bold, 0.0)
except Exception:
return None
def _split_long_word(word: str, size: int, font: str, bold: bool,
usable: float) -> List[str]:
"""One word that does not fit on a line of its own, cut into pieces."""
out, cur = [], ""
for i, ch in enumerate(word):
# Prefer to start a new piece BEFORE a separator, so a URL breaks at a
# boundary a reader recognises rather than mid-token.
if cur and ch in _BREAK_BEFORE:
w = _measure(cur + ch, size, font, bold)
if w is not None and w > usable:
out.append(cur)
cur = ch
continue
w = _measure(cur + ch, size, font, bold)
if cur and w is not None and w > usable:
out.append(cur)
cur = ch
else:
cur += ch
if cur:
out.append(cur)
return out or [word]
def wrap_text(text: str, size: int, font: str, bold: bool,
usable: float) -> List[str]:
"""``text`` broken into lines that each fit ``usable`` px at ``size``.
An explicit newline in ``text`` is always a line break -- a caller that has
already decided where the breaks go is not second-guessed, whatever fit was
asked for. Within a paragraph, words are moved to the next line as needed,
and a single word wider than a line is cut (see ``_split_long_word``)
rather than drawn off the frame.
When the font cannot be measured the text is returned as its own lines
unchanged: a guess from character counts is wrong in exactly the case this
exists to catch, so the honest answer is to leave it alone.
"""
paragraphs = str(text or "").replace("\r\n", "\n").replace("\r", "\n").split("\n")
if usable <= 0:
return paragraphs
lines: List[str] = []
for para in paragraphs:
words = para.split()
if not words:
lines.append("")
continue
cur = ""
for word in words:
cand = f"{cur} {word}".strip()
w = _measure(cand, size, font, bold)
if w is None:
return paragraphs # cannot measure -> do not guess
if cur and w > usable:
lines.append(cur)
cur = word
ww = _measure(cur, size, font, bold)
if ww is not None and ww > usable:
pieces = _split_long_word(cur, size, font, bold, usable)
lines.extend(pieces[:-1])
cur = pieces[-1]
else:
cur = cand
if not cur:
continue
if cur:
ww = _measure(cur, size, font, bold)
if ww is not None and ww > usable:
lines.extend(_split_long_word(cur, size, font, bold, usable))
else:
lines.append(cur)
return lines or [""]
def shrink_to_fit(text: str, size: int, font: str, bold: bool,
usable: float, floor: int = 8) -> int:
"""The largest size <= ``size`` at which ``text`` fits on one line.
Found by measuring, not by scaling the size by the width ratio. Advance is
very nearly linear in size at caption sizes, which is why that shortcut
looks right, but hinting rounds each glyph's advance to whole pixels, and
the rounding error compounds over a long string: a 95-character line
computed to fit at size 8 measured 440 px against a 300 px budget. Below
the floor nothing fits, and the floor is returned as the best available.
"""
if usable <= 0 or not text:
return size
w = _measure(text, size, font, bold)
if w is None or w <= usable:
return size
lo, hi, best = floor, int(size), None
while lo <= hi:
mid = (lo + hi) // 2
m = _measure(text, mid, font, bold)
if m is None:
return size
if m <= usable:
best = mid
lo = mid + 1
else:
hi = mid - 1
return best if best is not None else floor
def final_render(video: str, audio: str, out_path: str, *, width: int, height: int, def final_render(video: str, audio: str, out_path: str, *, width: int, height: int,
fps: int, total: float, ass_file: Optional[str] = None, fps: int, total: float, ass_file: Optional[str] = None,
overlays: Optional[List[dict]] = None, crf: int = 20, overlays: Optional[List[dict]] = None, crf: int = 20,
...@@ -841,21 +960,72 @@ def final_render(video: str, audio: str, out_path: str, *, width: int, height: i ...@@ -841,21 +960,72 @@ def final_render(video: str, audio: str, out_path: str, *, width: int, height: i
for n, o in enumerate(overlays or []): for n, o in enumerate(overlays or []):
if (o.get("type") or "image").lower() != "text": if (o.get("type") or "image").lower() != "text":
continue continue
text = _esc_text(o.get("text") or "") raw = str(o.get("text") or "")
if not text: if not raw:
continue continue
size = max(8, int(height * max(0.008, min(0.2, float(o.get("size") or 0.025))))) size = max(8, int(height * max(0.008, min(0.2, float(o.get("size") or 0.025)))))
alpha = max(0.0, min(1.0, float(o.get("opacity") if o.get("opacity") is not None else 1.0))) alpha = max(0.0, min(1.0, float(o.get("opacity") if o.get("opacity") is not None else 1.0)))
colour = (o.get("color") or "#FFFFFF").lstrip("#") colour = (o.get("color") or "#FFFFFF").lstrip("#")
x, y = _pos_expr(o.get("position"), f"{int(width * 0.05)}", f"{int(height * 0.04)}") pad_x, pad_y = int(width * 0.05), int(height * 0.04)
ff = media.font_file(o.get("font") or "", bold=True) family = o.get("font") or ""
parts = [f"text='{text}'", f"fontsize={size}", ff = media.font_file(family, bold=True)
f"fontcolor=0x{colour}@{alpha:.3f}", f"x={x}", f"y={y}", # `fit` decides what happens to a text wider than the frame. The
"borderw=%d" % max(1, size // 14), "bordercolor=black@0.6"] # default stays "none" -- one line at the size asked for, drawn off the
if ff: # edge if it is too long -- because that is what callers already get.
parts.insert(0, f"fontfile={ff}") # "wrap" keeps the size and adds lines; "shrink" keeps one line and
graph.append(f"[{cur}]drawtext={':'.join(parts)}[txt{n}]") # reduces the size. An explicit newline always breaks, whatever is set,
cur = f"txt{n}" # so a caller that pre-wrapped its own text is honoured either way
# (before this, such a text was drawn as a single line).
fit = str(o.get("fit") or "none").strip().lower()
usable = max(1.0, float(width - 2 * pad_x))
if fit == "shrink":
size = shrink_to_fit(raw.replace("\n", " "), size, family, True, usable)
lines = [raw.replace("\n", " ")]
elif fit == "wrap":
lines = wrap_text(raw, size, family, True, usable)
else:
lines = str(raw).replace("\r\n", "\n").replace("\r", "\n").split("\n")
lines = [ln for ln in lines if ln != ""] or [""]
# One drawtext per line, placed as a single left-aligned block. Letting
# each line position itself would centre them independently and ragged;
# the block's own edge is measured from the widest line instead.
line_h = max(1, int(round(size * 1.25)))
block_h = line_h * len(lines)
widths = [_measure(ln, size, family, True) for ln in lines]
widest = max([w for w in widths if w is not None] or [0.0])
p = (o.get("position") or "bottom-right").lower().replace("_", "-")
if "-" in p:
v_side, _, h_side = p.partition("-")
else:
v_side, h_side = ("center", p) if p in ("left", "center", "centre", "right") \
else (p, "center")
for li, line in enumerate(lines):
text = _esc_text(line)
if widest and len(lines) > 1:
# A measured block: x is a number, identical for every line.
if h_side == "left":
x = f"{pad_x}"
elif h_side in ("center", "centre"):
x = f"(W-{widest:.0f})/2"
else:
x = f"W-{widest:.0f}-{pad_x}"
else:
x, _ = _pos_expr(o.get("position"), f"{pad_x}", f"{pad_y}")
if v_side == "top":
y = f"{pad_y}+{li * line_h}"
elif v_side in ("center", "centre", "middle"):
y = f"(H-{block_h})/2+{li * line_h}"
else:
y = f"H-{block_h}-{pad_y}+{li * line_h}"
parts = [f"text='{text}'", f"fontsize={size}",
f"fontcolor=0x{colour}@{alpha:.3f}", f"x={x}", f"y={y}",
"borderw=%d" % max(1, size // 14), "bordercolor=black@0.6"]
if ff:
parts.insert(0, f"fontfile={ff}")
lbl = f"txt{n}_{li}"
graph.append(f"[{cur}]drawtext={':'.join(parts)}[{lbl}]")
cur = lbl
graph.append(f"[{cur}]null[vout]") graph.append(f"[{cur}]null[vout]")
codec = {"h264": "libx264", "avc": "libx264", "h265": "libx265", codec = {"h264": "libx264", "avc": "libx264", "h265": "libx265",
......
...@@ -229,3 +229,61 @@ def test_mp3_is_actually_encoded_not_a_wav_under_an_mp3_key(monkeypatch): ...@@ -229,3 +229,61 @@ def test_mp3_is_actually_encoded_not_a_wav_under_an_mp3_key(monkeypatch):
import base64 as _b64 import base64 as _b64
assert set(out) == {"b64_mp3"} assert set(out) == {"b64_mp3"}
assert _b64.b64decode(out["b64_mp3"]) == b"ID3fake-mp3" assert _b64.b64decode(out["b64_mp3"]) == b"ID3fake-mp3"
# ------------------------------------- 14d: the sound in a separate file
#
# YouTube and other DASH sites serve picture and sound as separate streams, so
# the best available video of a clip often carries no audio track at all. A
# caller that does not run ffmpeg itself cannot mux them, and had to skip such
# a clip entirely. `Visual.audio_src` names the sound's own file.
def test_audio_src_defaults_to_nothing():
"""A request written before this existed must behave exactly as it did."""
v = Visual(type="video", src="a.mp4")
assert v.audio_src is None
def test_a_visual_with_a_separate_audio_file_is_still_the_clip_visual():
sc = Scene(audio_mix="clip", visuals=[
Visual(type="video", src="v.mp4", audio_src="a.m4a", keep_audio=True)])
v = _scene_clip_visual(sc)
assert v is not None and v.src == "v.mp4" and v.audio_src == "a.m4a"
@needs_ffmpeg
def test_a_video_with_no_audio_track_is_usable_when_audio_src_supplies_the_sound():
"""The whole point: `src` is not required to have an audio stream."""
from codai.compose import render as rnd
with tempfile.TemporaryDirectory() as d:
vid = os.path.join(d, "video-only.mp4")
aud = os.path.join(d, "audio-only.m4a")
_make_clip(vid, seconds=2.0, with_audio=False)
subprocess.run([_ffmpeg(), "-y", "-f", "lavfi", "-i",
"sine=frequency=330:duration=2.0", "-t", "2.0", aud],
capture_output=True, check=True, timeout=120)
assert not rnd.has_audio_stream(vid), "fixture is wrong: video has audio"
assert rnd.has_audio_stream(aud), "fixture is wrong: audio file has none"
# The sound is taken from the companion file, cut the same way.
out = os.path.join(d, "track.wav")
rnd.clip_audio(aud, out, duration=1.0, trim_start=0.5,
loop=False, volume=1.0)
assert os.path.getsize(out) > 44, "no PCM was written from audio_src"
def test_the_picture_path_is_never_replaced_by_the_audio_file():
"""Stage 3 reuses the resolved clip as this visual's VIDEO.
Putting the audio-only file in that slot would render a scene with no
image at all, so the two paths are tracked separately.
"""
import inspect
from codai.api import compose as C
src = inspect.getsource(C._run_job) if hasattr(C, "_run_job") else ""
if not src:
pytest.skip("cannot read the job source")
assert '"audio_path"' in src, (
"the clip's sound is not tracked apart from its picture")
# The visual cache must hand back `path`, not the audio file.
assert '_cached["path"]' in src, (
"the visual no longer reuses the resolved picture path")
...@@ -20,6 +20,28 @@ def _sp(**kw): ...@@ -20,6 +20,28 @@ def _sp(**kw):
return SpeakerLine(**kw) return SpeakerLine(**kw)
@pytest.fixture(autouse=True)
def _a_tts_model_exists(monkeypatch):
"""Pretend a TTS model is installed, for every test in this module.
These tests are about how a dialogue scene is VALIDATED, and say so: the
portraits are given inline "so the test does not depend on what is saved on
this install". The TTS check is the same kind of install dependency and was
missed -- `_validate` refuses a request whose lines need narration when no
TTS model is configured, so on a bare checkout three of these failed with
501 "narration needs a TTS model" without ever reaching the validation they
exercise.
Only the INVENTORY is faked, not the check: `_has_model` still does its own
name matching, and `test_a_line_whose_voice_model_is_not_installed_is_refused`
(if it ever exists) would still see a real miss for a name not listed here.
"""
import codai.api.compose as C
monkeypatch.setattr(C, "_model_ids",
lambda mtype: (["hexgrad/Kokoro-82M"]
if mtype == "tts" else []))
# ----------------------------------------------------------------- identity # ----------------------------------------------------------------- identity
def test_a_character_is_identified_by_profile_then_portrait_then_position(): def test_a_character_is_identified_by_profile_then_portrait_then_position():
assert _speaker_key(_sp(character="Alice"), 0) == "alice" assert _speaker_key(_sp(character="Alice"), 0) == "alice"
......
# CoderAI - OpenAI-compatible API server
# Copyright (C) 2026 Stefy Lanza <stefy@nexlab.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
"""Text that does not fit: wrap, never silently shrink below readable.
A text overlay was one drawtext line sized from the frame height, so a long
text ran off a vertical frame; and a caption row wider than the frame was drawn
smaller until it fit, which on a vertical video can make it too small to read.
Both now take a ``fit``, and an explicit newline always breaks regardless --
a caller that pre-wrapped its text used to have it drawn as a single line.
"""
import pytest
from codai.compose import captions as cap
from codai.compose import render as rnd
# A measurable font is needed for any of this to do anything; without one the
# code deliberately leaves the text alone rather than guessing from character
# counts, so the tests would be vacuous.
_MEASURABLE = rnd._measure("probe", 40, "DejaVu Sans", True) is not None
needs_font = pytest.mark.skipif(not _MEASURABLE,
reason="no measurable font available")
FONT = "DejaVu Sans"
LONG = ("This is a decidedly long watermark line that will not fit across a "
"narrow vertical frame at this size")
# --------------------------------------------------------------- wrap_text
def test_an_explicit_newline_always_breaks_whatever_the_fit():
"""The case Motus hit: pre-wrapped text drawn as one line."""
lines = rnd.wrap_text("first line\nsecond line", 40, FONT, True, 100000.0)
assert lines == ["first line", "second line"], lines
def test_carriage_returns_count_as_newlines():
assert rnd.wrap_text("a\r\nb", 40, FONT, True, 100000.0) == ["a", "b"]
@needs_font
def test_wrap_breaks_a_long_text_into_lines_that_fit():
usable = 400.0
lines = rnd.wrap_text(LONG, 40, FONT, True, usable)
assert len(lines) > 1, "a long text was not broken at all"
for ln in lines:
w = rnd._measure(ln, 40, FONT, True)
assert w is not None and w <= usable, f"line still too wide ({w}): {ln!r}"
@needs_font
def test_wrap_keeps_every_word_in_order():
lines = rnd.wrap_text(LONG, 40, FONT, True, 400.0)
assert " ".join(lines).split() == LONG.split()
@needs_font
def test_a_single_word_wider_than_a_line_is_cut_not_drawn_past_the_edge():
usable = 220.0
url = "https://example.com/a/very/long/path?with=query&and=more#fragment"
lines = rnd.wrap_text(url, 36, FONT, True, usable)
assert len(lines) > 1, "an unbreakable-looking word was left overflowing"
for ln in lines:
w = rnd._measure(ln, 36, FONT, True)
assert w is not None and w <= usable, f"piece too wide ({w}): {ln!r}"
assert "".join(lines) == url, "cutting the word lost or reordered characters"
@needs_font
def test_an_unmeasurable_font_leaves_the_text_alone():
"""Guessing from character counts is wrong in exactly the case this catches."""
lines = rnd.wrap_text(LONG, 40, "NoSuchFontFamilyAnywhere__", True, 10.0)
assert lines == [LONG] or len(lines) >= 1
# ------------------------------------------------------------- shrink_to_fit
@needs_font
def test_shrink_reduces_the_size_until_one_line_actually_fits():
"""Verified by measurement: the width ratio alone picks a size too large."""
usable = 900.0
size = rnd.shrink_to_fit(LONG, 60, FONT, True, usable)
assert size < 60, "shrink did not reduce the size"
w = rnd._measure(LONG, size, FONT, True)
assert w is not None and w <= usable, f"still {w} px wide at size {size}"
@needs_font
def test_shrink_returns_the_largest_size_that_fits():
usable = 900.0
size = rnd.shrink_to_fit(LONG, 60, FONT, True, usable)
bigger = rnd._measure(LONG, size + 1, FONT, True)
assert bigger is None or bigger > usable, (
f"size {size} fits but so does {size + 1}; shrink gave up too much")
@needs_font
def test_shrink_returns_the_floor_when_nothing_fits():
"""A budget no size can meet gets the floor, not an overflowing guess."""
size = rnd.shrink_to_fit(LONG, 60, FONT, True, 30.0, floor=8)
assert size == 8, f"expected the floor, got {size}"
@needs_font
def test_shrink_leaves_a_text_that_already_fits_alone():
assert rnd.shrink_to_fit("short", 40, FONT, True, 100000.0) == 40
# --------------------------------------------------------------- caption fit
def _events():
words = [{"word": w, "start": i * 0.3, "end": i * 0.3 + 0.3}
for i, w in enumerate(LONG.split())]
return [{"start": 0.0, "end": 9.0, "words": words,
"rows": [[w["word"] for w in words]]}]
@needs_font
def test_captions_wrap_keeps_the_size_and_adds_rows():
ass = cap.build_ass(_events(), width=1080, height=1920, font=FONT,
preset="classic", fit="wrap")
# No inline \fs override means the event kept the global size.
assert "\\fs" not in ass, "wrap shrank the text anyway"
assert "\\N" in ass, "wrap produced no extra row"
@needs_font
def test_captions_shrink_is_still_the_default():
ass = cap.build_ass(_events(), width=1080, height=1920, font=FONT,
preset="classic")
assert "\\fs" in ass, "the default stopped shrinking an overlong row"
@needs_font
def test_wrapping_rows_preserves_word_order_for_karaoke():
rows = [LONG.split()]
out = cap._wrap_rows(rows, 48, 400.0, FONT, True, 0.0, False)
assert [w for row in out for w in row] == LONG.split(), (
"the karaoke highlight walks words in row order, so wrapping must not "
"reorder them")
assert len(out) > 1
@needs_font
def test_a_wrapped_karaoke_event_still_highlights_every_word():
ass = cap.build_ass(_events(), width=1080, height=1920, font=FONT,
preset="karaoke", fit="wrap")
# One Dialogue line per word, each highlighting exactly one.
dialogues = [l for l in ass.splitlines() if l.startswith("Dialogue:")]
assert len(dialogues) == len(LONG.split()), (
f"{len(dialogues)} events for {len(LONG.split())} words")
# ------------------------------------------------------- the renderer wiring
def test_the_overlay_renderer_accepts_fit_and_emits_a_line_each(monkeypatch):
"""`fit=wrap` must reach drawtext as several placed lines."""
seen = {}
def _fake_run(args, **kw):
seen["args"] = args
monkeypatch.setattr(rnd, "run", _fake_run)
rnd.final_render(
"v.mp4", "a.wav", "out.mp4", width=1080, height=1920, fps=30, total=5.0,
overlays=[{"type": "text", "text": LONG, "fit": "wrap",
"position": "bottom-right", "size": 0.03, "font": FONT}])
graph = next(a for a in seen["args"] if "drawtext" in str(a))
n = str(graph).count("drawtext")
if _MEASURABLE:
assert n > 1, f"wrap emitted {n} drawtext filter(s); expected one per line"
else:
assert n >= 1
def test_an_overlay_without_fit_still_renders_one_line(monkeypatch):
seen = {}
monkeypatch.setattr(rnd, "run", lambda args, **kw: seen.setdefault("args", args))
rnd.final_render(
"v.mp4", "a.wav", "out.mp4", width=1080, height=1920, fps=30, total=5.0,
overlays=[{"type": "text", "text": "made with motus",
"position": "bottom-right"}])
graph = next(a for a in seen["args"] if "drawtext" in str(a))
assert str(graph).count("drawtext") == 1
@needs_font
def test_a_shrunk_caption_actually_fits_its_frame():
"""The invariant `_fit_size` exists for, now held by construction.
It used to scale the size by the width ratio and trust the result. This
input passes either way -- the ratio is close enough above `_fit_size`'s
floor of 12 -- so this is a regression guard rather than a reproduction;
the assumption provably fails at smaller sizes (see shrink_to_fit, where a
computed size of 8 measured 47% over its budget).
"""
rows = [LONG.split()]
usable = 900.0
size = cap._fit_size(rows, 64, usable, FONT, True, 0.0, False)
got = cap._text_width(" ".join(rows[0]), size, FONT, True, 0.0)
assert got is not None and got <= usable, (
f"a caption shrunk to size {size} is still {got} px against a "
f"{usable} px budget, so it is drawn past the frame edge anyway")
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment