-
Stefy Lanza (nextime / spora ) authored
Motus request 4. The most common reel format is a person talking to camera, and only this server can build it: composition synthesises the narration, so it holds the exact audio of each scene, and the lip-sync engines and identity-locked video generation are here too. - codai/compose/presenter.py: per scene — a face (a character profile's front reference, or a portrait), something to move in (a still, the caller's clip, or a short identity-locked shot generated through the ordinary video endpoint so its model config/acceleration/keyframe identity all apply), then an engine driven by THAT scene's narration, held to the scene's exact length and the canvas size. wav2lip and sadtalker are known by name; anything else installed under its own name can be selected by that name, so a newer engine needs no API change. engine=auto prefers a video-driven engine when there is a clip and a portrait engine otherwise. - Layouts in render.py: full (replaces the scene's visuals — the scene may then have none), pip (a masked circle/rounded/rect bubble with an optional border, inside the caption safe area) and split (stacked halves). With a presenter a scene's visuals are joined without an internal transition and composited as one clip; without one nothing changes. - Bottom captions move above a bottom bubble: ASS takes a per-event MarginV, so only the scenes that need it move (captions.py carries margin_v through). - Failure never loses a render: no engine, no face, a crash or a failed composite falls back to the scene's visuals (a gradient when there are none) and warns, naming the scene. New stage "presenter"; result.scenes[i].presenter says which engine and layout, or null. - Lip-sync is the only GPU step in a composition, so it takes a lease from the central scheduler instead of fighting a generation for the card. - POST /v1/video/talking-head (+ /{id}, /{id}/cancel) is the same pipeline as a one-scene full-screen composition, for a character-page preview, capped at 15 s; GET /v1/video/presenter/engines tells a client what is installed so it can grey the feature out instead of meeting a 501. Both stay local in the capability gateway (a video pod has the shims but not this install's characters). - 400 for an unknown character / invalid layout, position, size, shape, motion, or a pip scene with no visuals; 501 with what to install when no engine is present. - docs/video-compose.md, README; tests/test_compose.py 32 -> 43 (engines stubbed, compositing verified on real pixels). Co-Authored-By:Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7EnDW6RSd2nhEPZMzMkqi
b8b5ba3a
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| __init__.py | ||
| captions.py | ||
| media.py | ||
| presenter.py | ||
| render.py |