tools: bundle a character studio demo (real footage -> new video)

Adds tools/character_studio.py, a demo app over the character-profile endpoints:
extract a character from real photos/videos (POST /v1/characters/extract), review
and prune the reference crops, then generate a brand-new video starring that
person (POST /v1/video/generations with character_profiles + keyframe_identity).

CLI core plus a `web` subcommand serving the same pipeline in the browser. The
UI honours X-Forwarded-Prefix, so it works at the root or under a sub-path mount.
Every generated mp4 gets its request body written alongside it, so the demo also
doubles as a worked API example.

Wired in as a bundled tool like the other UIs: supervisord program on port 7791
(autostart via CODERAI_TOOL_CHARACTER_STUDIO, on by default), entrypoint env
defaults, run_oci.sh --enable/--disable-tool + --tool-arg names, nginx route at
/character/, service smoke test, and the run banner/README URLs.

Version bumped to 0.1.88 so the in-image upgrade path ships it.
Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
parent 1003cafa
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API # Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here. # metadata and the admin web UI read from here.
__version__ = "0.1.87" __version__ = "0.1.88"
# Configure the CUDA caching allocator BEFORE torch is imported anywhere. # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
# expandable_segments lets the allocator return freed pages to the driver even # expandable_segments lets the allocator return freed pages to the driver even
......
...@@ -14,6 +14,7 @@ ways to mount each service; pick per service: ...@@ -14,6 +14,7 @@ ways to mount each service; pick per service:
| CoderAI server (`codai`) | ✅ | ✅ | | CoderAI server (`codai`) | ✅ | ✅ |
| `tools/video_editor.py` | ✅ | ✅ | | `tools/video_editor.py` | ✅ | ✅ |
| `tools/videogen.py` | ✅ | ⚠️ needs work | | `tools/videogen.py` | ✅ | ⚠️ needs work |
| `tools/character_studio.py` | ✅ | ✅ |
| `tools/review_outputs.py` | ✅ | ⚠️ needs work | | `tools/review_outputs.py` | ✅ | ⚠️ needs work |
| `tools/gen_township_fighters.py`| ✅ | ⚠️ needs work | | `tools/gen_township_fighters.py`| ✅ | ⚠️ needs work |
......
...@@ -35,6 +35,7 @@ The image publishes ONE port (8776). nginx inside the container fronts: ...@@ -35,6 +35,7 @@ The image publishes ONE port (8776). nginx inside the container fronts:
http://HOST:8776/ CoderAI server + OpenAI-compatible API + admin UI http://HOST:8776/ CoderAI server + OpenAI-compatible API + admin UI
http://HOST:8776/editor/ Video editor http://HOST:8776/editor/ Video editor
http://HOST:8776/videogen/ Videogen studio http://HOST:8776/videogen/ Videogen studio
http://HOST:8776/character/ Character studio (real photos/video -> new video)
http://HOST:8776/township/ Township fighters http://HOST:8776/township/ Township fighters
Three volumes hold all mutable state (everything else in the image is read-only): Three volumes hold all mutable state (everything else in the image is read-only):
...@@ -87,7 +88,7 @@ change — only the host side does. ...@@ -87,7 +88,7 @@ change — only the host side does.
coderai:local coderai:local
The launcher points HF_HOME at /cache/huggingface and writes tool outputs to The launcher points HF_HOME at /cache/huggingface and writes tool outputs to
/cache/{videogen_output,township_output}, so /cache on the big disk captures /cache/{videogen_output,township_output,character_output}, so /cache captures
downloads AND produced artifacts. downloads AND produced artifacts.
2) NFS (shared across machines) — back a Docker volume with the NFS driver: 2) NFS (shared across machines) — back a Docker volume with the NFS driver:
......
...@@ -629,7 +629,8 @@ Run examples (run as your own UID; create+own the dirs first): ...@@ -629,7 +629,8 @@ Run examples (run as your own UID; create+own the dirs first):
(Drop --user to run as container-root, or use rootless/userns-remap Docker.) (Drop --user to run as container-root, or use rootless/userns-remap Docker.)
One published port (8776) fronts everything via nginx: One published port (8776) fronts everything via nginx:
/ server+API+admin /editor/ video editor /videogen/ studio /township/ fighters / server+API+admin /editor/ video editor /videogen/ studio /township/ fighters
/character/ character studio (extract a person from photos/video -> new video)
External storage: point /models and /cache at a big disk or NFS volume — External storage: point /models and /cache at a big disk or NFS volume —
-v /mnt/bigstorage/coderai/models:/models -v /mnt/bigstorage/coderai/cache:/cache -v /mnt/bigstorage/coderai/models:/models -v /mnt/bigstorage/coderai/cache:/cache
......
...@@ -23,22 +23,26 @@ export CODERAI_OFFLOAD_DIR ...@@ -23,22 +23,26 @@ export CODERAI_OFFLOAD_DIR
# Don't write .pyc into the read-only /opt/coderai tree (esp. when run as --user). # Don't write .pyc into the read-only /opt/coderai tree (esp. when run as --user).
export PYTHONDONTWRITEBYTECODE=1 export PYTHONDONTWRITEBYTECODE=1
# Demo tool web-UI autostart toggles, consumed by supervisord.conf's # Demo tool web-UI autostart toggles, consumed by supervisord.conf's
# autostart=%(ENV_CODERAI_TOOL_*)s. Default the three UIs ON and parler OFF; the # autostart=%(ENV_CODERAI_TOOL_*)s. Default the four UIs ON and parler OFF; the
# run script (or any `-e CODERAI_TOOL_*=true|false`) overrides these. supervisord # run script (or any `-e CODERAI_TOOL_*=true|false`) overrides these. supervisord
# fails to start if a referenced ENV var is undefined, so always export them. # fails to start if a referenced ENV var is undefined, so always export them.
: "${CODERAI_TOOL_VIDEO_EDITOR:=true}" : "${CODERAI_TOOL_VIDEO_EDITOR:=true}"
: "${CODERAI_TOOL_VIDEOGEN:=true}" : "${CODERAI_TOOL_VIDEOGEN:=true}"
: "${CODERAI_TOOL_TOWNSHIP:=true}" : "${CODERAI_TOOL_TOWNSHIP:=true}"
: "${CODERAI_TOOL_CHARACTER_STUDIO:=true}"
: "${CODERAI_TOOL_PARLER:=false}" : "${CODERAI_TOOL_PARLER:=false}"
export CODERAI_TOOL_VIDEO_EDITOR CODERAI_TOOL_VIDEOGEN CODERAI_TOOL_TOWNSHIP CODERAI_TOOL_PARLER export CODERAI_TOOL_VIDEO_EDITOR CODERAI_TOOL_VIDEOGEN CODERAI_TOOL_TOWNSHIP
export CODERAI_TOOL_CHARACTER_STUDIO CODERAI_TOOL_PARLER
# Per-tool extra CLI args appended by supervisord (%(ENV_CODERAI_*_ARGS)s). The run # Per-tool extra CLI args appended by supervisord (%(ENV_CODERAI_*_ARGS)s). The run
# script sets these via --tool-arg/--tool-args; default empty. supervisord fails to # script sets these via --tool-arg/--tool-args; default empty. supervisord fails to
# start if a referenced ENV var is undefined, so always export them. # start if a referenced ENV var is undefined, so always export them.
: "${CODERAI_VIDEO_EDITOR_ARGS:=}" : "${CODERAI_VIDEO_EDITOR_ARGS:=}"
: "${CODERAI_VIDEOGEN_ARGS:=}" : "${CODERAI_VIDEOGEN_ARGS:=}"
: "${CODERAI_TOWNSHIP_ARGS:=}" : "${CODERAI_TOWNSHIP_ARGS:=}"
: "${CODERAI_CHARACTER_STUDIO_ARGS:=}"
: "${CODERAI_PARLER_ARGS:=}" : "${CODERAI_PARLER_ARGS:=}"
export CODERAI_VIDEO_EDITOR_ARGS CODERAI_VIDEOGEN_ARGS CODERAI_TOWNSHIP_ARGS CODERAI_PARLER_ARGS export CODERAI_VIDEO_EDITOR_ARGS CODERAI_VIDEOGEN_ARGS CODERAI_TOWNSHIP_ARGS
export CODERAI_CHARACTER_STUDIO_ARGS CODERAI_PARLER_ARGS
export CODERAI_CONFIG_DIR CODERAI_MODELS_DIR CODERAI_CACHE_DIR CODERAI_PARLER_MODEL CODERAI_TMP export CODERAI_CONFIG_DIR CODERAI_MODELS_DIR CODERAI_CACHE_DIR CODERAI_PARLER_MODEL CODERAI_TMP
mkdir -p \ mkdir -p \
......
...@@ -82,6 +82,10 @@ http { ...@@ -82,6 +82,10 @@ http {
"~/videogen$" $fwd_prefix; "~/videogen$" $fwd_prefix;
default $fwd_prefix/videogen; default $fwd_prefix/videogen;
} }
map $fwd_prefix $xfp_character {
"~/character$" $fwd_prefix;
default $fwd_prefix/character;
}
map $fwd_prefix $xfp_township { map $fwd_prefix $xfp_township {
"~/township$" $fwd_prefix; "~/township$" $fwd_prefix;
default $fwd_prefix/township; default $fwd_prefix/township;
...@@ -103,6 +107,7 @@ http { ...@@ -103,6 +107,7 @@ http {
upstream editor { server 127.0.0.1:8420; } upstream editor { server 127.0.0.1:8420; }
upstream videogen { server 127.0.0.1:7790; } upstream videogen { server 127.0.0.1:7790; }
upstream township { server 127.0.0.1:7788; } upstream township { server 127.0.0.1:7788; }
upstream character { server 127.0.0.1:7791; }
server { server {
listen 8776 default_server; listen 8776 default_server;
...@@ -125,6 +130,14 @@ http { ...@@ -125,6 +130,14 @@ http {
proxy_buffering off; proxy_buffering off;
} }
# --- Character studio: https://host:8776/character/ -----------------
location /character/ {
proxy_pass http://character/;
proxy_set_header X-Forwarded-Prefix $xfp_character; # smart nest (see map above)
proxy_request_buffering off; # base64 photo/video uploads
proxy_buffering off;
}
# --- Township fighters: https://host:8776/township/ ---------------- # --- Township fighters: https://host:8776/township/ ----------------
location /township/ { location /township/ {
proxy_pass http://township/; proxy_pass http://township/;
......
...@@ -77,8 +77,8 @@ MAPS=() ...@@ -77,8 +77,8 @@ MAPS=()
# volume so it's tailable on the host). # volume so it's tailable on the host).
DEBUG_SPEC="" DEBUG_SPEC=""
LOG_FILE_CONT="" LOG_FILE_CONT=""
# Demo tool web UIs (video editor, videogen, township, parler). Empty = image # Demo tool web UIs (video editor, videogen, township, character studio, parler).
# default (the three UIs on, parler off). Keyed by CODERAI_TOOL_* env var. # Empty = image default (the four UIs on, parler off). Keyed by CODERAI_TOOL_* env.
declare -A TOOL_STATE=() declare -A TOOL_STATE=()
DISABLE_ALL_TOOLS=0 DISABLE_ALL_TOOLS=0
# Extra CLI args appended to a tool's command line, keyed by CODERAI_*_ARGS env var. # Extra CLI args appended to a tool's command line, keyed by CODERAI_*_ARGS env var.
...@@ -90,6 +90,7 @@ tool_env_var() { ...@@ -90,6 +90,7 @@ tool_env_var() {
video-editor|video_editor|editor) echo CODERAI_TOOL_VIDEO_EDITOR ;; video-editor|video_editor|editor) echo CODERAI_TOOL_VIDEO_EDITOR ;;
videogen|video-gen) echo CODERAI_TOOL_VIDEOGEN ;; videogen|video-gen) echo CODERAI_TOOL_VIDEOGEN ;;
township|fighters) echo CODERAI_TOOL_TOWNSHIP ;; township|fighters) echo CODERAI_TOOL_TOWNSHIP ;;
character-studio|character|chars) echo CODERAI_TOOL_CHARACTER_STUDIO ;;
parler|tts) echo CODERAI_TOOL_PARLER ;; parler|tts) echo CODERAI_TOOL_PARLER ;;
*) return 1 ;; *) return 1 ;;
esac esac
...@@ -102,6 +103,7 @@ tool_args_var() { ...@@ -102,6 +103,7 @@ tool_args_var() {
video-editor|video_editor|editor) echo CODERAI_VIDEO_EDITOR_ARGS ;; video-editor|video_editor|editor) echo CODERAI_VIDEO_EDITOR_ARGS ;;
videogen|video-gen) echo CODERAI_VIDEOGEN_ARGS ;; videogen|video-gen) echo CODERAI_VIDEOGEN_ARGS ;;
township|fighters) echo CODERAI_TOWNSHIP_ARGS ;; township|fighters) echo CODERAI_TOWNSHIP_ARGS ;;
character-studio|character|chars) echo CODERAI_CHARACTER_STUDIO_ARGS ;;
parler|tts) echo CODERAI_PARLER_ARGS ;; parler|tts) echo CODERAI_PARLER_ARGS ;;
*) return 1 ;; *) return 1 ;;
esac esac
...@@ -185,15 +187,16 @@ Upgrade (refresh the in-image code instead of running the server): ...@@ -185,15 +187,16 @@ Upgrade (refresh the in-image code instead of running the server):
visible on the host under the cache mount). Implies a file visible on the host under the cache mount). Implies a file
log even without --debug. tee'd, so `docker logs` still works. log even without --debug. tee'd, so `docker logs` still works.
--no-tools Disable ALL bundled demo tool web UIs (video editor, --no-tools Disable ALL bundled demo tool web UIs (video editor,
videogen, township). They're on by default. videogen, township, character studio). On by default.
--enable-tool NAME Force-enable a demo tool. Repeatable. NAME is one of: --enable-tool NAME Force-enable a demo tool. Repeatable. NAME is one of:
video-editor | videogen | township | parler video-editor | videogen | township | character-studio
| parler
(parler TTS is off by default; this turns it on.) (parler TTS is off by default; this turns it on.)
--disable-tool NAME Disable a single demo tool. Repeatable. Same NAMEs as above. --disable-tool NAME Disable a single demo tool. Repeatable. Same NAMEs as above.
Explicit --enable/--disable-tool overrides --no-tools. Explicit --enable/--disable-tool overrides --no-tools.
--tool-arg TOOL VAL Append ONE extra CLI arg to a bundled tool's command line. --tool-arg TOOL VAL Append ONE extra CLI arg to a bundled tool's command line.
Repeatable. TOOL is one of video-editor|videogen|township| Repeatable. TOOL is one of video-editor|videogen|township|
parler. e.g. --tool-arg township --web-port --tool-arg township 9000 character-studio|parler. e.g. --tool-arg township --web-port --tool-arg township 9000
--tool-args TOOL STR --tool-args TOOL STR
Like --tool-arg but appends a whole whitespace-separated Like --tool-arg but appends a whole whitespace-separated
string at once, e.g. --tool-args video-editor "--voice masculine". string at once, e.g. --tool-args video-editor "--voice masculine".
...@@ -292,11 +295,11 @@ while [[ $# -gt 0 ]]; do ...@@ -292,11 +295,11 @@ while [[ $# -gt 0 ]]; do
--no-tools) DISABLE_ALL_TOOLS=1; shift ;; --no-tools) DISABLE_ALL_TOOLS=1; shift ;;
--enable-tool) --enable-tool)
[[ $# -ge 2 ]] || { echo "Error: --enable-tool requires a tool name" >&2; exit 2; } [[ $# -ge 2 ]] || { echo "Error: --enable-tool requires a tool name" >&2; exit 2; }
_v="$(tool_env_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|parler)" >&2; exit 2; } _v="$(tool_env_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|character-studio|parler)" >&2; exit 2; }
TOOL_STATE["$_v"]=true; shift 2 ;; TOOL_STATE["$_v"]=true; shift 2 ;;
--disable-tool) --disable-tool)
[[ $# -ge 2 ]] || { echo "Error: --disable-tool requires a tool name" >&2; exit 2; } [[ $# -ge 2 ]] || { echo "Error: --disable-tool requires a tool name" >&2; exit 2; }
_v="$(tool_env_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|parler)" >&2; exit 2; } _v="$(tool_env_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|character-studio|parler)" >&2; exit 2; }
TOOL_STATE["$_v"]=false; shift 2 ;; TOOL_STATE["$_v"]=false; shift 2 ;;
# Append extra CLI args to a bundled tool's command. --tool-arg adds ONE token; # Append extra CLI args to a bundled tool's command. --tool-arg adds ONE token;
# --tool-args adds a whitespace-separated string. Repeatable. e.g. # --tool-args adds a whitespace-separated string. Repeatable. e.g.
...@@ -304,11 +307,11 @@ while [[ $# -gt 0 ]]; do ...@@ -304,11 +307,11 @@ while [[ $# -gt 0 ]]; do
# --tool-args video-editor "--voice masculine" # --tool-args video-editor "--voice masculine"
--tool-arg) --tool-arg)
[[ $# -ge 3 ]] || { echo "Error: --tool-arg requires TOOL and a value" >&2; exit 2; } [[ $# -ge 3 ]] || { echo "Error: --tool-arg requires TOOL and a value" >&2; exit 2; }
_v="$(tool_args_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|parler)" >&2; exit 2; } _v="$(tool_args_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|character-studio|parler)" >&2; exit 2; }
TOOL_ARGS["$_v"]="${TOOL_ARGS[$_v]:+${TOOL_ARGS[$_v]} }$3"; shift 3 ;; TOOL_ARGS["$_v"]="${TOOL_ARGS[$_v]:+${TOOL_ARGS[$_v]} }$3"; shift 3 ;;
--tool-args) --tool-args)
[[ $# -ge 3 ]] || { echo "Error: --tool-args requires TOOL and a string" >&2; exit 2; } [[ $# -ge 3 ]] || { echo "Error: --tool-args requires TOOL and a string" >&2; exit 2; }
_v="$(tool_args_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|parler)" >&2; exit 2; } _v="$(tool_args_var "$2")" || { echo "Error: unknown tool '$2' (video-editor|videogen|township|character-studio|parler)" >&2; exit 2; }
TOOL_ARGS["$_v"]="${TOOL_ARGS[$_v]:+${TOOL_ARGS[$_v]} }$3"; shift 3 ;; TOOL_ARGS["$_v"]="${TOOL_ARGS[$_v]:+${TOOL_ARGS[$_v]} }$3"; shift 3 ;;
-d|--detach) DETACH=1; shift ;; -d|--detach) DETACH=1; shift ;;
--) --)
...@@ -607,11 +610,12 @@ fi ...@@ -607,11 +610,12 @@ fi
# Demo tool toggles → CODERAI_TOOL_* env. --no-tools turns the three UIs off # Demo tool toggles → CODERAI_TOOL_* env. --no-tools turns the three UIs off
# (unless a specific --enable-tool re-enabled one); explicit toggles always win. # (unless a specific --enable-tool re-enabled one); explicit toggles always win.
if [[ "$DISABLE_ALL_TOOLS" == "1" ]]; then if [[ "$DISABLE_ALL_TOOLS" == "1" ]]; then
for _v in CODERAI_TOOL_VIDEO_EDITOR CODERAI_TOOL_VIDEOGEN CODERAI_TOOL_TOWNSHIP; do for _v in CODERAI_TOOL_VIDEO_EDITOR CODERAI_TOOL_VIDEOGEN CODERAI_TOOL_TOWNSHIP \
CODERAI_TOOL_CHARACTER_STUDIO; do
[[ -n "${TOOL_STATE[$_v]:-}" ]] || TOOL_STATE["$_v"]=false [[ -n "${TOOL_STATE[$_v]:-}" ]] || TOOL_STATE["$_v"]=false
done done
fi fi
TOOLS_NOTE="image defaults (video-editor, videogen, township on; parler off)" TOOLS_NOTE="image defaults (video-editor, videogen, township, character-studio on; parler off)"
if [[ "${#TOOL_STATE[@]}" -gt 0 ]]; then if [[ "${#TOOL_STATE[@]}" -gt 0 ]]; then
TOOLS_NOTE="" TOOLS_NOTE=""
for _v in "${!TOOL_STATE[@]}"; do for _v in "${!TOOL_STATE[@]}"; do
......
...@@ -45,7 +45,7 @@ done ...@@ -45,7 +45,7 @@ done
[ "$up" = 1 ] && ok "front http://…:$PORT/ responds" || bad "front /" "no response" [ "$up" = 1 ] && ok "front http://…:$PORT/ responds" || bad "front /" "no response"
echo "== sub-path mounts ==" echo "== sub-path mounts =="
for p in editor videogen township; do for p in editor videogen township character; do
code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:$PORT/$p/" || true)" code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:$PORT/$p/" || true)"
case "$code" in 200|301|302|307) ok "/$p/ ($code)";; *) bad "/$p/" "http $code";; esac case "$code" in 200|301|302|307) ok "/$p/ ($code)";; *) bad "/$p/" "http $code";; esac
done done
......
...@@ -82,6 +82,25 @@ stdout_logfile=/dev/fd/1 ...@@ -82,6 +82,25 @@ stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0 stdout_logfile_maxbytes=0
redirect_stderr=true redirect_stderr=true
; Character studio: extract a character from real photos/videos, then generate a
; new video with them. Global options come BEFORE the `web` subcommand, so any
; --tool-arg character-studio ... lands on the web subparser (--host/--web-port).
[program:character_studio]
command=/usr/local/bin/with-env /opt/coderai/python/bin/python3 /opt/coderai/app/tools/character_studio.py
--base-url http://127.0.0.1:8776
--out-dir /cache/character_output
web --host 127.0.0.1 --web-port 7791
%(ENV_CODERAI_CHARACTER_STUDIO_ARGS)s
directory=/opt/coderai/app
autostart=%(ENV_CODERAI_TOOL_CHARACTER_STUDIO)s
autorestart=true
startsecs=5
startretries=5
priority=30
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true
[program:township] [program:township]
command=/usr/local/bin/with-env /opt/coderai/python/bin/python3 /opt/coderai/app/tools/gen_township_fighters.py command=/usr/local/bin/with-env /opt/coderai/python/bin/python3 /opt/coderai/app/tools/gen_township_fighters.py
--web-port 7788 --web-port 7788
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment