- cuda/vulkan backend improvements and config plumbing
- API updates across characters, text, environments, audio, embeddings, tts
- admin chat/settings template updates
- add hf_loading helper, video request fields, platform paths
- new docs (CODERAI_API_DOCUMENTATION.md) and tools (review_outputs, video_dubber)
- ignore generated township_output/
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The server exposes one global training progress (jobs run one at a time via the
queue), so every queued card was mirroring the active job's progress. Restore the
name match: a card shows real progress only when the global progress reports ITS
LoRA; otherwise it shows "queued — '<other>' training first… (elapsed)". Keeps the
elapsed-timer handling so a long model load still looks alive rather than frozen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Extend the cross-job cache from just the transformer expert(s) to the full Wan
stack: VAE, tokenizer and text encoder are kept on CPU between jobs (moved to GPU
only while encoding), experts stay on GPU. A back-to-back training against the
same base now reloads nothing from disk — previously the small VAE/text-encoder
still reloaded each job. The releaser and error path clear all cached components.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- Match detail page: Fighter 1/2 and Environment are now dropdowns of saved
  profiles + the built-in pools, so a match's fighters/location can be switched
  before re-rendering (Save match persists; env_desc updated with env).
- Video LoRA status on profile cards: resolve the active video model the same
  way training/generation do (configured, else auto-picked, cached), and show
  "trained for: <slug>" even when the active model can't be resolved so a trained
  LoRA never looks untrained.
- Video LoRA training progress: show a ticking elapsed timer during the long
  model-load/preparing phase (the progress endpoint is starved while a large
  model loads), drop the strict name filter that could freeze the bar at 6%, and
  use step-based % once training begins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Cache the Wan transformer expert(s) between consecutive trainings against the
same base (keyed by base_path+quantize) so a back-to-back job skips the very slow
reload (tens of minutes for A14B). Only this job's adapter + gradient-checkpoint
hooks are removed at teardown; the base transformer(s) stay resident. Since 4-bit
weights can't move to CPU, they hold GPU VRAM between jobs — so the external VRAM
releaser now drops the Wan cache too when a generation needs the GPU, and the
error path clears both caches.

Also report training progress every step (cheap dict update) instead of every
10, so the web UI bar advances smoothly once steps begin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

torch.rand defaults to fp32, so the rectified-flow interpolation promoted x_t to
fp32 while the patch-embedding Conv3d stays bf16 (bitsandbytes 4-bit quantizes
only Linear layers), raising "Input type (float) and bias type (BFloat16) should
be the same". Compute the interpolation in fp32 then cast x_t/target back to the
model compute dtype, and pass timestep as fp32 (Wan casts it internally).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Suppressing the whole uvicorn.access logger hid every request. Instead add a
logging.Filter that drops only /v1/loras/progress (polled ~1.5s by the web UI);
all other API request lines keep logging. A logger filter also survives uvicorn's
configure_logging(), which resets the access logger LEVEL at startup (that reset
is what defeated the earlier setLevel(WARNING)). --debug-web shows everything.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Several related changes accumulated in this session:

Wan video LoRAs (additive — image LoRAs kept for keyframes):
- New per-model maps video_loras.json/env_video_loras.json keyed
  name -> {model_slug: path}; on-disk names tagged with the video model slug.
- Video requests attach the video LoRA matching the current video model's slug;
  image LoRAs stay on the keyframe path only.
- Per-profile "Train video LoRA" button + step button + full-run checkbox +
  --video-loras/--only-video-loras; batch + CLI wiring; client target="video".

Final/outcome enhance (upscale 2x/4x + raise FPS):
- _enhance_video_file + Phase C stage; --upscale-factor/--fps-multiplier and
  Run-page selects; match-page Enhance card with live progress bars.

Match page UX:
- Video previews enlarge + center on play (video lightbox).
- Match render shows global + per-clip progress bars, surviving reload.

Outcome fixes:
- Re-rendering a match's outcomes now resolves legacy per-fighter outcomes
  (no match_name) by fighter membership, and forces them into the match's
  environment so a match stays in one location.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Adds a target="video" path that trains a LoRA directly against the configured
video model so it loads on the video pipeline (image LoRAs can't apply to a Wan
DiT). _train_wan: encodes stills as 1-frame latents via the Wan 3D VAE
(latents_mean/std normalized), encodes the prompt via UMT5, loads the
transformer expert(s) in 4-bit (QLoRA) with gradient checkpointing, adds PEFT
LoRA to the attention projections, and trains a rectified-flow loss. Handles
Wan2.2's dual experts (transformer + transformer_2) via boundary_ratio routing,
and saves both expert LoRA layers (falls back to high-noise only on older
diffusers). Reuses the queue, eviction, thermal checkpoints and progress.

LoraTrainRequest gains target/quantize_4bit/num_frames; base-path resolution
gains a "video" category so it resolves the video model entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

An SD/SDXL image LoRA loaded onto a Wan video transformer matches no keys, so
set_adapters() raised "not in the list of present adapters: set()" and aborted
the whole request. Now each adapter is checked against the pipe's PEFT-capable
components after load; ones that registered nothing (wrong architecture) are
skipped with a clear message and generation proceeds with whatever is
compatible (or no LoRA). The request signature is still cached so the futile
load isn't retried every clip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The video model stays cached across clips, but LoRAs were loaded from disk
and fully unloaded on every clip — wasted I/O and fusion latency, since
consecutive clips of a match request the identical fighter+env LoRAs.

Cache the active LoRA signature on the pipe (_coderai_active_loras) and only
swap when it changes: a request with the same set reuses the loaded adapters,
a different set (or none) triggers a clean unload + reload. Replaces the
apply-then-unload-every-clip path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Match re-render previously showed only a frozen text status. Now:
- _stage_videos_render reports per-clip start/end via clip_cb and overall
  done/total via progress_cb.
- _run_match_job seeds an items list and maps clip state into the job record.
- The match detail page renders a global progress bar plus one bar per
  clip/output (pending/rendering/done/failed), and these survive a reload
  via /active-jobs + resumeMatchJobs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Route /v1/loras/train through queue_manager.acquire/release with a constant
"lora-train" model key. Concurrent training requests now queue and run one after
another (serialized by the scheduler, protecting the shared base cache) and
participate in the same scheduling/metrics as every other model request, instead
of being rejected with 409. _train_lock is kept as the in-flight signal that
_release_base_cache checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Tag regen/train jobs with kind/name/jtype and add a /active-jobs endpoint listing
running ones. On Characters/Environments page load, resumeActiveJobs() re-attaches
the live progress display to any in-flight job for the matching card, so a reload
(or reopening the tab) keeps showing progress until completion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add a shared image lightbox (injected into every page via _page) and make the
reference-image thumbnails on the Characters/Environments pages clickable to view
them full-size. Click the backdrop or press Escape to close. Delete button still
works independently.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- LoRA trainer: cache the SD/SDXL base on CPU between jobs so back-to-back
  trainings against the same base skip the disk reload, and the base holds no
  VRAM between jobs (moved to GPU only while training). Fixes the post-training
  eviction failure that forced the next image request into CPU/disk offload.
- Model manager: add register_external_vram_releaser() + last-resort eviction
  pass so a generation can reclaim the trainer's cached base when needed (skips
  while a job runs).
- Thermal: average 3 CPU samples spread across a 3s budget for the resume/
  cooldown decision (CPU sensors swing +/-10C); pause stays single-read to react
  fast. Bounded so it never blocks past 3s of the poll interval.
- Debug flags: --debug-web (uvicorn access lines), --debug-thermal ([thermal]
  [debug] checks), --debug-lora (per-step training loss to terminal); all off by
  default and independent of --debug.
- Admin: lora_train_base_model field on the Models page; saves apply live to the
  running server (build_runtime_kwargs/apply_model_entry_live) with no restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- images.py: stop reusing cached prompt embeds for Z-Image. Its encode_prompt
  returns per-sample lists the pipeline consumes during a run, so cache-HIT
  reuse corrupted the batch (AssertionError: len(size) == bsz in unpatchify).
  Z-Image now always encodes natively; cache key also bound to id(pipeline).
- loras.py: fix wrong import (multi_model_manager lives in codai.models.manager,
  not codai.api.state) used for VRAM unload + base-model path resolution; add an
  up-front architecture guard so transformer/DiT models (Z-Image/Flux/SD3) fail
  with a clear message instead of crashing in the CLIP tokenizer.
- New train_base_model on LoraTrainRequest: train LoRAs against a separate
  UNet-based SD1.x/SDXL model while generation keeps using the DiT image model.
- gen_township_fighters.py: thread --lora-train-base-model / web field through
  train_lora, _train_profile_loras, stage_loras/stage_env_loras, all run + per-
  card call sites; CONFIG_FIELDS + live default_args apply on /start, /save-config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Browsers routinely abort media (video seek/scrub) and SSE connections
mid-response, which surfaced as BrokenPipeError/ConnectionResetError
tracebacks spamming the terminal. Swallow these in the request handler's
handle_one_request() and finish() so they're ignored cleanly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Matches:
- Outcomes are now per-match (<match>_<fighter>_<outcome>), generated for each
  match's two fighters, so each match's outcome scenes are its own/different.
- Support multiple matches for the same pair+environment via unique match names
  (numeric suffix); scanner and delete isolate siblings correctly.
- List view shows a lightweight preview (clip00 keyframe or metadata poster);
  detail view shows clips AND the match's outcomes per fighter, each with an
  editable prompt + re-render + remove. Clip/outcome prompts save to
  prompts.json. Legacy per-fighter outputs still listed.

Prompts page: reframed as master prompts that auto-write a unique prompt per
clip/outcome of every match; static templates marked fallback-only.

Terminal: per-card action jobs (regenerate, train LoRA, match render) now echo
their progress to the terminal too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Matches page:
- Lightweight list view (no embedded players); per-match/per-fighter detail
  loads only that item's videos.
- Edit a match: change fighters, environment, short/long targets, and each
  clip's prompt (saved to prompts.json). Fighter-outputs detail edits outcome
  prompts.
- Remove videos: per clip, per final, whole match, single output, or all of a
  fighter's outputs (keeps plan/keyframes for re-render). Traversal-guarded.

Global Prompts page (/prompts):
- Edit the script's global templates — LLM fight/outcome system prompts and the
  static fight-shot + win/ko_win/retire/draw fallback templates. Persisted to
  <out-dir>/prompts_config.json, applied live and re-loaded on every run, with
  reset-to-defaults.

Web UI:
- Only auto-open a browser when --browser is passed (avoids spawning a terminal
  text browser on headless servers).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

LoRA training (codai/api/loras.py):
- Fix instant crash "mat1 and mat2 have the same dtype (Half != float)":
  components loaded in their checkpoint's native dtype (fp16 text encoders +
  fp32 UNet). Force consistent fp32 (stable LoRA reference) for both SD1.5 and
  SDXL, and free VAE/text encoders after pre-encoding to keep memory in budget.

Web UI (tools/gen_township_fighters.py):
- Per-profile LoRA training launched from the Characters/Environments cards
  (steps/rank, live server progress, status badge); the "Train LoRAs" step now
  trains regardless of the consistency setting and logs what it does. Profiles
  are uploaded to CoderAI first so server-side training has reference images.
- New Matches page: final short/long videos, single clips, and outcome outputs
  per match. Re-render clips/outputs with the video model (reuses the render
  path, honoring keyframe/LoRA settings) or reassemble finals from existing
  clips (no model).
- Replace browser alert/confirm/prompt with proper in-page modal dialogs
  (uiConfirm/uiAlert/uiPrompt) on every page.
- Apply changed connection/model settings (image/video/text model) to the live
  session on Save config / Start, so per-profile jobs use them immediately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Web UI (tools/gen_township_fighters.py):
- Characters and Environments management pages: edit every meta field
  (data-driven form, not a fixed set), remove a profile, delete individual
  reference images — synced to CoderAI (DELETE/PATCH).
- Regenerate references: background job generates N new images guided by the
  kept references (IP-Adapter) and APPENDS them, preserving non-deleted ones.
- Upload your own reference image files (magic-byte validated, appended).
- Clear run banner + status-pill label indicating which run/step is executing.
- Config: -s/--save and -c/--config; web Save button writes server-side and,
  when launched with -c, defaults to that same config path.

Environment parity / consistency:
- Per-environment identity LoRAs (loras.py: environment field, env image
  resolver, "place" instance prompt; script stage_env_loras, env_loras.json,
  applied to keyframes + video alongside fighter LoRAs).
- Environment IP-Adapter: environment_profiles on the image request
  (imagerequest.py) resolved into the IP-Adapter reference set (images.py),
  so environment regeneration can match kept references.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

CLI: -s/--save FILE writes selected generation options to a JSON config
and exits; -c/--config FILE loads them as defaults (command-line args
still override, via a two-phase parse).

Web UI: "Save config" button writes the current form options to a file
on the server (relative paths land in the output dir, absolute paths as-is)
via a new /save-config endpoint. Web- and CLI-saved configs are
interchangeable (shared CONFIG_FIELDS / canonical mapping).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

- Character profiles: extract from images/video or generate from text prompt;
  up to 6 profiles per image/video generation via IP-Adapter conditioning
- Environment profiles: same multi-slot system for scene/background conditioning
- Multi-character dialog for video: per-character TTS synthesis with voice profile
  support, sequential or manual timing, ffmpeg amix, Wav2Lip/SadTalker lip sync
- 2D↔3D conversion: image/video to stereo/anaglyph/depth/mesh and back;
  text/image to 3D model (GLB); turntable video rendering
- Generation archive: auto-save all outputs with configurable retention and
  hourly cleanup; browse/delete from web UI and API
- Voice profile enhancements: extract from video, PATCH update, GET by name;
  TTS endpoint now accepts voice_profile for F5-TTS cloning
- Image generation progress tracking via GET /v1/images/progress
- Web Studio: Profiles tab (Characters, Environments, Voices) with full managers;
  multi-slot character/environment selectors in all panels; character dialog
  section in all video panels; archive settings in Settings page
- Fix: Profiles tab sub-tab switching now correctly updates the content panel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>