- 13 Sep, 2026 15 commits
-
-
Stefy Lanza (nextime / spora ) authored18 tests over local stub servers — no GPU, no weights, no network: worker service_url health enforcement, the mux service + RemoteMuxEngine (streaming, stats, mid-stream cancel, error propagation, dead service), remote GGUF text models (routing, /v1 normalisation, auth, chat, SSE, no-local-weights), RunPod engine selection (HF->vLLM, GGUF->llama.cpp, pins, the missing-hf_gguf error), and the gateway (transparency when nothing is remote, capability and per-model routing, verbatim multipart forwarding, chat exclusion, clean 502). Also stop the mux service dumping a broken-pipe traceback when a client cancels a stream — hanging up IS the cancel signal there. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
-
Stefy Lanza (nextime / spora ) authored
Images, video, embeddings, rerank, OCR, TTS, STT, voice cloning, audio generation, stems, 3D and pipelines all load their model in-process, so none of them had a service to point elsewhere. Writing thirteen services would be the long way round; another coderai already exposes all of them, so it can be the service. RemoteGatewayMiddleware sits in front of the routes and, when a request is addressed to something configured as remote, replays it verbatim to that endpoint and streams the answer back — JSON and multipart alike, since the body is forwarded as raw bytes. Routing is per model (`service_url` on the models.json entry) first, then per capability (`remotes.endpoints` in config.json, or CODERAI_REMOTE_<CAP>_URL). Placement matters and is deliberate: it is registered first, so it ends up innermost — auth, rate limiting and request logging all run before it, and a request still has to be authorised here before being forwarded. Chat and completions are excluded: those go through RemoteOpenAIBackend, which keeps usage accounting, tool parsing and the manager's bookkeeping. When nothing is configured remote it returns on its first branch and never buffers a body (cached 5s so it doesn't walk the model list per request). Bodies over `max_body_mb` are served locally rather than held in RAM. A non-forwarded request replays its buffered body downstream untouched — tested, including multipart uploads that still parse locally. docs/remote-execution.md documents all four remotization mechanisms together. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Two additions, one goal: no text model is stuck on this box. 1. codai/backends/remote_openai.py — set `service_url` on a model entry and coderai proxies that model to any OpenAI-compatible endpoint (another coderai, llama-server, vLLM, SGLang, a rented pod) instead of loading it. `served_model` renames it for the remote, `api_key` adds a bearer token. It reuses RunPod's request/SSE machinery and only replaces endpoint resolution, so streaming and usage accounting are the same code path. This is what makes the GGUF/llama.cpp catalogue remotizable: those models load in-process through llama-cpp-python, so there is no service to redirect — pointing the *model* at a URL is the only redirect available. The manager checks it first, ahead of every engine route, and `_model_is_runpod` grew into `_model_is_remote` so such models are never downloaded, cached, or charged local VRAM (old name kept as an alias). 2. RunPod pods pick their server per model. A GGUF cannot simply go to vLLM: its GGUF loader is experimental — single file only, a limited architecture list, and it still wants the original repo's tokenizer. So `engine: auto` sends an HF repo to vLLM and a GGUF to llama.cpp's own server image (ghcr.io/ggml-org/llama.cpp:server-cuda), which downloads the weights itself from `hf_gguf` ("user/repo:Q4_K_M") — a local /AI/….gguf path means nothing on a rented machine, and we don't upload multi-GB weights. `engine: vllm|llamacpp` overrides. Both servers answer /v1/models, so the readiness probe is unchanged. Tested against fake servers: routing, /v1 normalisation, auth header, chat, SSE streaming, usage, and each engine-selection branch. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
These two were the exception in the last commit: they are driven over a stdin/stdout mux protocol, so there was no URL to redirect. This adds the missing HTTP boundary. tools/mux_service.py hosts a real MuxEngine (colibri or k3 — both use the same class) and exposes /health, /run (NDJSON: text chunks then stats), /pause and /resume. codai/api/mux_remote.RemoteMuxEngine is the client and presents the exact surface the backends use — run/is_alive/pause/ resume/close/log_tail/family — so ensure_engine() can hand one back and nothing downstream can tell local from remote. Cancellation travels as a dropped connection: the service's write fails and it stops generating, matching the local `cancelled` callback semantics. `service_url` is now a real config field on all five engine backends (ds4, colibri, k3, ktransformers, vllm), settable through the admin API and persisted, rather than only an env var. Tested end-to-end against a stub engine: streaming, stats, mid-stream cancellation, pause/resume, and error propagation all round-trip. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Each `ensure_service()` already returns a base URL, so a worker can be pointed at a service running elsewhere — another host, a container, a rented RunPod pod — without anything downstream noticing. Only the TTS backends had that escape hatch (parler/melo, checked at the caller); now the check lives in the workers themselves, per-model via `service_url` in the config plus a CODERAI_*_SERVICE_URL env var: vllm, ds4, kt, h3, pyannote, melotts, canary, crisperwhisper, parler Each verifies the remote's health check before returning it, so a stale or wrong URL fails loudly instead of surfacing as a confusing timeout later. vLLM additionally checks that the remote actually serves the requested model — the OCR path asks for surya-2 by name, and a generic remote vLLM would otherwise be proxied silently. colibri/k3 are left out: they speak a stdin/stdout mux protocol, not HTTP, so there is no URL to redirect. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Ships the LoRA trainers for LTX-2, MiniMax-H3 and Krea 2, and the overlay venv that lets the last two run without moving the server off diffusers 0.38. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
H3 and Krea 2 need diffusers >= 0.40 while the server runs 0.38. Upgrading in place is not free: measured on this box, SDXL comes out bit-identical but Z-Image drifts (mean |diff| 0.0146, max 0.32, against a same-version control that is bit-identical) — so the upgrade would silently change image output for a model that is working fine. Since diffusers is imported once per process, the split only has to be per-process. The H3 venv is now created with --system-site-packages and given nothing but diffusers>=0.40 and huggingface_hub>=1.23; torch, transformers and the rest are inherited from the server's venv. That is 75 MB instead of the several GB a standalone venv costs, and keeps one torch on the machine. CODERAI_H3_STANDALONE_VENV=1 still builds a fully independent venv from requirements-h3.txt, for the day the parent's torch is too old. Training follows the same boundary: tools/lora_train_worker.py runs one job in the overlay — references written out as PNGs, request flattened to JSON, progress returned as JSON lines and mirrored onto the usual /v1/loras/progress so polling is unchanged. _train_via_overlay() drives it. Routing is by capability, not hardcoded: _diffusers_has(arch) tries to load the classes, so wan and ltx2 train in-process (0.38 has them) while h3 and krea go out-of-process — and if the main venv is ever upgraded they move back in-process with no config change. Verified: the overlay resolves diffusers 0.40 with torch/transformers inherited and leaves the parent untouched; a probe job ran under it, reached _train_flow_dit and reported back through the job file. The trainers themselves are still unrun against real weights. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
We had trainers for SD1.x/SDXL, Z-Image and Wan — none for the architectures the LoRAs people actually publish for video today. All three new ones are the same training problem as Wan (encode references to VAE latents, encode the prompt once, adapt the transformer's attention projections, regress the rectified-flow velocity), so rather than a fourth, fifth and sixth copy of that loop there is one shared _train_flow_dit driven by a per-architecture description in codai/api/lora_archs.py: which classes to import, the latent shape, the timestep shift, the text-encoder contract. Selection: `target` accepts image | video | wan | ltx2 | h3 | krea. `video` keeps meaning Wan for existing configs but now reads the model id, so an LTX-2 or H3 path routes itself; the explicit names force one. The character studio exposes the same choice as --target and a dropdown. Availability differs per architecture and fails with a message that says so rather than an ImportError from inside a training job: LTX-2 works on this diffusers (0.38 has LTX2VideoTransformer3DModel + AutoencoderKLLTX2Video), while H3 and Krea 2 need >= 0.40 — and Krea 2 additionally needs access to the gated krea/Krea-2-* weights. Verified: architecture detection and routing for every target/model combination, and that each trainer's classes either load or explain what they need. NOT run against real weights for any of the three — the contracts match the proven Wan path, but the first real run should be treated as a debugging session. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Read at README level. Its pipeline is topic -> LLM script -> LLM search terms -> stock/AI footage -> TTS -> subtitles -> music -> assemble -> publish, with every heavy step delegated to a cloud vendor. Four real gaps on our side: no planner (our tools start from prompts you write), no stock footage source, no 9:16/1:1 presets, no social publishing. Subtitles, background music and assembly we already have. Recorded with a proposal that takes its shape and keeps the work local. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Folds in the tool-detachment work: GET /v1/audio/speech/capabilities and the video editor asking for it over HTTP instead of importing codai. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Audit for detaching the bundled tools into their own containers, talking to coderai only through the HTTP API (aisbf proxying 1:1, assumed). One coupling existed: video_editor.py imported codai.api.tts_backends to fill its emotion and style pickers, which only works while it shares a process tree with the server. Added GET /v1/audio/speech/capabilities and the editor now asks over HTTP, falling back to the local import on an older server and to empty lists when neither answers — the pickers simply don't appear rather than the tool breaking. Everything else was already clean: no tool sends a local path into the API expecting the server to read it (media goes base64/multipart), none reads coderai's own state (models.json, /AI, ~/.coderai), and generated media is always fetched over HTTP rather than from a shared filesystem. docs/tools-as-services.md records the audit, what each container would need (ffmpeg, own volumes, base-url/api-key, long proxy timeouts, large bodies) and which tools are not API clients at all — review_outputs.py reviews the tools' own output dirs, and the *_service.py workers are coderai's own subprocesses. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Ships the voice-cloning work: F5 engine caching and reference trimming, the engine choice on /v1/audio/clone (f5/xtts/chain/melotts), multi-clip voice profiles, AudioSeal watermarking, and MeloTTS as an isolated-venv worker. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Everything OpenVoice does that we didn't, built from models we already run rather than adopting theirs — plus MeloTTS itself so their exact V2 pairing is available. Engines on /v1/audio/clone (`engine`, default `auto`): f5 today's in-context clone — timbre AND prosody, needs the transcript xtts XTTS-v2 zero-shot from the clip, 17 languages, no transcript chain base TTS -> seed-vc timbre transfer: OpenVoice's architecture with seed-vc in place of its tone-colour converter melotts the same chain with MeloTTS as the base voice `auto` picks a transcript-free engine when the profile has no transcript or the target language differs from the reference — the case where F5 carries the wrong accent across. GET /v1/audio/clone/engines reports what is installed. The transcript is no longer a hard requirement: it is now checked per engine instead of rejecting the request outright. Multi-clip profiles: PATCH /v1/audio/voices/{name} takes add_clips/remove_clips, and a clone prompts on the cleanest take. Scored on clipping, level and crest factor — explicitly NOT loudest-wins, since a clipped take has the highest RMS of the set and is the worst possible prompt. Single-clip profiles keep working. Watermarking with AudioSeal (MIT), on by default, per-request or global override, detection at POST /v1/audio/watermark/detect. Missing audioseal degrades to unmarked audio with one warning, never an error. MeloTTS runs in an isolated venv worker like parler-tts — its pins conflict with this stack the same way. _RemoteParlerBackend generalised to _RemoteSpeechBackend since both speak the same POST /speak contract. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Two costs on every cloned line. F5TTS(device=…) was constructed per request, so the model and its vocoder were loaded from scratch for each line of dialogue — seconds of latency and a fresh VRAM allocation every time. seed-vc next door already keeps a singleton; F5 now does too, rebuilt only if the device changes, and registered with register_external_vram_releaser so the manager can reclaim it like the LoRA base cache and the OCR engines. And nothing capped the reference. F5-TTS prompts on the clip and gains nothing past roughly 15 s, while a saved profile can hold minutes — the character studio uploads 90 s. Both the /v1/audio/clone path and the dialog path now trim to MAX_REF_SECONDS (15, override with CODERAI_F5_MAX_REF_SECONDS), leaving anything already short untouched. Co-Authored-By:
-
- 12 Sep, 2026 14 commits
-
-
Stefy Lanza (nextime / spora ) authored
A resident model is no longer evicted to make room for itself. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
0.1.95 fixed the size — "sizing 'lisa-32k' by its own config" now fires and the requirement fell from 52.0 GB to the correct 22.9 GB — but the second request still evicted: Loading '…gemma-4-26B…gguf': need 22.9 GB VRAM, have 4.6 GB free — evicting On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB) Only 4.6 GB was free BECAUSE lisa-32k was resident and holding 22.9 GB of the card. The reservation asked whether there was room for a model that had already been loaded, found none, and evicted that very model to make room for it. It reaches ensure_vram_for at all because residency is checked under one id while the reservation arrives under another — self.models is keyed by the alias, the reservation by the weights path. So the guard checks every id the request answers to (model_key, resolved_name, requested_id) and returns early when any is resident. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Sibling configs are now told apart by their alias field. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Third attempt at this, and the first two failed for the same reason: they asked _config_for_model(requested_id) to find the sibling, but that helper resolves an alias THROUGH self.model_aliases to the shared weights path — landing right back on the wrong sibling. So the comparison was always "same object", the swap never happened, and 0.1.94 still logged [vram-est][debug] key='…/gemma-4-26B-A4B-…gguf' -> 51.97 GB | measured_cfg=51.969 Loading '…gguf': need 52.0 GB VRAM — evicting On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.47 GB) The estimator now scans self.config for an entry whose `alias` field IS the requested id. Aliases are unique per entry, so it needs no path comparison and doesn't depend on how self.config is keyed — which is what the earlier attempts got wrong. The path-based rule stays as a fallback for entries without an alias. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Same-path sibling sizing, now on the path text generation actually takes (get_model_for_request -> _load_model_by_name -> ensure_vram_for). Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The previous two attempts wired request_model — which only video and image generation use. Chat completions go through get_model_for_request -> _load_model_by_name -> ensure_vram_for, and that path collapses the alias the same way, so on 0.1.93 the second lisa-32k request still logged [vram-est] key='…/gemma-4-26B-A4B-…gguf' source=measured(force) -> 51.97 GB Loading '…gguf': need 52.0 GB VRAM, have 4.6 GB free — evicting models On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB) get_model_for_request now keeps the requested id, passes it to _load_model_by_name, and from there to both ensure_vram_for and the expected_vram_gb the backend uses to size its GPU-layer split — so lisa-32k is sized as itself (22.9 GB) rather than as lisa (51.97 GB). Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Completes the same-path sibling fix: the requested id is now kept through alias resolution, so lisa-32k is sized as itself instead of lisa's 51.97 GB. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The previous attempt at this didn't fire. request_model overwrites the requested name with the alias target on its first resolution step: if requested_model in self.model_aliases: requested_model = self.model_aliases[requested_model] so by the time anything looks a config up, 'lisa-32k' no longer exists — model_key and resolved_name are both the weights path, they compare equal, and the same-path-sibling check had nothing to compare. Confirmed live on 0.1.92: the second lisa-32k request still logged "need 52.0 GB VRAM, have 4.6 GB free" and evicted the model that was serving it. Keep the original id in _requested_id before that overwrite, hand it to ensure_vram_for at all three reservation sites, and let _get_model_used_vram_gb try it ahead of resolved_name. Against the live config: lisa-32k now sizes at 22.44 GB instead of lisa's 51.97 GB, and qwen38-128k at 19.50 instead of qwen38's 18.00. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
Adds the same-path sibling fix: a request for lisa-32k was sized with lisa's 51.97 GB footprint (same weights file, 178000 vs 32768 context), which evicted the model that was serving the request and reloaded it at 18 of 65 layers. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The second lisa-32k request evicted the model that was already serving it, then reloaded the same file at 18 of 65 layers: # >>> REQUEST model=lisa-32k [vram-est] key='/AI/guffcache/gemma-4-26B-…gguf' -> 51.97 GB Loading '/AI/guffcache/gemma-4-26B-…gguf': need 52.0 GB, have 12.6 GB free — evicting On-request VRAM eviction: unloading active model 'lisa-32k' (measured 14.33 GB) n_gpu_layers : 18 | n_ctx : 32768 lisa and lisa-32k are the same weights file at two contexts — 178000 and 32768 — so they measure 51.97 GB and 14.33 GB. The VRAM estimate was keyed by the PATH, which matches whichever sibling registered it, so the small config was sized with the big one's footprint: a 52 GB requirement that nothing could satisfy, an eviction of the running model, and a crippled reload. _get_model_used_vram_gb now prefers the config of the name actually requested when it names a same-path sibling. Four pairs on this box were affected, including the qwen38 / qwen38-128k pair added yesterday. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Ships the auto-offload headroom fix: a measured footprint already includes the KV cache, so reserving a context-scaled amount on top of it spilled layers to CPU that had room on the card. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
qwen38 (27B, n_ctx 32000, q4_0 KV) loaded with 48 of 66 layers on the GPU and 4.9 GB of weights on the CPU, for no reason: the auto-offload sizer reserves "2 GB + n_ctx/12000" = 4.7 GB of headroom for the KV cache, while llama.cpp's own numbers put the whole KV at 0.55 GB (CUDA0 421.88 MiB + CPU 140.62 MiB). With 20.9 GB free and ~17 GB really needed, everything fitted. Two assumptions behind that reserve are wrong when they don't hold: that the VRAM estimate is weights-only, and that the KV is f16. So the reserve is now 1.5 GB (compute buffers) when the config carries a measured footprint — that number already includes the KV, adding a context-scaled reserve on top double counts — and 1.5 GB + n_ctx/40000 when the KV is quantized. The old heuristic stays for a raw, unmeasured, f16 case. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Ships the studio's streaming uploads and local frame sampling, the KV-cache placement planner (with swa_full and the quantization ladder), split-GGUF shard resolution, and the n_ctx fallback that no longer lands on 2048. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The qwen38 request loaded with "n_ctx : 2048" and failed on any real prompt. The config says 32000 — but it was added to models.json after the engine started, so the engine's registry didn't have it. The front DID know the alias and routed the request, the backend resolved the alias to a local GGUF through its bare-path fallback, and with no n_ctx in the load kwargs the code took llama.cpp's 2048 default. Nothing in the log explained why a 27B model suddenly had a 2k window. So when no n_ctx arrives with the request: look the config up by the resolved path (the manager's _config_for_model already resolves id/alias/basename), and only if that finds nothing fall back to min(native context, 32768) with a warning naming both numbers. On all three models on this box that turns 2048 into 32768. The immediate cause still needs an engine reload to pick the new entries up; this is about the failure being silent and the floor being useless. Co-Authored-By:
-
- 11 Sep, 2026 5 commits
-
-
Stefy Lanza (nextime / spora ) authored
A request for Qwen3.8-Flash-Next failed six times with nothing but "Failed to load model from file". The path was ...-00006-of-00006.gguf: the LAST shard of a 6-part split GGUF. llama.cpp opens a split model at shard 00001 and finds the rest itself; handed any other shard it just fails. And the set was incomplete anyway — only shard 6 was on disk, the download having pulled a single shard rather than the set. So a .gguf path is now resolved before the load: a shard filename maps to shard 1, and the whole set is checked first. An incomplete set raises once, naming which shards are present and which are missing, instead of six identical retries of a message that says nothing. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Two levers were missing from the KV plan. The one that costs nothing: llama-cpp-python 0.3.30 accepts `swa_full`, and we never passed it. On a sliding-window model — gemma-4-31B has a 1024-token window on a 5:1 pattern — swa_full=True makes the sliding layers reserve the FULL context like the global ones. Turning it off reserves only the window, and it costs no quality: it limits context reuse across requests, not attention. So it is now both a config passthrough and the FIRST rung of the fallback ladder, ahead of touching precision at all. The other: the ladder was q8_0 then q4_0, symmetric. It is now three rungs — (q8_0,q8_0) 24.3 GB, (q8_0,q4_0) 18.6 GB, (q4_0,q4_0) 12.9 GB on tonight's measured 45.7 GB — keeping K more precise than V in the middle, since V tolerates quantization better. There is no rung below that: 4.5 bits (q4_0 / iq4_nl) is llama.cpp's floor for a KV cache. The sub-4-bit ggml types this build exposes (IQ2_*, Q2_K, IQ3_*) are weight quantizations, not KV cache types. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Four SIGKILLs tonight (19:01, 19:32, 20:42, 21:00), each at ~45 GB RSS on a 54 GB box, every one of them the nvidia-gguf engine mid-load — and with it every model resident on that engine, which is why unrelated requests 502'd. The chain: gemma-4-31B at n_ctx 128000 doesn't fit 24 GB of VRAM, llama.cpp fails to create the context, and the retry silently allocates the KV cache in HOST RAM (`CPU KV buffer size = 46750.00 MiB`). Nothing checked whether host RAM could take it. The existing auto-offload sizes layers by weight only; its KV headroom heuristic (2 GB + n_ctx/12000) predicted ~12.7 GB against a real 45.7. So size the KV first and pick a placement that fits: VRAM when there's room, host RAM when there isn't, a quantized KV when neither can hold it at f16, and a clean refusal when even that won't fit — a failed load beats an OOM-killed engine. The context is never shrunk; that stays the caller's decision. It acts only on a MEASURED size. Predicting KV from GGUF metadata is architecture-specific — Gemma-4 carries a per-layer head_count_kv array, separate SWA key/value lengths and a sliding-window pattern — and the naive formula came out 5x high while the SWA-aware one came out 4x low on this very model. So the load now parses llama.cpp's own "KV buffer size" lines, reports the real total split by where it landed, and prints the measured_kv_gb line to record; estimates are logged for visibility but never block a load. Known gap: the first load of a model at a given n_ctx has no measurement yet, so it can still be the one that OOMs. Tonight's value is known (45.7 GB at 128000) and can be recorded by hand to cover it. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Uploads were read in the browser with FileReader, base64'd, and posted as one JSON document holding every source — so a phone video was a ~700 MB string in the browser, another copy in the request, and another in the handler, and nginx's 1 MB default rejected it long before any of that mattered. Now the browser POSTs each File as a raw body to /api/upload, which streams it to disk a megabyte at a time and hands back a path; extraction takes paths. Nothing base64s client-side, and neither end ever holds a whole file. JSON bodies are capped at 32 MB (they only carry settings now) and the upload endpoint is dispatched before the body is parsed, since it is binary. Big videos are also frame-sampled LOCALLY before anything is sent: the extractor only ever looks at evenly-spaced frames, so shipping 24 of them instead of the film is the same result for a fraction of the traffic (--max-upload-mb, default 64; 0 sends the file). Oversized stills are downscaled to 2048px, and voice cloning now sends a 90s mono clip rather than the entire soundtrack. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Ships the character studio's LoRA training and voice cloning, and MiniMax-H3 support (in-engine when the engine's diffusers can do it, isolated venv worker otherwise). Co-Authored-By:
-
- 10 Sep, 2026 5 commits
-
-
Stefy Lanza (nextime / spora ) authored
H3 doesn't need a permanent sidecar. Model work already happens in a supervised, per-GPU engine subprocess with its own CUDA context, VRAM manager and swap gate — so H3 belongs there, registered with the model manager like a Wan or LTX pipeline, and pinned to a CUDA engine. What actually forced the worker was the version floor, not the architecture: diffusers ships H3 as Modular blocks only (0.40+), and this venv is on 0.38. So the route is now chosen by capability, not hardcoded — engine_supports_h3() probes for MiniMaxH3ModularPipeline, and `in_process` in the model config forces either side. The same build works before and after a diffusers upgrade, and the worker stays as the fallback while H3 is unproven (an in-engine OOM or device assert poisons the whole engine's context; a subprocess crash doesn't). _load_h3_pipeline() builds the modular pipeline with ONE workflow (loading with none pulls both 61.7GB partitions), group-offloads the big modules per component since a ModularPipeline has no pipeline-level offload helper, and registers it with record_vram_delta so eviction sees its real footprint. A workflow change unloads first — the two partitions can't be co-resident. The checkpoint contracts now live in tools/h3_common.py, loaded by path from the engine and imported as a sibling module inside the isolated venv, so both routes snap frames, round the canvas, pick the workflow and mux the audio identically. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
MiniMax-H3 is a 33B flow-matching DiT that denoises video and audio in ONE packed sequence. diffusers ships it as Modular blocks only (MiniMaxH3ModularPipeline, 0.40+) — there is no DiffusionPipeline half, so it fits neither _detect_pipeline_class() nor this venv's diffusers 0.38. It runs in an isolated venv, the same way pyannote, NeMo-Canary and vLLM do: tools/h3_service.py generation inside the isolated venv (HTTP) codai/api/h3_worker.py venv bootstrap, process lifecycle, eviction handle requirements-h3.txt the venv's pins (diffusers>=0.40, transformers 5.x) _generate_h3() adapts the request to the checkpoint's contracts instead of forwarding it, and reports every adaptation in the response warnings: guidance is baked in (negative_prompt and guidance_scale dropped), 24 fps is fixed and num_frames snaps up to the next 17n+5 within 5-15s, canvas axes round to 32, and v2v/interp are rejected. character_profiles become native ref2va references (max 9) so H3 skips the identity-keyframe bridge; an explicit keyframe wins, since fl2va and ref2va are separate transformer partitions. The worker picks a workflow per request and never loads without one — that pulls both 61.7GB partitions. The worker registers as an evictable, VRAM-tracked model; eviction stops the subprocess, the only way the transformer's pages actually come back. Character studio: clone the voice off the same footage the face came from (/v1/audio/voices/extract) with `extract --voice`, the new `voice` subcommand or the UI checkbox, then `--say` a line that is lip-synced in that cloned voice. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
IP-Adapter conditioning drifts on long shots and profile angles, so the studio now has the middle step: train a LoRA straight from the profile's reference crops and apply it to the generation. `POST /v1/loras/train` already resolves images server-side from `character: <name>`, so the trainer sees exactly the crops you reviewed and pruned — no re-upload. The job runs detached (`wait: false`); the studio polls `/v1/loras/progress`, persists the server job id per LoRA and re-attaches to a run still in flight after a client restart instead of starting a second one, resubmitting to resume from checkpoint when a job was interrupted. CLI: `train`, plus `--lora NAME[:WEIGHT]` / `--image-lora` on `video` and a `--train` flag on `demo` for extract -> train -> generate in one command. Web UI: a training panel (target, steps, rank, resolution, frames, LR, QLoRA toggle, cancel) and a LoRA multi-select on the scene panel. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The repo root accumulates `coderai-base-<date>.tar.gz` docker-save exports (~13 GB each, 79 GB today) plus LoRA `checkpoints/`, none of which .dockerignore excluded — every image build shipped them to the daemon as build context. Also excludes the character studio's local output dir. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Adds tools/character_studio.py, a demo app over the character-profile endpoints: extract a character from real photos/videos (POST /v1/characters/extract), review and prune the reference crops, then generate a brand-new video starring that person (POST /v1/video/generations with character_profiles + keyframe_identity). CLI core plus a `web` subcommand serving the same pipeline in the browser. The UI honours X-Forwarded-Prefix, so it works at the root or under a sub-path mount. Every generated mp4 gets its request body written alongside it, so the demo also doubles as a worked API example. Wired in as a bundled tool like the other UIs: supervisord program on port 7791 (autostart via CODERAI_TOOL_CHARACTER_STUDIO, on by default), entrypoint env defaults, run_oci.sh --enable/--disable-tool + --tool-arg names, nginx route at /character/, service smoke test, and the run banner/README URLs. Version bumped to 0.1.88 so the in-image upgrade path ships it. Co-Authored-By:
-
- 09 Sep, 2026 1 commit
-
-
Stefy Lanza (nextime / spora ) authored
Ships the audio pipeline work (real music dub, demucs separation, DeepFilterNet restoration), the video fixes (camera_motion, dialog voice cloning, managed Wav2Lip, IP-Adapter identity conditioning + keyframe bridge), the huggingface_hub compat shim, and the documentation refresh. Co-Authored-By:
-