- Sep 16, 2026
-
-
Stefy Lanza (nextime / spora ) authored
The HF token fix worked: pyannote got past the gate and failed one layer deeper, instantiating the pipeline. Reproduced by running the real service with a GPU: ModuleNotFoundError: No module named 'matplotlib'. pyannote's segmentation tasks import matplotlib.pyplot at module load; it is not declared, and 'import pyannote.audio' never reaches it, so the build's import check passed and the venv shipped broken. This is the fourth venv whose bare import check passed while the real load failed. A <profile>.venv-<name>.check script now runs at build time in the venv's own python and imports what the REAL load reaches — pipeline modules, model classes — without a GPU or a token, which a build has neither of. The pyannote check fails on the shipped image and passes with the fix; the XTTS check passes as-is, so that venv is genuinely complete. Also: every isolated-venv worker reported failures as the LAST FIVE LINES of the subprocess tail. A Python traceback's last five lines are frame headers — the exception is the final line, and cutting to five dropped it whenever the trace was deep. Ten workers said where a process died and never why. Fixed in all of them. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
-
- Sep 13, 2026
-
-
Stefy Lanza (nextime / spora ) authored
Each `ensure_service()` already returns a base URL, so a worker can be pointed at a service running elsewhere — another host, a container, a rented RunPod pod — without anything downstream noticing. Only the TTS backends had that escape hatch (parler/melo, checked at the caller); now the check lives in the workers themselves, per-model via `service_url` in the config plus a CODERAI_*_SERVICE_URL env var: vllm, ds4, kt, h3, pyannote, melotts, canary, crisperwhisper, parler Each verifies the remote's health check before returning it, so a stale or wrong URL fails loudly instead of surfacing as a confusing timeout later. vLLM additionally checks that the remote actually serves the requested model — the OCR path asks for surya-2 by name, and a generic remote vLLM would otherwise be proxied silently. colibri/k3 are left out: they speak a stdin/stdout mux protocol, not HTTP, so there is no URL to redirect. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
H3 and Krea 2 need diffusers >= 0.40 while the server runs 0.38. Upgrading in place is not free: measured on this box, SDXL comes out bit-identical but Z-Image drifts (mean |diff| 0.0146, max 0.32, against a same-version control that is bit-identical) — so the upgrade would silently change image output for a model that is working fine. Since diffusers is imported once per process, the split only has to be per-process. The H3 venv is now created with --system-site-packages and given nothing but diffusers>=0.40 and huggingface_hub>=1.23; torch, transformers and the rest are inherited from the server's venv. That is 75 MB instead of the several GB a standalone venv costs, and keeps one torch on the machine. CODERAI_H3_STANDALONE_VENV=1 still builds a fully independent venv from requirements-h3.txt, for the day the parent's torch is too old. Training follows the same boundary: tools/lora_train_worker.py runs one job in the overlay — re...
-
- Sep 10, 2026
-
-
Stefy Lanza (nextime / spora ) authored
H3 doesn't need a permanent sidecar. Model work already happens in a supervised, per-GPU engine subprocess with its own CUDA context, VRAM manager and swap gate — so H3 belongs there, registered with the model manager like a Wan or LTX pipeline, and pinned to a CUDA engine. What actually forced the worker was the version floor, not the architecture: diffusers ships H3 as Modular blocks only (0.40+), and this venv is on 0.38. So the route is now chosen by capability, not hardcoded — engine_supports_h3() probes for MiniMaxH3ModularPipeline, and `in_process` in the model config forces either side. The same build works before and after a diffusers upgrade, and the worker stays as the fallback while H3 is unproven (an in-engine OOM or device assert poisons the whole engine's context; a subprocess crash doesn't). _load_h3_pipeline() builds the modular pipeline with ONE workflow (loading with none pulls both 61.7GB partitions), group-offloads the big modules per component since a ModularPipeline has no pipeline-level offload helper, and registers it with record_vram_delta so eviction sees its real footprint. A workflow change unloads first — the two partitions can't be co-resident. The checkpoint contracts now live in tools/h3_common.py, loaded by path from the engine and imported as a sibling module inside the isolated venv, so both routes snap frames, round the canvas, pick the workflow and mux the audio identically. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
MiniMax-H3 is a 33B flow-matching DiT that denoises video and audio in ONE packed sequence. diffusers ships it as Modular blocks only (MiniMaxH3ModularPipeline, 0.40+) — there is no DiffusionPipeline half, so it fits neither _detect_pipeline_class() nor this venv's diffusers 0.38. It runs in an isolated venv, the same way pyannote, NeMo-Canary and vLLM do: tools/h3_service.py generation inside the isolated venv (HTTP) codai/api/h3_worker.py venv bootstrap, process lifecycle, eviction handle requirements-h3.txt the venv's pins (diffusers>=0.40, transformers 5.x) _generate_h3() adapts the request to the checkpoint's contracts instead of forwarding it, and reports every adaptation in the response warnings: guidance is baked in (negative_prompt and guidance_scale dropped), 24 fps is fixed and num_frames snaps up to the next 17n+5 within 5-15s, canvas axes round to 32, and v2v/interp are rejected. character_profiles become native ref2va references (max 9) so H3 skips the identity-keyframe bridge; an explicit keyframe wins, since fl2va and ref2va are separate transformer partitions. The worker picks a workflow per request and never loads without one — that pulls both 61.7GB partitions. The worker registers as an evictable, VRAM-tracked model; eviction stops the subprocess, the only way the transformer's pages actually come back. Character studio: clone the voice off the same footage the face came from (/v1/audio/voices/extract) with `extract --voice`, the new `voice` subcommand or the UI checkbox, then `--say` a line that is lip-synced in that cloned voice. Co-Authored-By:
-
