• Stefy Lanza (nextime / spora )'s avatar
    video: serve MiniMax-H3 through an isolated worker, and clone voices in the studio · 4abca4a9
    Stefy Lanza (nextime / spora ) authored
    MiniMax-H3 is a 33B flow-matching DiT that denoises video and audio in ONE packed
    sequence. diffusers ships it as Modular blocks only (MiniMaxH3ModularPipeline,
    0.40+) — there is no DiffusionPipeline half, so it fits neither
    _detect_pipeline_class() nor this venv's diffusers 0.38. It runs in an isolated
    venv, the same way pyannote, NeMo-Canary and vLLM do:
    
      tools/h3_service.py   generation inside the isolated venv (HTTP)
      codai/api/h3_worker.py venv bootstrap, process lifecycle, eviction handle
      requirements-h3.txt   the venv's pins (diffusers>=0.40, transformers 5.x)
    
    _generate_h3() adapts the request to the checkpoint's contracts instead of
    forwarding it, and reports every adaptation in the response warnings: guidance is
    baked in (negative_prompt and guidance_scale dropped), 24 fps is fixed and
    num_frames snaps up to the next 17n+5 within 5-15s, canvas axes round to 32, and
    v2v/interp are rejected. character_profiles become native ref2va references (max
    9) so H3 skips the identity-keyframe bridge; an explicit keyframe wins, since
    fl2va and ref2va are separate transformer partitions. The worker picks a workflow
    per request and never loads without one — that pulls both 61.7GB partitions.
    
    The worker registers as an evictable, VRAM-tracked model; eviction stops the
    subprocess, the only way the transformer's pages actually come back.
    
    Character studio: clone the voice off the same footage the face came from
    (/v1/audio/voices/extract) with `extract --voice`, the new `voice` subcommand or
    the UI checkbox, then `--say` a line that is lip-synced in that cloned voice.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    4abca4a9
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...