• Stefy Lanza (nextime / spora )'s avatar
    township: build the LongCat training dataset from the clips it rendered (v0.2.42) · 2015cdf1
    Stefy Lanza (nextime / spora ) authored
    Step 4b failed in production:
    
      RuntimeError: longcat LoRA training needs `dataset_config` — a path to a
      SimpleTuner data-backend config describing your captioned video clips
    
    The refusal is right: LongCat trains through SimpleTuner, which wants captioned
    VIDEO clips, and synthesising a dataset from a fighter's reference stills would
    train something other than a video LoRA. But the UI offered no way to supply one,
    so the button could not succeed — and the tool was already sitting on exactly the
    corpus SimpleTuner asks for. Every clip it renders was rendered FROM a prompt, and
    that prompt is kept beside it in videos/prompts.json. So it now assembles the
    dataset rather than demanding it.
    
    Per fighter and per location, against a LongCat video model:
    
      <out>/video_datasets/character_<name>__<slug>/
        clips/0000.mp4 -> ../../../videos/match_a_vs_b_clip03.mp4
        clips/0000.txt            the prompt that produced those pixels
        cache/vae  cache/text
        data_backend.json         sent as dataset_config
        manifest.json             which render each clip came from
    
    Decisions worth stating:
    
    - SYMLINKS, not copies. A fighter appears in dozens of clips; a dataset per
      fighter would otherwise duplicate the whole render per fighter.
    - Captions come from the PLAN, never from filenames, so a caption is always the
      text that actually produced the pixels.
    - _2x/_2xfps derivatives are never picked up. They are the upscaler's and the
      frame interpolator's output; training on them teaches their artefacts.
    - An outcome clip is training data for BOTH fighters — they are both on screen.
    - Clips shorter than the training frame count are filtered out HERE, because
      SimpleTuner's loader drops them silently. A dataset where every clip is too
      short refuses with the numbers: "all 12 clips are shorter than 93 frames
      (longest 70)". Likewise "only 3 usable clips, need 8" and "no rendered clips
      yet (run the Videos step first)" — a skip with a reason, not a stack trace.
    - Rebuilt from scratch each time, so a symlink to a deleted render cannot fail
      deep inside SimpleTuner's loader instead of here.
    
    Two knobs on the Run page, in the config and on the CLI: Training clip frames
    (default 93 — one whole LongCat segment, and 4n+1 as the VAE requires) and
    Minimum usable clips (default 8; upstream suggests 50-100, this is a floor so a
    part-rendered show does not quietly train on three).
    
    The backend schema is upstream's documented one, including exactly one
    text_embeds backend carrying default: true. The client sends dataset_config only
    when it has one; the server has always accepted the field and the in-process Wan
    path ignores it.
    
    Checked against the real township render: 151 sources (102 clips + 49 outcomes),
    19 usable for the busiest fighter at 93 frames. 82 tests.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    2015cdf1
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...