• Stefy Lanza (nextime / spora )'s avatar
    township: build the LongCat training dataset from the clips it rendered (v0.2.42) · 2015cdf1
    Stefy Lanza (nextime / spora ) authored
    Step 4b failed in production:
    
      RuntimeError: longcat LoRA training needs `dataset_config` — a path to a
      SimpleTuner data-backend config describing your captioned video clips
    
    The refusal is right: LongCat trains through SimpleTuner, which wants captioned
    VIDEO clips, and synthesising a dataset from a fighter's reference stills would
    train something other than a video LoRA. But the UI offered no way to supply one,
    so the button could not succeed — and the tool was already sitting on exactly the
    corpus SimpleTuner asks for. Every clip it renders was rendered FROM a prompt, and
    that prompt is kept beside it in videos/prompts.json. So it now assembles the
    dataset rather than demanding it.
    
    Per fighter and per location, against a LongCat video model:
    
      <out>/video_datasets/character_<name>__<slug>/
        clips/0000.mp4 -> ../../../videos/match_a_vs_b_clip03.mp4
        clips/0000.txt            the prompt that produced those pixels
        cache/vae  cache/text
        data_backend.json         sent as dataset_config
        manifest.json             which render each clip came from
    
    Decisions worth stating:
    
    - SYMLINKS, not copies. A fighter appears in dozens of clips; a dataset per
      fighter would otherwise duplicate the whole render per fighter.
    - Captions come from the PLAN, never from filenames, so a caption is always the
      text that actually produced the pixels.
    - _2x/_2xfps derivatives are never picked up. They are the upscaler's and the
      frame interpolator's output; training on them teaches their artefacts.
    - An outcome clip is training data for BOTH fighters — they are both on screen.
    - Clips shorter than the training frame count are filtered out HERE, because
      SimpleTuner's loader drops them silently. A dataset where every clip is too
      short refuses with the numbers: "all 12 clips are shorter than 93 frames
      (longest 70)". Likewise "only 3 usable clips, need 8" and "no rendered clips
      yet (run the Videos step first)" — a skip with a reason, not a stack trace.
    - Rebuilt from scratch each time, so a symlink to a deleted render cannot fail
      deep inside SimpleTuner's loader instead of here.
    
    Two knobs on the Run page, in the config and on the CLI: Training clip frames
    (default 93 — one whole LongCat segment, and 4n+1 as the VAE requires) and
    Minimum usable clips (default 8; upstream suggests 50-100, this is a floor so a
    part-rendered show does not quietly train on three).
    
    The backend schema is upstream's documented one, including exactly one
    text_embeds backend carrying default: true. The client sends dataset_config only
    when it has one; the server has always accepted the field and the in-process Wan
    path ignores it.
    
    Checked against the real township render: 151 sources (102 clips + 49 outcomes),
    19 usable for the busiest fighter at 93 frames. 82 tests.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    2015cdf1
longcat-video.md 13.9 KB