• Stefy Lanza (nextime / spora )'s avatar
    longcat: it generates (v0.2.70) · ac62f4b4
    Stefy Lanza (nextime / spora ) authored
    Two defects between a loading model and a finished clip, both found by running it.
    
    A repo id is not a directory. models.json holds what the model page registered —
    "meituan-longcat/LongCat-Video" — and resolve_model_path handed it straight to
    the service, which opens config.json, the VAE and the INT8 shards by path. Every
    load died with "checkpoint directory does not exist" before reading a byte.
    resolve_checkpoint() maps a repo id onto its newest snapshot in the hub cache,
    by hand rather than through huggingface_hub so the server's 3.13 venv and the
    service's 3.10 venv cannot disagree about where the weights are, and so a
    checkpoint that was never fetched stays an error instead of becoming a silent
    74 GB download.
    
    And the offload added in v0.2.67 was only half of one. It held the text encoder
    off the card until encode_prompt and then brought it over WITH the DiT still
    resident: 13.6 GB of INT8 DiT plus ~11 GB of UMT5-XXL is 24.8 GB on a 24 GB
    card. It OOM'd on the first prompt of the first generation. Model offload means
    only the active component is on the card, so they take turns now — the DiT steps
    off for the encode and comes back for the loop, in a finally, because a failed
    prompt that stranded the DiT on the host would leave every later generation
    running at PCIe speed with nothing to say why. Peak is max(), not sum.
    
    The placement test passed on the broken version: it asserted the encoder arrived
    and left, never that the DiT was not there too. It asserts co-residency now, and
    fails on the old code.
    
    Measured end to end on the 3090: 33 frames at 480x832, distilled stage, 16 steps
    at 9.4 s/it, 13.6 GB resident, HTTP 200 with the mp4.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ac62f4b4
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...