• Stefy Lanza (nextime / spora )'s avatar
    longcat: it generates (v0.2.70) · ac62f4b4
    Stefy Lanza (nextime / spora ) authored
    Two defects between a loading model and a finished clip, both found by running it.
    
    A repo id is not a directory. models.json holds what the model page registered —
    "meituan-longcat/LongCat-Video" — and resolve_model_path handed it straight to
    the service, which opens config.json, the VAE and the INT8 shards by path. Every
    load died with "checkpoint directory does not exist" before reading a byte.
    resolve_checkpoint() maps a repo id onto its newest snapshot in the hub cache,
    by hand rather than through huggingface_hub so the server's 3.13 venv and the
    service's 3.10 venv cannot disagree about where the weights are, and so a
    checkpoint that was never fetched stays an error instead of becoming a silent
    74 GB download.
    
    And the offload added in v0.2.67 was only half of one. It held the text encoder
    off the card until encode_prompt and then brought it over WITH the DiT still
    resident: 13.6 GB of INT8 DiT plus ~11 GB of UMT5-XXL is 24.8 GB on a 24 GB
    card. It OOM'd on the first prompt of the first generation. Model offload means
    only the active component is on the card, so they take turns now — the DiT steps
    off for the encode and comes back for the loop, in a finally, because a failed
    prompt that stranded the DiT on the host would leave every later generation
    running at PCIe speed with nothing to say why. Peak is max(), not sum.
    
    The placement test passed on the broken version: it asserted the encoder arrived
    and left, never that the DiT was not there too. It asserts co-residency now, and
    fails on the old code.
    
    Measured end to end on the 3090: 33 frames at 480x832, distilled stage, 16 steps
    at 9.4 s/it, 13.6 GB resident, HTTP 200 with the mp4.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ac62f4b4
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...