Skip to content
Commit 049e2c2a authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

longcat: encode the prompt once per request, not once per segment (v0.2.76)



_run_segments calls generate_t2v once PER SEGMENT, and every one of those calls
self.encode_prompt. A one-minute video at 15 fps is 12 segments, so that was 12
identical encodes of the same prompt — and under offload each one also moved the
13.6 GB DiT off the card and the ~11 GB text encoder on and back again. About
35 GB over PCIe per segment, twelve times, for an answer that cannot change.

The embeddings are a few MB, so they are now cached on the arguments that
produce them, bounded to the last four. Segment 2 onward never touches the
encoder: for the rest of a long generation the card holds nothing but the DiT
and the VAE. Caching applies without offload too, where it saves the redundant
forwards if not the transfers.

The wrapper passes its arguments through verbatim instead of naming the first
two — naming them baked in an assumption about upstream's signature that the
avatar pipeline need not share, and it collided on the first test that used a
differently-shaped one.

Co-Authored-By: default avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
parent 998a810c
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment