-
Stefy Lanza (nextime / spora ) authored
Attention is the expensive part at video token counts — ~37k tokens for 93 frames at 480x832, and it is O(N^2) in that. The checkpoint already carries bsa_params with sparsity 0.9375, roughly a 16x cut, and upstream's implementation is pure Triton, so none of it needs the flash-attn source build. So it is now a setting: longcat.bsa server-wide, "bsa" per model, --bsa on the service. Off by default because it is an approximation and video shows approximation as temporal flicker, not one soft frame. "auto" stays off when triton is missing; "on" says so instead, because someone who asked for it would otherwise be left wondering why nothing got faster. BSA replaces the dense backend rather than joining it, since Attention tests enable_bsa first. And cp_split_hw, found while wiring this: the DiT's forward opens with `if self.cp_split_hw[0] * self.cp_split_hw[1] > 1:` and has no None guard, but dit/config.json ships "cp_split_hw": null — so a DiT built from the config as published raises TypeError on its first forward. Our only assignment made it worse by storing cp_size, a process count, where a [h, w] pair belongs: an int is not subscriptable either. It defaults to [1, 1] now, derives [1, cp_size] under context parallelism, and takes an explicit tile that is checked against the group size. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
3532636c