longcat: a conditioning pass tiles twice (v0.3.3)

An i2v generation died in the first denoising step with a bare AssertionError out
of flash_attn_bsa_3d, on a geometry the guard had just declared valid: 93 frames,
sides divisible by 64.

bsa_problems only ever checked the noise branch. A conditioning pass does not tile
the latent once — Attention.forward splits the sequence into a conditioning block
and a noise block and calls flash_attn_bsa_3d on each (modules/attention.py:123-134),
so the cond depth and the remaining noise depth must divide by the chunk as well.
generate_i2v hardcodes one conditioning latent (pipeline:796) and 1 % 4 != 0, which
means block-sparse attention has never once been able to run an i2v pass, at any
frame count or resolution. The avatar pipeline hardcodes the same 1 in four places,
and generate_vc computes a count it never rounds. Only generate_refine gets it
right, by padding both counts to the granularity itself (pipeline:1245-1250).

(a) The guard now knows about the split. _cond_latent_passes enumerates every
conditioning count the request can produce — segment 0 uses the task's own entry
point and every later segment uses generate_vc, so even t2v runs a conditioning
pass once num_segments > 1, and BSA is one flag for the whole request — and a
count that cannot tile puts the pass on dense attention with a line saying which
condition failed.

(b) bsa_pad_cond, OFF by default, rounds the count up to the granularity on the way
into the DiT, which is what generate_refine does for itself, so an i2v or
continuation pass can keep BSA. Patched onto the DiT instance rather than the
vendored source, the same way the denoising bars are rebound: the count is born
inside generate_i2v's loop, so there is no outer argument to correct. Off by
default because the pipeline marks only the real conditioning frames clean
(timestep[:, :1] = 0), so rounding 1 up to 4 hands the conditioning block three
latent frames that are still noise — well-defined attention, but not what the model
was trained on, and on a video model that kind of error shows up as temporal drift.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.2"
+__version__ = "0.3.3"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/admin/routes.py

@@ -2959,7 +2959,8 @@ async def api_model_configure(request: Request, username: str = Depends(require_
                 
                     # LongCat component quantisation, block-sparse attention, the base
                     # checkpoint an avatar family borrows from, and the CP tile.
-                    "text_encoder_quant", "bsa", "base_model", "cp_split_hw",
+                    "text_encoder_quant", "bsa", "bsa_pad_cond", "base_model",
+                    "cp_split_hw",
                     # The prompt-embedding cache: an encode outlives the request that
                     # paid for it, and the text encoder is then built only on a miss.
                     "prompt_cache", "prompt_cache_dir", "prompt_cache_max_gb",):

codai/admin/templates/models.html

@@ -718,6 +718,11 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
               <option value="on">on — needs sides /64 and 16n+13 frames</option>
               <option value="auto">auto — on when triton is available</option>
             </select></div>
+          <div class="form-row" style="margin:0"><label class="form-label">Pad conditioning for BSA</label>
+            <select id="cfg-longcat-bsa-pad-cond" class="form-input">
+              <option value="">off — i2v/continuation run dense (safe)</option>
+              <option value="on">on — round cond latents up so BSA applies (experimental)</option>
+            </select></div>
         </div>
         <div style="display:grid;grid-template-columns:1fr 1fr 1fr;gap:.6rem;margin-top:.5rem">
           <div class="form-row" style="margin:0"><label class="form-label">Prompt cache</label>
@@ -4642,6 +4647,7 @@ function _populateCluster(m, s){
   set('cfg-longcat-offload', s.offload_strategy || '');
   set('cfg-longcat-te-quant', s.text_encoder_quant || '');
   set('cfg-longcat-bsa', s.bsa || '');
+  set('cfg-longcat-bsa-pad-cond', s.bsa_pad_cond || '');
   set('cfg-longcat-prompt-cache', s.prompt_cache || '');
   set('cfg-longcat-prompt-cache-dir', s.prompt_cache_dir || '');
   set('cfg-longcat-prompt-cache-max', s.prompt_cache_max_gb || '');
@@ -4692,6 +4698,7 @@ function _collectCluster(){
     out.offload_strategy = val('cfg-longcat-offload') || '';
     out.text_encoder_quant = val('cfg-longcat-te-quant') || '';
     out.bsa = val('cfg-longcat-bsa') || '';
+    out.bsa_pad_cond = val('cfg-longcat-bsa-pad-cond') || '';
     out.prompt_cache = val('cfg-longcat-prompt-cache') || '';
     out.prompt_cache_dir = val('cfg-longcat-prompt-cache-dir') || '';
     out.prompt_cache_max_gb = num('cfg-longcat-prompt-cache-max');

codai/api/longcat_worker.py

@@ -45,6 +45,10 @@ Per-model config keys (models.json entry, all optional)::
     "quality": "fast",             # draft | fast | best
     "offload_strategy": "",        # '' | model | sequential
     "bsa": "off",                  # block-sparse attention: on | off | auto
+    "bsa_pad_cond": "off",         # on | off — let i2v/continuation passes use BSA by
+                                   #   rounding the conditioning latents up to the
+                                   #   granularity; off, because it widens the
+                                   #   conditioning block past what is actually clean
     "base_model": "",              # avatar families borrow tokenizer/text_encoder/vae
     "text_encoder_quant": "none",   # none | int8 | nf4 | fp4 (keeps it resident)
     "prompt_cache": "off",         # off | disk — cache prompt embeddings between
@@ -498,6 +502,12 @@ def ensure_service(model_path: str, config: dict = None,
                   (getattr(sec, "bsa", "") if sec is not None else "") or "").strip()
         if bsa:
             cmd += ["--bsa", bsa]
+        # Only meaningful with bsa on, and the service validates it either way.
+        pad = str(config.get("bsa_pad_cond") or
+                  (getattr(sec, "bsa_pad_cond", "") if sec is not None else "")
+                  or "").strip()
+        if pad:
+            cmd += ["--bsa-pad-cond", pad]
         split = str(config.get("cp_split_hw") or "").strip()
         if split:
             cmd += ["--cp-split-hw", split]

docs/longcat-video.md

@@ -74,6 +74,45 @@ checkout and venv, and the avatar fields. Global runtime settings (venv, the 3.1
 interpreter, `auto_build`, attention backend, timeouts, the eviction drain) live in
 **Settings → LongCat**.
 
+## Block-sparse attention
+
+Attention dominates at video token counts — ~37k tokens for 93 frames at 480x832, and it
+is O(N^2) in that — so the checkpoint carries `bsa_params` with sparsity 0.9375, roughly a
+16x cut, implemented in pure Triton. **Block-sparse attention** on the model page turns it
+on. It ships off: it is an approximation, and on a video model the artifacts show up as
+temporal flicker rather than one soft frame.
+
+It also cannot run every geometry. `flash_attn_bsa_3d` tiles the latent into 3D chunks and
+asserts the latent divides evenly by the chunk on every axis, which with the shipped
+`(4, 4, 4)` means:
+
+- **sides divisible by 64** — the VAE's 16-pixel latent cell times 4. Upstream's own
+  default 480x832 is *not* one (480/16 = 30).
+- **frame counts of 16n+13** — 13, 29, 45, 61, 77, 93, … not the 4n+1 the generation rule
+  suggests, because the latent depth is `(F-1)/4 + 1` and *that* is what must divide by 4.
+
+And there is a third condition that is easy to miss. A **conditioning** pass — i2v, or any
+continuation — does not tile the latent once: `Attention.forward` splits the sequence into
+a conditioning block and a noise block and tiles each separately, so both depths must
+divide by the chunk too. `generate_i2v` hardcodes a single conditioning latent, and
+`1 % 4 != 0`, so block-sparse attention can never run an i2v pass at any frame count or
+resolution. coderai detects all three before the first denoising step and runs that pass
+**dense**, with a log line saying which condition failed — rather than letting a bare
+`AssertionError` surface from inside the DiT.
+
+### Padding the conditioning (experimental, off)
+
+**Pad conditioning for BSA** rounds the conditioning latent count up to the chunk
+granularity on the way into the DiT, which is exactly what upstream's own `generate_refine`
+does for itself. With it on, an i2v or continuation pass keeps block-sparse attention
+instead of falling back.
+
+It is off by default, and it is not free: the pipeline marks only the real conditioning
+frames clean (`timestep[:, :1] = 0`), so rounding 1 up to 4 hands the conditioning block
+three latent frames that are still noise. The attention is well-defined but it is not what
+the model was trained on. Measure a known-good clip against the dense path before trusting
+it on anything you care about.
+
 ## Generating
 
 ```jsonc

tools/longcat_common.py

@@ -286,7 +286,35 @@ VAE_TEMPORAL_STRIDE = 4
 DEFAULT_BSA_CHUNK = (4, 4, 4)
 
 
-def bsa_problems(num_frames: int, height: int, width: int, chunk=None) -> list:
+# What generate_i2v passes, hardcoded, whatever num_frames is
+# (pipeline_longcat_video.py:796; the avatar pipeline does the same at 1076-1113).
+I2V_COND_LATENTS = 1
+
+
+def cond_latents_for(num_cond_frames) -> int:
+    """How many latent frames a continuation pass treats as conditioning.
+
+    Mirrors pipeline_longcat_video.py:1016 — and, like it, does NOT round up to the
+    BSA granularity the way generate_refine does at 1245.
+    """
+    try:
+        frames = int(num_cond_frames)
+    except (TypeError, ValueError):
+        return 0
+    if frames <= 0:
+        return 0
+    return 1 + (frames - 1) // VAE_TEMPORAL_STRIDE
+
+
+def round_to_granularity(count, chunk=None) -> int:
+    """``count`` rounded UP to the BSA temporal granularity, as generate_refine does."""
+    tq = int((tuple(chunk or DEFAULT_BSA_CHUNK))[0]) or 1
+    n = max(0, int(count))
+    return -(-n // tq) * tq
+
+
+def bsa_problems(num_frames: int, height: int, width: int, chunk=None,
+                 num_cond_latents=None) -> list:
     """Why block-sparse attention cannot run THIS geometry, or [].
 
     flash_attn_bsa_3d tiles the latent into 3D chunks and asserts the latent divides
@@ -299,6 +327,15 @@ def bsa_problems(num_frames: int, height: int, width: int, chunk=None) -> list:
     num_frames at 13, 29, 45, 61, 77, 93 … Upstream's own default 480x832 is NOT one of
     them (480/16 = 30), which is presumably why the flag ships off: an AssertionError
     out of the first denoising step says nothing about geometry.
+
+    ``num_cond_latents`` is the OTHER half of the same assert, and the half that cost a
+    production run. A conditioning pass does not tile the latent once: Attention.forward
+    slices the sequence into a conditioning block and a noise block and calls
+    flash_attn_bsa_3d on EACH (modules/attention.py:123-134), so the cond depth and the
+    remaining noise depth must both divide by ``tq`` as well. generate_i2v hardcodes one
+    cond latent and 1 % 4 != 0, which means block-sparse attention has never once been
+    able to run an i2v pass at any frame count or resolution. Pass 0 or None for a pass
+    with no conditioning branch.
     """
     tq, hq, wq = tuple(chunk or DEFAULT_BSA_CHUNK)
     problems = []
@@ -329,6 +366,18 @@ def bsa_problems(num_frames: int, height: int, width: int, chunk=None) -> list:
             f"num_frames must leave a latent depth divisible by {tq} for block-sparse "
             f"attention ({step}n+1: 13, 29, 45, 61, 77, 93 …); got {frames}"
             + (f", try {' or '.join(str(n) for n in options)}" if options else ""))
+
+    if num_cond_latents:
+        cond = int(num_cond_latents)
+        noise = latent_t - cond
+        if cond % tq or noise <= 0 or noise % tq:
+            problems.append(
+                f"a conditioning pass tiles {cond} cond + {noise} noise latent frames "
+                f"separately, so BOTH must divide by {tq} for block-sparse attention; "
+                f"generate_i2v hardcodes {I2V_COND_LATENTS}, which never can. Turn "
+                f"bsa_pad_cond on to round the conditioning up to "
+                f"{round_to_granularity(cond, chunk)}, or leave block-sparse attention "
+                f"off for conditioning passes")
     return problems