video-longcat.light
0 Bytes
-
Stefy Lanza (nextime / spora ) authored
Step 7, the last of the server-side work. Two things that are more than flags. CONTEXT PARALLELISM is N processes under torchrun, not N threads, and every rank must enter the same pipeline call or the collectives deadlock. So: - NCCL and init_context_parallel() run BEFORE the model loads — that call is what makes the DiT split its spatial dims, so load_pipeline cannot come first; - rank 0 owns the HTTP socket and BROADCASTS each job before entering the pipeline; the other ranks sit in cp_worker_loop waiting for one and discard their copy of the result. A rank-0-only server that simply called generate() would hang on the first collective; - the group is told to shut down when the server stops, or the non-zero ranks outlive it; - cp_size 1 (the default) initialises no distributed group at all; - asking for more ranks than there are visible GPUs is refused up front, because NCCL's own failure for that names nothing useful. THE POD IMAGE could not use the generic builder: Dockerfile.capability is python:3.12 on the cu128 torch index, and LongCat needs 3.10 on cu124 — two interpreters in one image, the light core running coderai and a 3.10 venv running the model. So video-longcat carries its own Dockerfile (the escape hatch the engines profile uses), builds the venv from the SAME requirements-longcat.txt a local install uses, asserts 3.10 / torch 2.6 / transformers 4.41 at build time rather than discovering a different resolution on a rented pod, clones the pipeline source (it is not on PyPI) and checks it imports, and bakes no weights. model_capability() now routes a `backend: longcat` entry to that image. Left alone it would land on the plain 'video' image, which cannot run LongCat at all — a pod that boots, passes its health check, accepts the request and then has nothing to generate with, exactly the trap the OCR profile documents for Surya. An explicit `capability` override still wins. 24 new tests, 212 across the seven LongCat suites. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
759892c4