• Stefy Lanza (nextime / spora )'s avatar
    longcat: context-parallel multi-GPU and the video-longcat pod image (v0.2.36) · 759892c4
    Stefy Lanza (nextime / spora ) authored
    Step 7, the last of the server-side work. Two things that are more than flags.
    
    CONTEXT PARALLELISM is N processes under torchrun, not N threads, and every rank must
    enter the same pipeline call or the collectives deadlock. So:
    - NCCL and init_context_parallel() run BEFORE the model loads — that call is what makes
      the DiT split its spatial dims, so load_pipeline cannot come first;
    - rank 0 owns the HTTP socket and BROADCASTS each job before entering the pipeline; the
      other ranks sit in cp_worker_loop waiting for one and discard their copy of the result.
      A rank-0-only server that simply called generate() would hang on the first collective;
    - the group is told to shut down when the server stops, or the non-zero ranks outlive it;
    - cp_size 1 (the default) initialises no distributed group at all;
    - asking for more ranks than there are visible GPUs is refused up front, because NCCL's
      own failure for that names nothing useful.
    
    THE POD IMAGE could not use the generic builder: Dockerfile.capability is python:3.12 on
    the cu128 torch index, and LongCat needs 3.10 on cu124 — two interpreters in one image,
    the light core running coderai and a 3.10 venv running the model. So video-longcat
    carries its own Dockerfile (the escape hatch the engines profile uses), builds the venv
    from the SAME requirements-longcat.txt a local install uses, asserts 3.10 / torch 2.6 /
    transformers 4.41 at build time rather than discovering a different resolution on a
    rented pod, clones the pipeline source (it is not on PyPI) and checks it imports, and
    bakes no weights.
    
    model_capability() now routes a `backend: longcat` entry to that image. Left alone it
    would land on the plain 'video' image, which cannot run LongCat at all — a pod that
    boots, passes its health check, accepts the request and then has nothing to generate
    with, exactly the trap the OCR profile documents for Surya. An explicit `capability`
    override still wins.
    
    24 new tests, 212 across the seven LongCat suites.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    759892c4
Name
Last commit
Last update
..
profiles Loading commit data...
Dockerfile.capability Loading commit data...
Dockerfile.capability-base Loading commit data...
Dockerfile.capability-base-light Loading commit data...
Dockerfile.capability-engines Loading commit data...
Dockerfile.capability-llama Loading commit data...
Dockerfile.capability-slim Loading commit data...
Dockerfile.capability-video-longcat Loading commit data...
boot.sh Loading commit data...
build_all_capability_images.sh Loading commit data...
build_capability_image.sh Loading commit data...
check_llama_wheel.py Loading commit data...
publish_capability_images.sh Loading commit data...