• Stefy Lanza (nextime / spora )'s avatar
    longcat: pick the attention backend that is installed (v0.2.68) · a2b991e5
    Stefy Lanza (nextime / spora ) authored
    dit/config.json asks for enable_flashattn2, but requirements-longcat.txt ships
    xformers — flash-attn 2.7.4.post1 is a source build needing ninja and a CUDA
    toolchain. Upstream's Attention has no fallback: anything unselected reaches
    `raise RuntimeError("Unsupported attention operations.")`, and the FA2 branch
    imports flash_attn, which is not in the venv.
    
    --attention was only ever written into the environment. Nothing read
    LONGCAT_ATTENTION, so the checkpoint's config won every time.
    
    Now both load paths — bf16 through from_pretrained and INT8 through the config
    dict — are built with the backend that is actually importable, preferring what
    was asked for and saying so when it falls back. None installed is an error
    here, with the reason, rather than 48 blocks into the first step of a load that
    already cost minutes.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    a2b991e5
__init__.py 4.21 KB