• Stefy Lanza (nextime / spora )'s avatar
    a GGUF component is pre-built too, so placement must treat it as one · bcdd5ffb
    Stefy Lanza (nextime / spora ) authored
    video.py already knew the hazard and wrote it down: accelerate's device_map
    only dispatches components it LOADS, so one handed to from_pretrained keeps
    whatever device it is on with no offload hook, and the forward pass hits
    "index on cuda:0, weights on cpu". It acted on that only for pipeline-cache
    injections, while a GGUF component arrives exactly the same way.
    
    LTX-2.5 therefore failed under every strategy it tried -- auto, balanced,
    model and pure-disk, three restarts and half an hour each -- always with
    
        Expected all tensors to be on the same device, but got index is on
        cuda:0, different from other tensors on cpu
    
    and video_resident_experts, the flag that exists for precisely this, was
    never consulted because the gate in front of it was false.
    
    An explicit offload_strategy is still respected; only the strategies that let
    accelerate place components itself are overridden.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
    bcdd5ffb
test_gguf_components_are_injected.py 2.07 KB