• Stefy Lanza (nextime / spora )'s avatar
    a GGUF component is pre-built too, so placement must treat it as one · bcdd5ffb
    Stefy Lanza (nextime / spora ) authored
    video.py already knew the hazard and wrote it down: accelerate's device_map
    only dispatches components it LOADS, so one handed to from_pretrained keeps
    whatever device it is on with no offload hook, and the forward pass hits
    "index on cuda:0, weights on cpu". It acted on that only for pipeline-cache
    injections, while a GGUF component arrives exactly the same way.
    
    LTX-2.5 therefore failed under every strategy it tried -- auto, balanced,
    model and pure-disk, three restarts and half an hour each -- always with
    
        Expected all tensors to be on the same device, but got index is on
        cuda:0, different from other tensors on cpu
    
    and video_resident_experts, the flag that exists for precisely this, was
    never consulted because the gate in front of it was false.
    
    An explicit offload_strategy is still respected; only the strategies that let
    accelerate place components itself are overridden.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
    bcdd5ffb
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...