-
Stefy Lanza (nextime / spora ) authored
video.py already knew the hazard and wrote it down: accelerate's device_map only dispatches components it LOADS, so one handed to from_pretrained keeps whatever device it is on with no offload hook, and the forward pass hits "index on cuda:0, weights on cpu". It acted on that only for pipeline-cache injections, while a GGUF component arrives exactly the same way. LTX-2.5 therefore failed under every strategy it tried -- auto, balanced, model and pure-disk, three restarts and half an hour each -- always with Expected all tensors to be on the same device, but got index is on cuda:0, different from other tensors on cpu and video_resident_experts, the flag that exists for precisely this, was never consulted because the gate in front of it was false. An explicit offload_strategy is still respected; only the strategies that let accelerate place components itself are overridden. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014T3rZnLhydX9QqUcfBbqLu
bcdd5ffb
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| cluster | ||
| compose | ||
| frontproxy | ||
| models | ||
| ocr | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| util | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| hf_cache.py | ||
| main.py | ||
| ocr_fleet_workers.py | ||
| platform_paths.py | ||
| system_app.py |