-
Stefy Lanza (nextime / spora ) authored
Two eviction fixes folded into 0.1.30: - VRAM eviction: a diffusers pipeline under device_map/accelerate offload (balanced/disk/model/sequential) or bitsandbytes quantization REJECTS .to('cpu') — the naive move silently left the weights resident and stranded VRAM, feeding the OOM death-spiral where the next load OOMs on a near-full card. Route pipelines (those exposing `components`) through the thorough _free_pipeline_vram (remove accelerate hooks -> drop component refs -> empty_cache); plain non-pipeline models still use a simple .to('cpu'). - RAM eviction: never unload the LIVE model (active_in_vram OR current_model_key) that a request loop keeps reusing. An offloaded pipeline's host RAM IS the live model, so evicting it between same-model requests just forces an immediate reload — churning VRAM/RAM and re-stranding device_map weights while freeing nothing lasting. The last-resort active-model eviction now fires only for a STALE active model (no longer the current one). Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
bb95ab1d