-
Stefy Lanza (nextime / spora ) authored
After the GGUF-isolation split, a torch engine and a gguf engine share one NVIDIA card but each can only evict its OWN models. So loading an image model on the torch engine while the gguf engine still holds a resident GGUF text model failed: local eviction "freed 0.0 GB ... VRAM held elsewhere" → CUDA OOM. Fix — co-located VRAM release: - front passes each engine its same-GPU siblings' internal URLs via CODERAI_COSITED_URLS (matched by identical CODERAI_ENGINE_GPUS selectors). - engine registers an external_vram_releaser that POSTs to each sibling's new /internal/evict-vram when local eviction can't free enough. - /internal/evict-vram → manager.release_idle_vram(): evicts all idle (non-busy) models and returns GB freed; busy/actively-serving models are left alone. Symmetric: the gguf engine can likewise reclaim VRAM from the torch engine's idle diffusers models. Driver-level free VRAM (cross-process) is re-checked after each releaser, so the loader proceeds once the sibling has freed enough. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
88063940