-
Stefy Lanza (nextime / spora ) authored
When the GPU is held by another workload (video job, concurrent load racing request_model's free-VRAM check), the embedding load/encode hit CUDA OOM and 500'd. Now both paths behave like other model loads: run the manager's eviction (which frees idle models and waits for busy ones to go idle) and retry once. No CPU fallback — the request participates in the same VRAM queue as everything else. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
872f6c61