-
Stefy Lanza (nextime / spora ) authored
The Surya-2 engine serves its VLM via a managed vLLM subprocess (vllm_worker.ensure_service). That subprocess is not a manager-tracked model, and the OCR VRAM releaser only tore down the lightweight worker pools (killing the HTTP workers) — it never stopped the vLLM holding the bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the VRAM as used, found nothing it could free, and (with no CPU fallback) a docTR/Paddle load on the same card could OOM. - vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE service, return estimated GB freed) + is_running() helper. - OcrManager._release_vram now also stops the managed Surya-2 vLLM (_stop_surya_vllm) when surya_serve == "vllm", so the registered external releaser actually reclaims its VRAM. Next OCR request re-boots it via ensure_service. Targeted by (model_path, served_name) so an LLM vLLM instance is left alone. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
2c419247