Skip to content
Commit 2c419247 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

ocr: make managed Surya-2 vLLM reclaimable by VRAM eviction



The Surya-2 engine serves its VLM via a managed vLLM subprocess
(vllm_worker.ensure_service). That subprocess is not a manager-tracked
model, and the OCR VRAM releaser only tore down the lightweight worker
pools (killing the HTTP workers) — it never stopped the vLLM holding the
bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the
VRAM as used, found nothing it could free, and (with no CPU fallback) a
docTR/Paddle load on the same card could OOM.

- vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE
  service, return estimated GB freed) + is_running() helper.
- OcrManager._release_vram now also stops the managed Surya-2 vLLM
  (_stop_surya_vllm) when surya_serve == "vllm", so the registered external
  releaser actually reclaims its VRAM. Next OCR request re-boots it via
  ensure_service. Targeted by (model_path, served_name) so an LLM vLLM
  instance is left alone.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
parent 3a01becb
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment