-
Stefy Lanza (nextime / spora ) authored
The Surya-2 engine serves its VLM via a managed vLLM subprocess (vllm_worker.ensure_service). That subprocess is not a manager-tracked model, and the OCR VRAM releaser only tore down the lightweight worker pools (killing the HTTP workers) — it never stopped the vLLM holding the bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the VRAM as used, found nothing it could free, and (with no CPU fallback) a docTR/Paddle load on the same card could OOM. - vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE service, return estimated GB freed) + is_running() helper. - OcrManager._release_vram now also stops the managed Surya-2 vLLM (_stop_surya_vllm) when surya_serve == "vllm", so the registered external releaser actually reclaims its VRAM. Next OCR request re-boots it via ensure_service. Targeted by (model_path, served_name) so an LLM vLLM instance is left alone. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
2c419247
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| schemas_builtin | ||
| workers | ||
| __init__.py | ||
| base.py | ||
| detect.py | ||
| doctr.py | ||
| extract.py | ||
| manager.py | ||
| paddle.py | ||
| schemas.py | ||
| subprocess_engine.py | ||
| surya.py | ||
| venv_build.py |