• Stefy Lanza (nextime / spora )'s avatar
    ocr: make managed Surya-2 vLLM reclaimable by VRAM eviction · 2c419247
    Stefy Lanza (nextime / spora ) authored
    The Surya-2 engine serves its VLM via a managed vLLM subprocess
    (vllm_worker.ensure_service). That subprocess is not a manager-tracked
    model, and the OCR VRAM releaser only tore down the lightweight worker
    pools (killing the HTTP workers) — it never stopped the vLLM holding the
    bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the
    VRAM as used, found nothing it could free, and (with no CPU fallback) a
    docTR/Paddle load on the same card could OOM.
    
    - vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE
      service, return estimated GB freed) + is_running() helper.
    - OcrManager._release_vram now also stops the managed Surya-2 vLLM
      (_stop_surya_vllm) when surya_serve == "vllm", so the registered external
      releaser actually reclaims its VRAM. Next OCR request re-boots it via
      ensure_service. Targeted by (model_path, served_name) so an LLM vLLM
      instance is left alone.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    2c419247
Name
Last commit
Last update
..
schemas_builtin Loading commit data...
workers Loading commit data...
__init__.py Loading commit data...
base.py Loading commit data...
detect.py Loading commit data...
doctr.py Loading commit data...
extract.py Loading commit data...
manager.py Loading commit data...
paddle.py Loading commit data...
schemas.py Loading commit data...
subprocess_engine.py Loading commit data...
surya.py Loading commit data...
venv_build.py Loading commit data...