• Stefy Lanza (nextime / spora )'s avatar
    ocr: make managed Surya-2 vLLM reclaimable by VRAM eviction · 2c419247
    Stefy Lanza (nextime / spora ) authored
    The Surya-2 engine serves its VLM via a managed vLLM subprocess
    (vllm_worker.ensure_service). That subprocess is not a manager-tracked
    model, and the OCR VRAM releaser only tore down the lightweight worker
    pools (killing the HTTP workers) — it never stopped the vLLM holding the
    bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the
    VRAM as used, found nothing it could free, and (with no CPU fallback) a
    docTR/Paddle load on the same card could OOM.
    
    - vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE
      service, return estimated GB freed) + is_running() helper.
    - OcrManager._release_vram now also stops the managed Surya-2 vLLM
      (_stop_surya_vllm) when surya_serve == "vllm", so the registered external
      releaser actually reclaims its VRAM. Next OCR request re-boots it via
      ensure_service. Targeted by (model_path, served_name) so an LLM vLLM
      instance is left alone.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    2c419247
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
characters.py Loading commit data...
colibri_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
log.py Loading commit data...
loras.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
ratelimit.py Loading commit data...
spatial.py Loading commit data...
state.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...