• Stefy Lanza (nextime / spora )'s avatar
    ocr: make managed Surya-2 vLLM reclaimable by VRAM eviction · 2c419247
    Stefy Lanza (nextime / spora ) authored
    The Surya-2 engine serves its VLM via a managed vLLM subprocess
    (vllm_worker.ensure_service). That subprocess is not a manager-tracked
    model, and the OCR VRAM releaser only tore down the lightweight worker
    pools (killing the HTTP workers) — it never stopped the vLLM holding the
    bulk VRAM (gpu_memory_utilization × card). So on-request eviction saw the
    VRAM as used, found nothing it could free, and (with no CPU fallback) a
    docTR/Paddle load on the same card could OOM.
    
    - vllm_worker: add stop_service_for(cfg, model_path, served_name) (stop ONE
      service, return estimated GB freed) + is_running() helper.
    - OcrManager._release_vram now also stops the managed Surya-2 vLLM
      (_stop_surya_vllm) when surya_serve == "vllm", so the registered external
      releaser actually reclaims its VRAM. Next OCR request re-boots it via
      ensure_service. Targeted by (model_path, served_name) so an LLM vLLM
      instance is left alone.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    2c419247
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...