• Stefy Lanza (nextime / spora )'s avatar
    ocr: participate in thermal throttling + VRAM eviction · 3a01becb
    Stefy Lanza (nextime / spora ) authored
    OCR ran outside the shared GPU governors: /v1/ocr(/batch) and run_ocr_step
    call OcrManager.ocr_document directly, bypassing request_model, so OCR
    neither waited on thermal cooldown nor evicted/released VRAM against
    LLM/diffusion models.
    
    - thermal: ocr_document now awaits thermal.wait_until_safe (via to_thread)
      before OCR-ing pages, so OCR honours the same cooldown/front-pause gate as
      every other GPU workload.
    - VRAM (both directions):
      * evict-before-build: when a pool loads its instances, ask the model
        manager to free size*per-instance-VRAM first (_evict_models_for_vram),
        so OCR contends on equal footing instead of OOMing.
      * releaser: OcrManager registers _release_vram(needed_gb) via
        register_external_vram_releaser, tearing down built pools so loading an
        LLM/diffusion model can reclaim OCR VRAM. Pools rebuild lazily next request.
    - engine cleanup() (SYNC, eviction-thread safe) + vram_gb() footprint per
      engine: docTR ~0.7 GB (frees model + empty_cache), subprocess ~2.5 GB
      (kills worker). _Pool tracks all instances and release_sync() tears them down.
    - configure() now tears down stale pools on reconfigure (prompt VRAM free
      instead of GC-deferred).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    3a01becb
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...