• Stefy Lanza (nextime / spora )'s avatar
    ocr: surya should ask who serves the model, not where the GPU is (v0.3.7) · 51aa0d2f
    Stefy Lanza (nextime / spora ) authored
    OCR on the Digesta install answered 503: "vLLM isolated venv not found at
    /cache/vllm_venv". That host has no GPU at all — renting A40 pods is its whole
    job — so the venv it was told to build could never have run, while the Surya-2
    checkpoint it needed was already a configured model (backend: runpod,
    engine: coderai-vllm) nothing in the OCR path could ask for.
    
    The model layer was never the problem; `ocr.surya_serve` was asking the wrong
    question. It enumerated server PROCESSES — torch in a local venv, our own vLLM,
    a URL typed in by hand — when the answer the operator had already written down
    was a MODEL. olmOCR has had that bridge since it shipped ("model" mode, through
    our own chat path); Surya never got one, because Surya2 drives its server
    directly with guided-JSON decoding and logprobs that a proxy hop would drop.
    
    So it gets the same bridge, in the only shape it can take: resolve the model
    through whichever backend owns it and hand the worker the address. Three values,
    because Surya refuses a server whose /v1/models disagrees with the checkpoint it
    expects, and a pod URL is useless without its token — which only Surya's vLLM
    client reads, so a bearer-locked endpoint is always attached to as "vllm".
    
    The address is a lease, not a location: a min_pods=0 pod is reaped when OCR goes
    quiet, and the page that finds it gone re-resolves once — renting a fresh one —
    instead of every later page failing against a dead URL. A failure against a
    server that still answers /health is a real error and is reported as one.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    51aa0d2f
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...