-
Stefy Lanza (nextime / spora ) authored
OCR on the Digesta install answered 503: "vLLM isolated venv not found at /cache/vllm_venv". That host has no GPU at all — renting A40 pods is its whole job — so the venv it was told to build could never have run, while the Surya-2 checkpoint it needed was already a configured model (backend: runpod, engine: coderai-vllm) nothing in the OCR path could ask for. The model layer was never the problem; `ocr.surya_serve` was asking the wrong question. It enumerated server PROCESSES — torch in a local venv, our own vLLM, a URL typed in by hand — when the answer the operator had already written down was a MODEL. olmOCR has had that bridge since it shipped ("model" mode, through our own chat path); Surya never got one, because Surya2 drives its server directly with guided-JSON decoding and logprobs that a proxy hop would drop. So it gets the same bridge, in the only shape it can take: resolve the model through whichever backend owns it and hand the worker the address. Three values, because Surya refuses a server whose /v1/models disagrees with the checkpoint it expects, and a pod URL is useless without its token — which only Surya's vLLM client reads, so a bearer-locked endpoint is always attached to as "vllm". The address is a lease, not a location: a min_pods=0 pod is reaped when OCR goes quiet, and the page that finds it gone re-resolves once — renting a fresh one — instead of every later page failing against a dead URL. A failure against a server that still answers /health is a real error and is reported as one. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
51aa0d2f
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| archive.html | ||
| base.html | ||
| change_password.html | ||
| chat.html | ||
| cluster.html | ||
| dashboard.html | ||
| login.html | ||
| models.html | ||
| runpod.html | ||
| settings.html | ||
| tasks.html | ||
| tokens.html | ||
| users.html |