-
Stefy Lanza (nextime / spora ) authored
Reported from the Digesta host, where coderai runs GPU-less and proxies to A40 pods. 1. surya-ocr's vLLM client probes {service_url}/health and reads anything else as "backend down", so /v1/ocr never became ready against a coderai front that only served /healthz. /health and /v1/health now alias it, and /v1/health is exempt from the Bearer check — a readiness probe that needs a token is not a readiness probe. 2. A rented pod ran the shared-workstation admission defaults (2 in flight, 6 queued), so its own gate capped an A40 at ~6 pages/s and everything above 16 concurrent got 429 while the GPU idled. The pod has no config file, so the knobs now arrive as env (CODERAI_SERVER_MAX_PARALLEL_REQUESTS / _QUEUE_MAX_SIZE) and are set per model from the model page (pod_max_parallel_requests / pod_queue_max_size), seeded when the pod is provisioned. 3. The image baked no surya venv, and could not have: ocr.txt put surya-ocr in the MAIN venv, where its pillow<11 cap fights coderai's pillow>=12. It now has its own ocr.venv-surya profile with a deep check that proves the pin held. The venv belongs on the orchestrator, not the card: with surya_serve=vllm the model runs in vLLM on the pod and this side only renders and post-processes. Also a bug that fix exposed: _resolve_venv_dir looked only at /opt/coderai/<name>_venv while the image builder bakes /opt/coderai/venvs/<alias>, so a baked venv was invisible and got rebuilt on the cache mount at first use. ocr-paddle has been shipping a 2 GB paddleocr venv nothing ever found. Co-Authored-By:Claude Opus 5 <noreply@anthropic.com>
fd6d7873
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| common | ||
| dinov2cpp | ||
| linux | ||
| runpod | ||
| windows | ||
| build-rpc-server.sh | ||
| cosign.pub | ||
| patch-colibri.py | ||
| patch-k3.py | ||
| setup-salad-venv.sh | ||
| sign-images.sh | ||
| versions.env |