• Stefy Lanza (nextime / spora )'s avatar
    ocr: the three blockers to native Surya on rented pods · fd6d7873
    Stefy Lanza (nextime / spora ) authored
    Reported from the Digesta host, where coderai runs GPU-less and proxies to
    A40 pods.
    
    1. surya-ocr's vLLM client probes {service_url}/health and reads anything
       else as "backend down", so /v1/ocr never became ready against a coderai
       front that only served /healthz. /health and /v1/health now alias it,
       and /v1/health is exempt from the Bearer check — a readiness probe that
       needs a token is not a readiness probe.
    
    2. A rented pod ran the shared-workstation admission defaults (2 in
       flight, 6 queued), so its own gate capped an A40 at ~6 pages/s and
       everything above 16 concurrent got 429 while the GPU idled. The pod has
       no config file, so the knobs now arrive as env
       (CODERAI_SERVER_MAX_PARALLEL_REQUESTS / _QUEUE_MAX_SIZE) and are set
       per model from the model page (pod_max_parallel_requests /
       pod_queue_max_size), seeded when the pod is provisioned.
    
    3. The image baked no surya venv, and could not have: ocr.txt put
       surya-ocr in the MAIN venv, where its pillow<11 cap fights
       coderai's pillow>=12. It now has its own ocr.venv-surya profile with a
       deep check that proves the pin held. The venv belongs on the
       orchestrator, not the card: with surya_serve=vllm the model runs in
       vLLM on the pod and this side only renders and post-processes.
    
    Also a bug that fix exposed: _resolve_venv_dir looked only at
    /opt/coderai/<name>_venv while the image builder bakes
    /opt/coderai/venvs/<alias>, so a baked venv was invisible and got rebuilt
    on the cache mount at first use. ocr-paddle has been shipping a 2 GB
    paddleocr venv nothing ever found.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    fd6d7873
Name
Last commit
Last update
..
common Loading commit data...
dinov2cpp Loading commit data...
linux Loading commit data...
runpod Loading commit data...
windows Loading commit data...
build-rpc-server.sh Loading commit data...
cosign.pub Loading commit data...
patch-colibri.py Loading commit data...
patch-k3.py Loading commit data...
setup-salad-venv.sh Loading commit data...
sign-images.sh Loading commit data...
versions.env Loading commit data...