• Stefy Lanza (nextime / spora )'s avatar
    ocr: the three blockers to native Surya on rented pods · fd6d7873
    Stefy Lanza (nextime / spora ) authored
    Reported from the Digesta host, where coderai runs GPU-less and proxies to
    A40 pods.
    
    1. surya-ocr's vLLM client probes {service_url}/health and reads anything
       else as "backend down", so /v1/ocr never became ready against a coderai
       front that only served /healthz. /health and /v1/health now alias it,
       and /v1/health is exempt from the Bearer check — a readiness probe that
       needs a token is not a readiness probe.
    
    2. A rented pod ran the shared-workstation admission defaults (2 in
       flight, 6 queued), so its own gate capped an A40 at ~6 pages/s and
       everything above 16 concurrent got 429 while the GPU idled. The pod has
       no config file, so the knobs now arrive as env
       (CODERAI_SERVER_MAX_PARALLEL_REQUESTS / _QUEUE_MAX_SIZE) and are set
       per model from the model page (pod_max_parallel_requests /
       pod_queue_max_size), seeded when the pod is provisioned.
    
    3. The image baked no surya venv, and could not have: ocr.txt put
       surya-ocr in the MAIN venv, where its pillow<11 cap fights
       coderai's pillow>=12. It now has its own ocr.venv-surya profile with a
       deep check that proves the pin held. The venv belongs on the
       orchestrator, not the card: with surya_serve=vllm the model runs in
       vLLM on the pod and this side only renders and post-processes.
    
    Also a bug that fix exposed: _resolve_venv_dir looked only at
    /opt/coderai/<name>_venv while the image builder bakes
    /opt/coderai/venvs/<alias>, so a baked venv was invisible and got rebuilt
    on the cache mount at first use. ocr-paddle has been shipping a 2 GB
    paddleocr venv nothing ever found.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    fd6d7873
Name
Last commit
Last update
..
superpowers Loading commit data...
README.md Loading commit data...
cluster.md Loading commit data...
deepseek-ds4.md Loading commit data...
dtype-auto-selection.md Loading commit data...
expressive-tts.md Loading commit data...
frontend-engine-split.md Loading commit data...
gguf-process-isolation.md Loading commit data...
glm-colibri.md Loading commit data...
install-from-packages.md Loading commit data...
kimi-k3.md Loading commit data...
ktransformers.md Loading commit data...
longcat-video.md Loading commit data...
minimax-h3.md Loading commit data...
ocr.md Loading commit data...
process-isolation-plans.md Loading commit data...
reference-projects.md Loading commit data...
remote-execution.md Loading commit data...
reverse-proxy-nginx.md Loading commit data...
runpod.md Loading commit data...
tools-as-services.md Loading commit data...
video-compose.md Loading commit data...
vllm.md Loading commit data...
zimage-lora-training.md Loading commit data...