ocr: the three blockers to native Surya on rented pods

Reported from the Digesta host, where coderai runs GPU-less and proxies to
A40 pods.

1. surya-ocr's vLLM client probes {service_url}/health and reads anything
   else as "backend down", so /v1/ocr never became ready against a coderai
   front that only served /healthz. /health and /v1/health now alias it,
   and /v1/health is exempt from the Bearer check — a readiness probe that
   needs a token is not a readiness probe.

2. A rented pod ran the shared-workstation admission defaults (2 in
   flight, 6 queued), so its own gate capped an A40 at ~6 pages/s and
   everything above 16 concurrent got 429 while the GPU idled. The pod has
   no config file, so the knobs now arrive as env
   (CODERAI_SERVER_MAX_PARALLEL_REQUESTS / _QUEUE_MAX_SIZE) and are set
   per model from the model page (pod_max_parallel_requests /
   pod_queue_max_size), seeded when the pod is provisioned.

3. The image baked no surya venv, and could not have: ocr.txt put
   surya-ocr in the MAIN venv, where its pillow<11 cap fights
   coderai's pillow>=12. It now has its own ocr.venv-surya profile with a
   deep check that proves the pin held. The venv belongs on the
   orchestrator, not the card: with surya_serve=vllm the model runs in
   vLLM on the pod and this side only renders and post-processes.

Also a bug that fix exposed: _resolve_venv_dir looked only at
/opt/coderai/<name>_venv while the image builder bakes
/opt/coderai/venvs/<alias>, so a baked venv was invisible and got rebuilt
on the cache mount at first use. ocr-paddle has been shipping a 2 GB
paddleocr venv nothing ever found.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
parent 5670b855
...@@ -3122,6 +3122,8 @@ async def api_model_configure(request: Request, username: str = Depends(require_ ...@@ -3122,6 +3122,8 @@ async def api_model_configure(request: Request, username: str = Depends(require_
for k in ("container_disk_gb", "volume_gb", "port", "ctx", "min_pods", for k in ("container_disk_gb", "volume_gb", "port", "ctx", "min_pods",
"max_pods", "scale_up_inflight_per_pod", "max_inflight_per_pod", "max_pods", "scale_up_inflight_per_pod", "max_inflight_per_pod",
"idle_timeout_s", "idle_timeout_s",
# how hard the POD's own server may be driven
"pod_max_parallel_requests", "pod_queue_max_size",
"boot_timeout_s", "load_timeout_s", "min_workers", "max_workers", "boot_timeout_s", "load_timeout_s", "min_workers", "max_workers",
"gpu_count"): "gpu_count"):
v = src.get(k) v = src.get(k)
......
...@@ -947,6 +947,19 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson } ...@@ -947,6 +947,19 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
<span class="form-hint" style="font-size:11px">None ticked = every day. An end earlier than the start runs overnight, and belongs to the day it starts on.</span> <span class="form-hint" style="font-size:11px">None ticked = every day. An end earlier than the start runs overnight, and belongs to the day it starts on.</span>
<span class="form-hint" id="cfg-rp-schedule-status" style="font-size:11px"></span> <span class="form-hint" id="cfg-rp-schedule-status" style="font-size:11px"></span>
</div> </div>
<div style="display:grid;grid-template-columns:1fr 1fr;gap:.5rem;margin-top:.5rem">
<div class="form-row" style="margin:0">
<label class="form-label">Pod max parallel requests</label>
<input type="number" id="cfg-rp-pod-parallel" class="form-input" min="0" placeholder="0 = the pod's default (2)">
</div>
<div class="form-row" style="margin:0">
<label class="form-label">Pod queue size</label>
<input type="number" id="cfg-rp-pod-queue" class="form-input" min="0" placeholder="0 = the pod's default (6)">
</div>
</div>
<div class="form-row" style="margin:.25rem 0 0">
<span class="form-hint" style="font-size:11px">How many requests the pod's own server admits at once. The defaults are sized for a shared workstation and will throttle a dedicated card long before its GPU is busy — everything over the limit gets a 429. Set these to roughly the remote engine's batch width (e.g. vLLM's <code>max_num_seqs</code>).</span>
</div>
<div class="form-row" style="margin:.5rem 0 0"> <div class="form-row" style="margin:.5rem 0 0">
<label class="form-label">Region (data centre)</label> <label class="form-label">Region (data centre)</label>
<input type="text" id="cfg-rp-data-center" class="form-input" placeholder="blank = the account setting, e.g. EU-RO-1"> <input type="text" id="cfg-rp-data-center" class="form-input" placeholder="blank = the account setting, e.g. EU-RO-1">
...@@ -4886,6 +4899,8 @@ function _populateRunpod(s){ ...@@ -4886,6 +4899,8 @@ function _populateRunpod(s){
set('cfg-rp-schedule-end', rp.schedule_end || ''); set('cfg-rp-schedule-end', rp.schedule_end || '');
set('cfg-rp-schedule-tz', rp.schedule_tz || ''); set('cfg-rp-schedule-tz', rp.schedule_tz || '');
set('cfg-rp-data-center', rp.data_center || ''); set('cfg-rp-data-center', rp.data_center || '');
set('cfg-rp-pod-parallel', rp.pod_max_parallel_requests != null ? rp.pod_max_parallel_requests : '');
set('cfg-rp-pod-queue', rp.pod_queue_max_size != null ? rp.pod_queue_max_size : '');
(function(){ (function(){
// Days come back normalised to 0=Mon…6=Sun; an empty list means every day, // Days come back normalised to 0=Mon…6=Sun; an empty list means every day,
// which is shown as nothing ticked rather than everything. // which is shown as nothing ticked rather than everything.
...@@ -5096,6 +5111,8 @@ function _collectRunpod(){ ...@@ -5096,6 +5111,8 @@ function _collectRunpod(){
.filter(function(cb){ return cb.checked; }) .filter(function(cb){ return cb.checked; })
.map(function(cb){ return cb.value; }), .map(function(cb){ return cb.value; }),
data_center: (val('cfg-rp-data-center') || '').trim(), data_center: (val('cfg-rp-data-center') || '').trim(),
pod_max_parallel_requests: intOrU('cfg-rp-pod-parallel'),
pod_queue_max_size: intOrU('cfg-rp-pod-queue'),
}; };
return o; return o;
} }
......
...@@ -339,6 +339,18 @@ async def healthz(): ...@@ -339,6 +339,18 @@ async def healthz():
return {"ok": True, "pid": _os.getpid()} return {"ok": True, "pid": _os.getpid()}
# Stock vLLM serves its readiness probe at /health, and clients written against
# it hard-code that path: surya-ocr's vLLM client probes {service_url}/health and
# declares the backend down on anything else, which left /v1/ocr permanently
# "not ready" against a coderai front. Aliases, so both conventions work.
# /v1/health is exempt from the Bearer check (see ratelimit._PROGRESS_PATHS):
# a readiness probe that needs a token is not a readiness probe.
@app.get("/health", include_in_schema=False)
@app.get("/v1/health", include_in_schema=False)
async def health_alias():
return await healthz()
_ENGINE_VRAM_CACHE = {"at": 0.0, "vram": None, "names": {}} _ENGINE_VRAM_CACHE = {"at": 0.0, "vram": None, "names": {}}
_ENGINE_VRAM_TTL = 4.0 # seconds _ENGINE_VRAM_TTL = 4.0 # seconds
......
...@@ -43,6 +43,10 @@ _PROGRESS_PATHS = { ...@@ -43,6 +43,10 @@ _PROGRESS_PATHS = {
"/v1/video/progress", "/v1/video/progress",
"/v1/audio/progress", "/v1/audio/progress",
"/v1/loras/progress", "/v1/loras/progress",
# Readiness probes. vLLM-shaped clients (surya-ocr's among them) probe
# /health or /v1/health before sending anything and treat a 401 as "down",
# so these must answer without a token, exactly as /healthz does.
"/v1/health",
} }
......
...@@ -158,6 +158,15 @@ class RunpodModelConfig: ...@@ -158,6 +158,15 @@ class RunpodModelConfig:
# over both: a volume lives in one region and a pod elsewhere cannot attach # over both: a volume lives in one region and a pod elsewhere cannot attach
# it. Pin a region for latency or for where the data is allowed to be. # it. Pin a region for latency or for where the data is allowed to be.
data_center: str = "" data_center: str = ""
# --- how hard the pod's own server may be driven ---
# A rented pod carries no config file, so these ride as env. The defaults
# (2 in flight, 6 queued) are sized for a shared workstation engine and
# throttle a dedicated A40 to a fraction of what vLLM's max_num_seqs would
# serve: the app's admission gate saturates long before the GPU does, and
# everything over the limit gets 429. Set these to roughly the remote
# engine's own batch width. 0 = leave the pod's defaults alone.
pod_max_parallel_requests: int = 0
pod_queue_max_size: int = 0
# --- warm-pod schedule --- # --- warm-pod schedule ---
# When set, the warm floor (min_pods/keep_warm) only applies inside the # When set, the warm floor (min_pods/keep_warm) only applies inside the
# window; outside it the floor is 0 and warm pods are terminated, so a model # window; outside it the floor is 0 and warm pods are terminated, so a model
...@@ -954,6 +963,13 @@ def _plan(engine, image, args, mcfg, api_key, entry, served, ...@@ -954,6 +963,13 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
env = {} env = {}
if api_key: if api_key:
env["CODERAI_API_TOKEN"] = api_key env["CODERAI_API_TOKEN"] = api_key
# How hard the pod's own server may be driven. Without this the pod runs the
# shared-workstation defaults and its admission gate, not its GPU, is the
# ceiling.
if getattr(mcfg, "pod_max_parallel_requests", 0):
env["CODERAI_SERVER_MAX_PARALLEL_REQUESTS"] = str(int(mcfg.pod_max_parallel_requests))
if getattr(mcfg, "pod_queue_max_size", 0):
env["CODERAI_SERVER_QUEUE_MAX_SIZE"] = str(int(mcfg.pod_queue_max_size))
seeds = [] seeds = []
if entry: if entry:
kind, value = resolve_model_source(entry, mcfg) kind, value = resolve_model_source(entry, mcfg)
...@@ -1512,6 +1528,10 @@ def parse_model_runpod(block: Optional[dict]) -> RunpodModelConfig: ...@@ -1512,6 +1528,10 @@ def parse_model_runpod(block: Optional[dict]) -> RunpodModelConfig:
cfg.cost_limit_usd = _as_float(b.get("cost_limit_usd"), cfg.cost_limit_usd) cfg.cost_limit_usd = _as_float(b.get("cost_limit_usd"), cfg.cost_limit_usd)
cfg.cost_period = (b.get("cost_period") or cfg.cost_period).strip().lower() cfg.cost_period = (b.get("cost_period") or cfg.cost_period).strip().lower()
cfg.data_center = (b.get("data_center") or "").strip().upper() cfg.data_center = (b.get("data_center") or "").strip().upper()
cfg.pod_max_parallel_requests = max(0, _as_int(b.get("pod_max_parallel_requests"),
cfg.pod_max_parallel_requests))
cfg.pod_queue_max_size = max(0, _as_int(b.get("pod_queue_max_size"),
cfg.pod_queue_max_size))
cfg.schedule_enabled = _as_bool(b.get("schedule_enabled"), cfg.schedule_enabled) cfg.schedule_enabled = _as_bool(b.get("schedule_enabled"), cfg.schedule_enabled)
cfg.schedule_days = _parse_days(b.get("schedule_days")) cfg.schedule_days = _parse_days(b.get("schedule_days"))
cfg.schedule_start = _parse_hhmm(b.get("schedule_start")) cfg.schedule_start = _parse_hhmm(b.get("schedule_start"))
......
...@@ -1397,13 +1397,30 @@ def main(): ...@@ -1397,13 +1397,30 @@ def main():
queue_manager.max_parallel_requests = config.server.max_parallel_requests queue_manager.max_parallel_requests = config.server.max_parallel_requests
# In an engine the front may override this engine's concurrency (per-engine # In an engine the front may override this engine's concurrency (per-engine
# limit) via env, so a bigger card runs more in parallel than a smaller one. # limit) via env, so a bigger card runs more in parallel than a smaller one.
_mp = os.environ.get("CODERAI_MAX_PARALLEL") # On a RENTED POD the same knobs matter even more: the pod has no auth.json
# and no models.json to carry settings, so env is the only channel, and the
# defaults (2 in flight, 6 queued) throttle an A40 to a fraction of what
# vLLM's max_num_seqs would serve — the admission gate, not the GPU, becomes
# the ceiling and everything above it gets 429. CODERAI_SERVER_* are the
# documented names; CODERAI_MAX_PARALLEL stays as the front's existing one.
_mp = (os.environ.get("CODERAI_SERVER_MAX_PARALLEL_REQUESTS")
or os.environ.get("CODERAI_MAX_PARALLEL"))
if _mp: if _mp:
try: try:
queue_manager.max_parallel_requests = int(_mp) queue_manager.max_parallel_requests = int(_mp)
config.server.max_parallel_requests = int(_mp) config.server.max_parallel_requests = int(_mp)
except ValueError: except ValueError:
pass pass
_qs = (os.environ.get("CODERAI_SERVER_QUEUE_MAX_SIZE")
or os.environ.get("CODERAI_QUEUE_MAX_SIZE"))
if _qs:
try:
queue_manager.max_size = int(_qs)
config.server.queue_max_size = int(_qs)
except ValueError:
pass
print(f"[queue] max_parallel_requests={queue_manager.max_parallel_requests} "
f"max_size={queue_manager.max_size}", flush=True)
# RunPod: on the PRIMARY engine (which hosts runpod pods), start the scaler + # RunPod: on the PRIMARY engine (which hosts runpod pods), start the scaler +
# the independent stale-pod reaper as soon as RunPod is enabled — so pods # the independent stale-pod reaper as soon as RunPod is enabled — so pods
......
...@@ -23,7 +23,7 @@ class PaddleEngine(SubprocessOcrEngine): ...@@ -23,7 +23,7 @@ class PaddleEngine(SubprocessOcrEngine):
_worker_engine = "paddle" _worker_engine = "paddle"
def _venv_dir(self) -> str: def _venv_dir(self) -> str:
return _resolve_venv_dir(self.cfg.paddle_venv, "paddle_venv") return _resolve_venv_dir(self.cfg.paddle_venv, "paddle_venv", ("paddleocr", "paddle"))
def _requirements(self) -> str: def _requirements(self) -> str:
return os.path.abspath(_REQ) return os.path.abspath(_REQ)
......
...@@ -25,23 +25,30 @@ from codai.ocr.base import OcrEngine, OcrPage, OcrLine, OcrRegion, OcrError ...@@ -25,23 +25,30 @@ from codai.ocr.base import OcrEngine, OcrPage, OcrLine, OcrRegion, OcrError
_WORKER = os.path.join(os.path.dirname(__file__), "workers", "ocr_worker.py") _WORKER = os.path.join(os.path.dirname(__file__), "workers", "ocr_worker.py")
def _resolve_venv_dir(configured: str, name: str) -> str: def _resolve_venv_dir(configured: str, name: str, baked_aliases: tuple = ()) -> str:
"""Resolve an isolated-venv directory for an OCR engine. """Resolve an isolated-venv directory for an OCR engine.
Precedence: explicit config path > a venv BAKED into the image Precedence: explicit config path > a venv BAKED into the image
(/opt/coderai/<name>) > a build on the PERSISTENT cache mount (/opt/coderai/<name>, or /opt/coderai/venvs/<alias>) > a build on the
(CODERAI_CACHE_DIR or /cache → <cache>/ocr/<name>, survives container restarts) > PERSISTENT cache mount (CODERAI_CACHE_DIR or /cache → <cache>/ocr/<name>,
~/.coderai/<name> for host/source installs. survives container restarts) > ~/.coderai/<name> for host/source installs.
The order matters: a runtime-built venv must NOT land in the image's writable layer The order matters: a runtime-built venv must NOT land in the image's writable layer
(lost on `coderai docker` restart) — it goes on the /cache mount, like the colibri/k3 (lost on `coderai docker` restart) — it goes on the /cache mount, like the colibri/k3
engines. engines.
``baked_aliases`` are the names the capability-image builder uses: a profile's
``<profile>.venv-<alias>.txt`` becomes ``/opt/coderai/venvs/<alias>``, which is
NOT ``/opt/coderai/<engine>_venv``. Without them a baked venv is invisible and
the engine rebuilds it on the cache mount at first use — paying minutes (or,
on a pod, failing) for something already in the image.
""" """
if configured: if configured:
return configured return configured
baked = f"/opt/coderai/{name}" for baked in (f"/opt/coderai/{name}",
if os.path.isdir(baked): *(f"/opt/coderai/venvs/{a}" for a in baked_aliases)):
return baked if os.path.isdir(baked):
return baked
cache = os.environ.get("CODERAI_CACHE_DIR") or ("/cache" if os.path.isdir("/cache") else "") cache = os.environ.get("CODERAI_CACHE_DIR") or ("/cache" if os.path.isdir("/cache") else "")
if cache and os.path.isdir(cache): if cache and os.path.isdir(cache):
return os.path.join(cache, "ocr", name) return os.path.join(cache, "ocr", name)
......
...@@ -25,7 +25,7 @@ class SuryaEngine(SubprocessOcrEngine): ...@@ -25,7 +25,7 @@ class SuryaEngine(SubprocessOcrEngine):
_worker_engine = "surya" _worker_engine = "surya"
def _venv_dir(self) -> str: def _venv_dir(self) -> str:
return _resolve_venv_dir(self.cfg.surya_venv, "surya_venv") return _resolve_venv_dir(self.cfg.surya_venv, "surya_venv", ("surya",))
def _requirements(self) -> str: def _requirements(self) -> str:
return os.path.abspath(_REQ) return os.path.abspath(_REQ)
......
...@@ -144,6 +144,8 @@ sync.) In `models.json` that is a `backend` pin plus a `runpod` block: ...@@ -144,6 +144,8 @@ sync.) In `models.json` that is a `backend` pin plus a `runpod` block:
| `max_pods` | `1` | Hard ceiling on concurrent pods for this model. | | `max_pods` | `1` | Hard ceiling on concurrent pods for this model. |
| `scale_up_inflight_per_pod` | `4` | Add a pod when in-flight requests per pod exceeds this. | | `scale_up_inflight_per_pod` | `4` | Add a pod when in-flight requests per pod exceeds this. |
| `idle_timeout_s` | `300` | Destroy a pod this long after its **last** request. | | `idle_timeout_s` | `300` | Destroy a pod this long after its **last** request. |
| `pod_max_parallel_requests` | `0` (the pod's own default, 2) | How many requests the **pod's** coderai server admits at once. The default is sized for a shared workstation and throttles a dedicated card long before its GPU is busy — requests over the limit get `429` while vLLM's `max_num_seqs` sits unused. Set it to roughly the remote engine's batch width. Rides to the pod as `CODERAI_SERVER_MAX_PARALLEL_REQUESTS`. |
| `pod_queue_max_size` | `0` (the pod's own default, 6) | How deep the pod queues admitted-but-waiting requests. Rides as `CODERAI_SERVER_QUEUE_MAX_SIZE`. |
| `data_center` | — | Pin this model's pods to one RunPod data centre (region), e.g. `EU-RO-1`. Blank uses the account-wide setting. A network volume still wins over both: a volume lives in one region and a pod elsewhere cannot attach it. | | `data_center` | — | Pin this model's pods to one RunPod data centre (region), e.g. `EU-RO-1`. Blank uses the account-wide setting. A network volume still wins over both: a volume lives in one region and a pod elsewhere cannot attach it. |
| `schedule_enabled` | `false` | Apply the warm floor (`min_pods`/`keep_warm`) only inside a daily window. Outside it the floor is `0` and the warm pod is terminated, so a model used in office hours stops billing overnight. It does **not** block requests: one arriving out of hours cold-starts a pod exactly as `min_pods: 0` always has. Ignored unless both `schedule_start` and `schedule_end` are set. | | `schedule_enabled` | `false` | Apply the warm floor (`min_pods`/`keep_warm`) only inside a daily window. Outside it the floor is `0` and the warm pod is terminated, so a model used in office hours stops billing overnight. It does **not** block requests: one arriving out of hours cold-starts a pod exactly as `min_pods: 0` always has. Ignored unless both `schedule_start` and `schedule_end` are set. |
| `schedule_start` / `schedule_end` | — | `HH:MM`. The start is inclusive, the end exclusive. An end **earlier** than the start runs overnight and belongs to the day it started on, so `22:00`–`06:00` on `fri` covers Friday night into Saturday morning. | | `schedule_start` / `schedule_end` | — | `HH:MM`. The start is inclusive, the end exclusive. An end **earlier** than the start runs overnight and belongs to the day it started on, so `22:00`–`06:00` on `fri` covers Friday night into Saturday morning. |
......
...@@ -26,7 +26,8 @@ jsonschema>=4.0.0 ...@@ -26,7 +26,8 @@ jsonschema>=4.0.0
# #
# SuryaEngine still runs as a subprocess (it is a SubprocessOcrEngine), so the # SuryaEngine still runs as a subprocess (it is a SubprocessOcrEngine), so the
# pod points ocr.surya_venv at this interpreter rather than building one. # pod points ocr.surya_venv at this interpreter rather than building one.
surya-ocr==0.22.1 # surya-ocr is NOT here: it caps pillow<11 against coderai's pillow>=12, and the
# engine runs it in an isolated venv subprocess anyway. See ocr.venv-surya.txt.
# olmOCR-2 needs NOTHING here. It is a document VLM, not a library: it imports only # olmOCR-2 needs NOTHING here. It is a document VLM, not a library: it imports only
# PIL and requests, and reaches its model one of three ways — coderai's own model # PIL and requests, and reaches its model one of three ways — coderai's own model
......
# Deep check for the surya venv: the vLLM-backed path must import, not just the
# package. surya's vLLM client is what talks to the pod, and an import error
# here is the difference between a working /v1/ocr and one that is never ready.
import surya # noqa
from surya.common.surya.schema import TaskNames # noqa
from PIL import Image
import PIL
assert int(PIL.__version__.split(".")[0]) < 11, f"pillow {PIL.__version__} — surya caps <11"
print("surya", getattr(surya, "__version__", "ok"), "pillow", PIL.__version__)
# Surya in its own venv, baked at /opt/coderai/venvs/surya (the OCR engine finds
# it there — see codai/ocr/subprocess_engine._resolve_venv_dir).
#
# It CANNOT live in the main venv: surya-ocr caps pillow<11 and coderai needs
# pillow>=12. Installing it alongside either breaks image loading or silently
# downgrades pillow for everything else.
#
# This venv holds the surya CLIENT, which is also where it belongs in the split:
# with surya_serve="vllm" the heavy model runs in vLLM on the GPU pod and this
# side only renders, batches and post-processes. So the venv goes on the
# ORCHESTRATOR image, never on the rented card.
surya-ocr==0.22.1
pillow<11
...@@ -13,3 +13,9 @@ vllm==0.27.1+cu129 ...@@ -13,3 +13,9 @@ vllm==0.27.1+cu129
torch==2.13.0+cu129 torch==2.13.0+cu129
torchvision==0.28.0+cu129 torchvision==0.28.0+cu129
torchaudio==2.11.0+cu129 torchaudio==2.11.0+cu129
# torchcodec is a vLLM requirement (>=0.14) and PyPI's newest, 0.17.0, is built
# against CUDA 13: it dlopens libnvrtc.so.13 while this venv has nvrtc 12.9, so
# `import vllm.entrypoints.openai.api_server` dies even though `import vllm`
# works. Pinned to the newest +cu129 build on torch's index. The same CUDA
# 13-vs-12 trap as the torch pins above, one layer further down.
torchcodec==0.16.0+cu129
"""The three Surya-on-RunPod blockers reported from the Digesta host.
1. surya-ocr's vLLM client probes {service_url}/health; coderai served only
/healthz, so /v1/ocr never became ready.
2. A rented pod ran the shared-workstation admission defaults (2 in flight,
6 queued), so its own gate — not the GPU — capped throughput and everything
above it got 429. The pod carries no config file, so env is the channel.
"""
import pathlib
import pytest
ROOT = pathlib.Path(__file__).resolve().parents[1]
# ----------------------------------------------------------------- bug 1
def test_health_and_v1_health_alias_healthz():
src = (ROOT / "codai/api/app.py").read_text()
assert '@app.get("/health", include_in_schema=False)' in src
assert '@app.get("/v1/health", include_in_schema=False)' in src
body = src.split('async def health_alias')[1][:200]
assert "return await healthz()" in body, "the alias must reuse the real probe"
def test_v1_health_needs_no_bearer_token():
"""A readiness probe that 401s is read as 'backend down'."""
from codai.api.ratelimit import _PROGRESS_PATHS
assert "/v1/health" in _PROGRESS_PATHS
def test_the_alias_returns_what_healthz_returns():
import asyncio
from codai.api.app import healthz, health_alias
a = asyncio.run(healthz())
b = asyncio.run(health_alias())
assert a == b and a.get("ok") is True
# ----------------------------------------------------------------- bug 2
def test_the_pod_server_honours_concurrency_env():
src = (ROOT / "codai/main.py").read_text()
for name in ("CODERAI_SERVER_MAX_PARALLEL_REQUESTS", "CODERAI_SERVER_QUEUE_MAX_SIZE"):
assert name in src, name
# The front's existing knob must keep working.
assert "CODERAI_MAX_PARALLEL" in src
# Both the queue manager and the config object get updated, or something
# downstream reads the stale value.
blk = src.split("Apply queue scheduler settings")[1][:2000]
assert "queue_manager.max_parallel_requests = int(_mp)" in blk
assert "config.server.max_parallel_requests = int(_mp)" in blk
assert "queue_manager.max_size = int(_qs)" in blk
assert "config.server.queue_max_size = int(_qs)" in blk
def test_per_model_pod_concurrency_parses():
from codai.api.runpod_worker import parse_model_runpod
cfg = parse_model_runpod({"pod_max_parallel_requests": 48, "pod_queue_max_size": 256})
assert cfg.pod_max_parallel_requests == 48
assert cfg.pod_queue_max_size == 256
# Absent means "leave the pod's own defaults alone", not zero concurrency.
d = parse_model_runpod({})
assert d.pod_max_parallel_requests == 0 and d.pod_queue_max_size == 0
# Junk must not crash the provisioning path.
j = parse_model_runpod({"pod_max_parallel_requests": "lots"})
assert j.pod_max_parallel_requests == 0
def test_the_settings_reach_the_pod_as_env():
src = (ROOT / "codai/api/runpod_worker.py").read_text()
blk = src.split("def _plan(")[1].split("seeds = []")[0]
assert 'env["CODERAI_SERVER_MAX_PARALLEL_REQUESTS"]' in blk
assert 'env["CODERAI_SERVER_QUEUE_MAX_SIZE"]' in blk
# Only when set: an unset value must not send "0" and wedge the pod shut.
assert 'getattr(mcfg, "pod_max_parallel_requests", 0)' in blk
def test_zero_sends_nothing():
"""Sending CODERAI_SERVER_MAX_PARALLEL_REQUESTS=0 would admit nothing."""
src = (ROOT / "codai/api/runpod_worker.py").read_text()
blk = src.split("def _plan(")[1].split("seeds = []")[0]
i = blk.index("pod_max_parallel_requests")
assert "if getattr" in blk[max(0, i - 40):i], "must be guarded by a truthiness check"
# --------------------------------------------------- orchestration surfaces
def test_the_new_keys_survive_the_whitelist():
src = (ROOT / "codai/admin/routes.py").read_text()
blk = src.split("# Per-model runpod block")[1].split("# Per-model `host` block")[0]
for k in ("pod_max_parallel_requests", "pod_queue_max_size"):
assert f'"{k}"' in blk, k
@pytest.mark.parametrize("field", ["cfg-rp-pod-parallel", "cfg-rp-pod-queue"])
def test_configurable_from_the_model_page(field):
html = (ROOT / "codai/admin/templates/models.html").read_text()
assert html.count(field) >= 3, f"{field}: {html.count(field)} occurrences"
# ----------------------------------------------- packaging: the surya venv
def test_the_resolver_finds_a_venv_where_the_image_builder_bakes_it(tmp_path, monkeypatch):
"""<profile>.venv-<alias>.txt bakes /opt/coderai/venvs/<alias>, which is not
/opt/coderai/<engine>_venv. Missing that made the baked venv invisible and
the engine rebuilt it at first use."""
from codai.ocr import subprocess_engine as se
root = tmp_path / "opt" / "coderai"
(root / "venvs" / "surya").mkdir(parents=True)
monkeypatch.setattr(se.os.path, "isdir",
lambda p: str(p).startswith(str(root)) and se.os.path.exists(p))
# Patch the literal prefix the resolver builds.
src_fn = se._resolve_venv_dir
monkeypatch.setattr(se, "_resolve_venv_dir", src_fn)
got = None
import os as _os
real_isdir = _os.path.isdir
def fake_isdir(p):
p = str(p)
if p.startswith("/opt/coderai/"):
return real_isdir(str(root) + p[len("/opt/coderai"):])
return real_isdir(p)
monkeypatch.setattr(se.os.path, "isdir", fake_isdir)
got = se._resolve_venv_dir("", "surya_venv", ("surya",))
assert got == "/opt/coderai/venvs/surya"
def test_an_explicit_config_path_still_wins():
from codai.ocr.subprocess_engine import _resolve_venv_dir
assert _resolve_venv_dir("/somewhere/mine", "surya_venv", ("surya",)) == "/somewhere/mine"
def test_both_ocr_engines_declare_their_packaging_names():
assert 'self.cfg.surya_venv, "surya_venv", ("surya",)' in \
(ROOT / "codai/ocr/surya.py").read_text()
assert 'self.cfg.paddle_venv, "paddle_venv", ("paddleocr", "paddle")' in \
(ROOT / "codai/ocr/paddle.py").read_text()
def test_surya_is_not_in_the_ocr_main_venv():
"""surya caps pillow<11; coderai needs >=12. One of them loses."""
main = (ROOT / "packaging/runpod/profiles/ocr.txt").read_text()
body = [l for l in main.splitlines() if l.strip() and not l.strip().startswith("#")]
assert not any("surya" in l for l in body), body
def test_the_ocr_image_bakes_a_surya_venv():
req = ROOT / "packaging/runpod/profiles/ocr.venv-surya.txt"
chk = ROOT / "packaging/runpod/profiles/ocr.venv-surya.check"
assert req.exists() and chk.exists()
body = req.read_text()
assert "surya-ocr==0.22.1" in body
assert "pillow<11" in body, "without the cap the resolver may pull pillow 12 in"
# The check must prove the pillow pin held, not just that surya imports.
assert "pillow" in chk.read_text().lower()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment