- 16 Sep, 2026 14 commits
-
-
Stefy Lanza (nextime / spora ) authoredgpu_count was passed to RunPod's gpuCount and then ignored: the price ceiling checked one card, min_vram_gb one card's memory, and the engine loaded on GPU 0 while the rest sat idle. Now the pod is ranked as a whole (price x N, VRAM x N, '2x A100'), vLLM gets --tensor-parallel-size N and llama.cpp --split-mode layer. Exposed on the model page, clamped 1-8. Requests forwarded to a pod or host never appeared on the Tasks page, so a running remote job was invisible from the admin. _forward now registers a 'remote' task whose message names the pod it landed on. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
-
Stefy Lanza (nextime / spora ) authored
weights_gb, on the model entry or its runpod block, is taken as-is and the HuggingFace estimate is skipped. It is the only honest answer for a URL download, an upload or a local file — none of which the estimator can see — and for an HF repo it beats a heuristic that deliberately errs high (SDXL: estimated 35 GB, actually ~7 for one precision). A nonsense value is ignored rather than trusted, and the field is never seeded to a pod: it describes the pod's disk, not the model. /v1/models/test/state now reports the disk a pod would be rented with, the weight size used, and whether that size was stated or estimated — decided before any money is spent. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
remote-execution.md: the capability images are two kinds now — ten on the full core and five on a torch-free light core for stacks that cannot share a venv — and the doc says which, why each split was measured rather than assumed, what the .venv-*.check scripts catch that a bare import cannot, and why the light core exists (RunPod refuses images above ~10 GB; a specialised image on the full core carried 7 GB of torch it never used). The 'speaker rides stt' line was wrong — that image shipped nothing for speaker work — and is fixed. A new section covers what travels to a pod as environment: the HF token, licence acceptances, baked-venv paths, OCR engine gates. Surya on a pod is documented as served by docTR, with the reason. The host backend gets its own section: always-on and on-demand shapes, what it deliberately lacks, and service_token. runpod.md: the size line with the numbers behind it, EXITED as a dead pod rather than a slow pull, container-disk sizing from HF metadata, and the field table corrected — boot_timeout_s is 900, container_disk_gb grows to fit. README: the fifteen images and the host backend, one line each. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The pod was launched with docTR as its default and still answered 'OCR engine surya is not enabled' — because the request carried engine=surya, and the OCR route honours the request's own field over the configured default. Setting the pod's env changed the default; it did not change what the request asked for. A pod target for OCR now rewrites the request's engine field on the way out, to the engine the pod will actually serve (pod_ocr_engine mirrors the launch env). Multipart is edited on the bytes with every other part untouched and Content-Length corrected; verified through FastAPI's own parser with the real probe body, file intact. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
XTTS reached its own loader on a pod and died on 'unparseable worker reply'. coqui prints an interactive CPML licence prompt to stdout and then blocks on input() — on a pipe, that corrupts the JSON protocol and then waits forever. Two fixes. The service now keeps the real stdout for the protocol and sends everything else the process prints to stderr, so no library chatter can reach the caller's parser again. And the CPML acceptance travels as COQUI_TOS_AGREED, locally and to a pod: configuring an XTTS model at all IS the decision, exactly as surya's licence is handled, and the pod inherits it. Verified in the image: the load returns clean JSON with the full speaker list. Surya: 0.22 is a VLM (surya-ocr-2), not a self-contained OCR library. It needs a server — vLLM, which it spawns IN DOCKER, or llama-server on a GGUF — and the pod image has neither; locally it rides coderai's own vLLM engine. A pod asked for surya now serves the request with docTR, which is in the image and has passed, and logs why — instead of loading surya, reaching inference, and dying on 'docker binary not found'. Surya on a pod belongs with the engine images, where a llama-server and the GGUF are the same work as colibri and ds4. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The pod reported 'No module named surya.ocr' — an API that has not existed for a year. The real error was two tiers up: the loader wrapped each version branch in a bare except, so a failure INSIDE the right branch was swallowed and it fell through to the oldest one. The reported error was from code that never should have run. The real error was surya 0.17.1's config class choking on transformers 5.17. The pin came from a stale requirements-surya.txt. The local install — read from its venv's dist-info — runs surya 0.22.1 on transformers 5.15, and works. Both now pin 0.22.1. 0.22 dropped surya.foundation and takes RecognitionPredictor() alone, which the loader already handled — but only as a fallback reached by swallowed exception. It now selects the API by what IMPORTS: foundation present is 0.17, absent is 0.22 (or pre-0.17). Verified loading 0.22.1 through the engine class on the pod stack beside docTR. tts-xtts: coqui's TTS(name) splits on '/' and wants its own four-part catalogue name; a HuggingFace id has two parts and fails 'expected 4, got 2'. The service translates (coqui/XTTS-v2 -> tts_models/multilingual/multi-dataset/xtts_v2), verified against coqui's catalogue. This means XTTS through _CoquiBackend had never worked with an HF id anywhere, not only on a pod. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The capability images are ordinary containers; nothing about them needs RunPod. The same image can run on a box you own, a server rented by the month, a second GPU in the next room — and coderai should use it the way it uses a pod: per model, with an endpoint and a token, and nothing else. always on: {"backend": "host", "host": {"url": ..., "api_key": ...}} on demand: add start_cmd (+ optional stop_cmd, boot_timeout_s, idle_timeout_s) start_cmd is whatever starts the thing — docker run, ssh box docker start, a systemd unit. coderai runs it, waits for /healthz, uses the host, and runs stop_cmd after idle_timeout_s with nothing in flight. It only ever stops a host it started itself; one that was already up is never touched. Deliberately absent: GPU search, price ranking, budgets, spot, capacity fallback, the orphan reaper. Those exist because a cloud rents an anonymous card by the second and can lose it. A host is a named machine you are responsible for — the machinery would be wrong, not just unnecessary. Verified for real: a capability container started by command from a cold state, healthy in 6s, used, and stopped by the idle timer. Also fixed on the way: a plain service_url carried no token, so a hand-run capability image behind one answered 401 to everything. service_token travels now. Neither host nor service_* is seeded to a pod: they say where a model runs from HERE, and would make a pod try to forward to itself. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
'function object has no attribute tqdm', on a rented pod, after the model had loaded. Current f5-tts defaults its progress argument to the tqdm MODULE and calls progress.tqdm(iterable); older releases called progress(iterable). Our call site passed a bare lambda, written for the old shape, and the new shape went looking for a .tqdm attribute on it. _NoProgress answers both — callable, and with a .tqdm — and shows nothing, which is what a server wants. Confirmed against the f5-tts actually in the voice image: its infer_batch_process calls progress.tqdm(...). Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The HF token fix worked: pyannote got past the gate and failed one layer deeper, instantiating the pipeline. Reproduced by running the real service with a GPU: ModuleNotFoundError: No module named 'matplotlib'. pyannote's segmentation tasks import matplotlib.pyplot at module load; it is not declared, and 'import pyannote.audio' never reaches it, so the build's import check passed and the venv shipped broken. This is the fourth venv whose bare import check passed while the real load failed. A <profile>.venv-<name>.check script now runs at build time in the venv's own python and imports what the REAL load reaches — pipeline modules, model classes — without a GPU or a token, which a build has neither of. The pyannote check fails on the shipped image and passes with the fix; the XTTS check passes as-is, so that venv is genuinely complete. Also: every isolated-venv worker reported failures as the LAST FIVE LINES of the subprocess tail. A Python traceback's last five lines are frame headers — the exception is the final line, and cutting to five dropped it whenever the trace was deep. Ten workers said where a process died and never why. Fixed in all of them. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
coqui-tts imports a transformers 4.x symbol that 5.x removed. Unlike parler and the OCR engines it had never been isolated locally: _CoquiBackend did a plain 'from TTS.api import TTS' and gave up when it was missing. On a pod that meant an image with a verified, working TTS venv answered 'pip install coqui-tts' — the venv was there and nothing knew to use it. xtts_worker finds the venv the way the others do (explicit env var, the baked /opt/coderai/venvs/TTS, then per-user) and drives tools/xtts_service.py in it over stdin/stdout. XttsSubprocess wears the TTS object's shape — speakers, tts(**kwargs), synthesizer.output_sample_rate, and the TypeError the backend relies on to retry without speed — so synthesize() runs unchanged whether the library is in this process or next door. Verified inside the pushed image: the venv is found and the service answers from it. The fall-through is unit-tested with a fake worker, and the error when no venv exists names the places it looked. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
pyannote is gated. On a pod it loaded from its baked venv, reached from_pretrained, and stopped: 'the model is gated/unavailable'. The local install has HF_TOKEN in its environment; a pod was never sent one — every gated model was unservable remotely, not just this one. It is resolved the way pyannote_worker already resolves it (request, env, token file) and travels as pod ENVIRONMENT only. A test proves it reaches the pod and appears nowhere else in the plan — not the seed list the pod parses, not the catalogue, not the args. Fixing this exposed a precedence bug: at provision time the plan's env was applied OVER the model's own env, so a token a model was explicitly configured with was silently replaced by whatever this process happened to have. The model's own env wins now. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The failure this prevents was observable, not hypothetical: a pod boots, pulls the image, starts downloading, fills its 40 GB disk and dies at load_timeout_s having paid for every minute. The size of a HuggingFace repo is one metadata call, so it is asked before the pod exists, and the disk is grown to fit — weights plus ~25 GB for the unpacked image and the download cache. It counts what a load will actually FETCH, not the whole repo. SDXL's repo is 77 GB, of which a diffusers load pulls 28: the rest is the same weights again as Flax .msgpack, legacy .bin and ONNX exports. Sizing to the repo would have rented 100 GB for a 28 GB model. An explicit container_disk_gb that is already large enough is respected. And with a network volume attached the weights land there, not on the container disk, so the disk stays at its configured size — which is the point of the volume. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
RunPod refuses images above a size line — measured: 10.1 GB boots, 14.5 GB is 'Exited by Runpod' two seconds after rent on every machine, no other reason given. Container disk (tried 80 GB), layer format and the full image config were each ruled out; size was the only difference. All five specialised images were above the line. They were above it for one reason: each was built on the full core, so it carried 7 GB of CUDA, torch and triton the main venv never used on that pod, beside the venv that did the work. The speaker image was 14.8 GB, of which pyannote's own venv was 5.6. The light core is coderai's Python dependencies with no GPU stack: 1.2 GB. codai.api.app imports with no torch present (verified first), so the main interpreter still boots the API, serves /healthz and /boot, and drives the subprocess worker. A profile marked <name>.light builds on it, its main venv carries only what the API layer needs, and the build FAILS if a torch reaches that venv — because that is exactly the leak that would put the image back over the line. speaker: 14,796 MB -> 7,077 MB, complete, nothing on a volume. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
A dead pod produced three hundred seconds of 'still no port' and no reason at all, because the one Pod field that carries the reason was never queried. It now travels into the progress line and the failure message, so an EXITED pod says what RunPod says about it instead of leaving the cause to be guessed. The strict-mode removal in boot.sh was one such guess: the image boots cleanly here with a minimal environment and with a GPU exposed, so it does not reproduce the exit. This is how the real cause gets found. Co-Authored-By:
-
- 15 Sep, 2026 26 commits
-
-
Stefy Lanza (nextime / spora ) authored
A speaker pod reported EXITED with uptime 0s from its very first poll. The loop waited out the full budget on it — three machines in a row — printing 'image pull in progress', which it never was: wait_ready treated only TERMINATED and FAILED as terminal. EXITED past a short restart grace now fails fast and says what it means: the image pulled and the process died on boot. The budget it waited out was 300s, not the 900s set on the pool class: the dataclass default was never updated. Two numbers for one thing, and the smaller one silently won. There is one now. Why the process died: boot.sh ran under set -u, and every line before the exec is a diagnostic — it exists to describe the pod, and a diagnostic must never be able to stop the boot it describes. The image boots fine locally; on RunPod an environment difference tripped a strict-mode abort before uvicorn was reached. That is the best-supported explanation, not a proven one — RunPod publishes no log API to confirm it — so the script is now unable to abort before its exec under any environment, verified against a bare sh with nothing set. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The 150 GB warning's call site in _provision_one used a name that did not exist in that scope. The helper was unit-tested and passed; every pod test then failed in two seconds with "name 'entry' is not defined" before renting anything. A helper can be correct while the line that calls it is not. The entry is looked up the way the rest of the pool does it. And since pyflakes sees exactly this class of bug, a test now runs it over the modules a pod boot goes through, so an undefined name in that path fails the suite instead of the first real request. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The venv built and then failed its import check with ModuleNotFoundError. Verified the whole spec in a throwaway container first this time: with click, 'from TTS.api import TTS' works on torch 2.5.1+cu124 with transformers 4.46.3 — the stack this image exists to provide. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Everything around pods is tuned for images of ~10 GB and models of a few. The native MoE engines are a different shape: DeepSeek-V4 is ~154 GB and they go well past that, so a pod spends one to two hours filling its disk before it answers anything — on EVERY cold start, since a pod is disposable storage. The warning names the model, the size, and the fix: attach a network volume so the weights are fetched once and every later pod attaches to them instead. It appears at provision time, boxed so it cannot be missed in a log, and in the test run — which is where someone looks BEFORE committing to a configuration, while at provision time the money is already being spent. It says nothing when a volume is attached, and nothing for ordinary models. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Diarization had no probe, so a speaker pod could only ever report reachability — and reachability is what reported ok:true for an STT image that shipped nothing for speaker work at all. espeak provides the audio; the check is that the model returns at least one labelled turn, which a pod with no diarization stack cannot do. One speaker saying one sentence is a thin test musically and an honest one here. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The pod images were built to satisfy imports, not to serve models. The audit found coqui/XTTS-v2, every whisper.cpp model, diarization, speaker ID, SALAD place recognition and Surya all unservable on a pod, and the OCR image shipping no OCR engine at all. Three of the eight 'isolated' stacks are not isolated any more. Surya, speechbrain and pytorch_lightning all import cleanly beside torch 2.11 — their venvs were decided against an older torch and never re-checked, exactly like audiocraft's torch==2.1.0. They go straight into the ocr, stt and embeddings profiles: place recognition, speaker ID and Surya now work for a few hundred MB instead of three new images. Surya's subprocess engine is pointed at the pod's own interpreter, since building a venv to hold what is already installed would only cost rental time. Five stacks genuinely cannot share, each verified rather than assumed: pyannote (torchaudio.AudioMetaData is gone in 2.11), coqui-tts (a transformers 4.x symbol), CrisperWhisper (transformers 4.46 AND torch 2.5), NeMo (large, own torch) and paddle (its own CUDA runtime). Each gets an image carrying a baked venv, rather than 6 GB added to an image everyone pulls. The `speaker` capability no longer points at the STT image: that was the assumption that it was 'the STT deps plus a bit', and it shipped nothing for diarization or voiceprints at all. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The OCR pod image installed NO OCR engine — only transformers and pypdfium2. A pod could boot, pass its health check, accept a request and have nothing to do it with. Three gates in front of that each reported their own refusal first — 'OCR subsystem is disabled', then 'OCR engine surya is not enabled' — so it took three rounds to see that the engine was never there. docTR goes in, because it is the only one of the three that runs in-process. Surya caps pillow<11 and lives in an isolated venv by design; paddle wants paddlepaddle in one of its own. Either would put a second torch in the image — 6 GB to avoid a Pillow conflict — and the venv mechanism is there if a pod ever needs Surya's layout quality specifically. Each engine also has its own enable flag on top of the subsystem's, and two of the three default to off. A pod now enables the engine it was asked for, falls back to docTR when the request names something the image cannot serve, and still inherits the Surya licence decision from here rather than making it itself. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The isolated venv was solving a problem that was not real. audiocraft is pure Python: its torch==2.1.0 is a declared pin, not an ABI requirement, so --no-deps sidesteps it. The only genuine constraint was xformers, which it imports at module load — and 0.0.35 ships a py39-none wheel requiring torch>=2.10, which this image satisfies. I had ruled that out by grepping wheel filenames for cp312 and missing that abi3/py39 wheels cover 3.12. Verified in the built core image: torch 2.11.0+cu128 with xformers 0.0.35, MusicGen and AudioGen import, ops.memory_efficient_attention present. Cost in the main venv: 950 MB. The second venv cost 6.2 GB, because it carried a duplicate torch and a duplicate CUDA stack. The audio image drops from 15.8 GB to about 10.2 GB, which also removes the reason to split it in two. audio_backend (per model, or CODERAI_AUDIO_BACKEND for a pod) selects audiocraft or transformers explicitly, because they are not equivalent — melody conditioning and AudioGen exist only in the first. Asking for audiocraft when it is absent raises rather than quietly substituting something less capable. A model's own setting beats the environment it lands in, and the choice travels to a pod instead of being decided by what happens to be installed there. The isolated-venv worker stays in the tree: if audiocraft 1.3.0 turns out to misbehave against torch 2.11 at generation time, restoring one profile file brings it back. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Three build cycles, three different missing modules — xformers, then torch at install time, then transformers. The list had been trimmed against what the MAIN venv provides, which is exactly backwards: an isolated venv starts with nothing, so it needs the whole declared set, minus only the torch pins we choose and the dev-only tools. Verified in a throwaway container against the real core image before building anything: audiocraft 1.3.0 on torch 2.5.1+cu121 with xformers 0.0.28.post3, and MusicGen and AudioGen both import. That check takes two minutes; the image build it replaces takes fifteen, which is why the last three rounds were so expensive. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Voice cloning failed with 'Could not load libtorchcodec', whose message blames FFmpeg and the torch version. It was neither: the CUDA build links against NVIDIA Performance Primitives (libnppicc.so.12), torch does not use NPP, and torchcodec arrives as a transitive dependency of f5-tts — so nothing ever installed it. Installing it is only half: torch preloads its own nvidia libs and NPP is not among them, so the library sits in site-packages and the loader never looks there. LD_LIBRARY_PATH covers the other half. Verified in the built image: torch 2.11.0+cu128, torchcodec 0.11.1+cu128, AudioDecoder imports. Two wrong theories preceded this — an FFmpeg major mismatch and a torch/torchcodec ABI mismatch — both suggested by that error message, and both disproved by loading each .so by hand with torch imported. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Second failure in that venv: 'No module named torch' while installing. xformers below 0.0.27 has no cp312 wheel, so pip tried to build it from source, and its setup.py imports torch — which does not exist yet in a fresh venv. The binding constraint was never audiocraft's torch==2.1.0 (unsatisfiable on this Python) but xformers, which audiocraft imports at module load: its cp312 wheels start at 0.0.27, and 0.0.28.post3 requires torch==2.5.1 exactly. So the venv is torch 2.5.1 + xformers 0.0.28.post3 — chosen by what actually has wheels for this interpreter rather than by what the libraries would prefer. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Dropped with the rest of its unsatisfiable pins, but audiocraft imports xformers at module load, so the venv built cleanly and then failed its import check with ModuleNotFoundError — which is what that check exists for: on a rented GPU this would have surfaced minutes into a generation instead. Its pin (<0.0.23) pairs with the torch it cannot have here; 0.0.25.post1 is the build that matches torch 2.2.2. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The gateway fix worked — the OCR request reached the pod, which booted, took it, and answered 'OCR subsystem is disabled (enable it in Settings → OCR)'. That is the right default for a fresh install and the wrong one for a machine that exists for the next four minutes and was rented specifically to do OCR. An ocr-capability pod now gets CODERAI_OCR_ENABLED, the engine it was asked for, and — when that engine is surya, or this deployment has already accepted it — the licence acknowledgement, because surya is GPL-gated and a pod cannot make that decision for itself. Asking for surya by name is the decision. Env only, never written to config.json: like the seed list, this describes one disposable instance, not a deployment. A test asserts it cannot leak to disk. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The audio build failed with 'No matching distribution found for torch==2.1.0': audiocraft's pin is unsatisfiable on Python 3.12 at all, because torch support for 3.12 starts at 2.2. So the venv uses the oldest torch this Python can run and installs audiocraft --no-deps, with the deps it actually uses listed explicitly. --no-deps is reckless in a shared venv and routine in an isolated one: nothing in there can reach the server's stack, and the build's import check fails the image if the version judgement turns out to be wrong. The step is now generic — any <profile>.venv-<name>.txt builds /opt/coderai/venvs/<name>, with an optional .nodeps beside it — rather than hardcoding audiocraft into the Dockerfile. The next library that pins an impossible torch will not need a new build step. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
0.2.14 was published for three profiles before the audiocraft venv landed, so that number now means different things for different images. Skipping it keeps a version honest: one tag, one tree. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The transformers fallback keeps MusicGen working on a pod, but it is not the same thing: audiocraft has melody conditioning and AudioGen, and transformers has neither. Supporting it is not a matter of adding a requirement — it pins torch==2.1.0, torchvision==0.16.0 and torchtext==0.16.0, and torch 2.1.0 has no build for the CUDA these images run, so installing it beside the server would leave the pod with no working GPU torch at all. So it gets its own venv and is driven as a subprocess, the same answer parler-tts and the OCR engines already use here. AudiocraftModel deliberately wears audiocraft's own interface (set_generation_params / generate / generate_with_chroma / sample_rate) so the generation path cannot tell whether the library is in this process, in a venv next door, or absent. The venv is baked into the audio image rather than built on first use: a pod is rented by the second, and pip-installing a library during a cold start costs more than carrying the layer. Order of preference: in-process audiocraft, then the isolated venv, then transformers — which needs no venv but cannot do melody. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
None of these were placement or pod bugs: each image booted, pulled its model and got as far as executing. The remote machinery worked; the payloads did not. voice — torchcodec arrived as a transitive dependency of f5-tts, PyPI served the CUDA-13 build, and it died with 'libnvrtc.so.13: cannot open shared object file' after loading the model. The same class of bug as the torch pin: a CUDA-coupled package escaping the index. It now installs from the CUDA index with the rest, resolving to torchcodec 0.11.1+cu128. audio — 'No module named audiocraft'. audiocraft pins an old torch, so putting it in the image would drag torch backwards and undo the CUDA pin. transformers ships MusicGen itself, so a pod serves it through transformers instead; an install that has audiocraft still uses it. Melody conditioning stays audiocraft-only rather than silently doing something different. video — 'TextToVideoSDPipeline.__call__() got an unexpected keyword argument callback_on_step_end'. Older diffusers pipelines do not take it and raise inside the call, minutes in, with the model loaded and the GPU paid for. The kwarg is now added only when the pipeline's signature accepts it. The try/except that used to wrap it caught nothing: assigning to a dict never raises. This one bites local generations too, not just pods. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
/v1/ocr names its engine in a multipart field called `engine`, not `model`. The gateway only ever looked for `model`, so an OCR request carried no model as far as it was concerned: per-model placement resolved correctly, the test reported 'target: its own pod', and then the gateway refused with 'nothing configures ocr to run remotely'. Two layers disagreeing about the same request. _MODEL_FIELD maps the endpoints whose field is named something else, and the extractor takes the field name for JSON, multipart and urlencoded alike. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Voice cloning runs on a TTS model (XTTS, F5) but is a different endpoint served by a different pod image, so section alone maps it to 'tts' and it could never be placed as 'voice'. That is why the voice image had never been tested: not a missing probe, a model that could not be pointed anywhere. The harness also gains --add-as, to create a temporary catalogue entry for a capability nobody has a model configured for (music generation, a small video model), instead of requiring the real catalogue to be edited by hand first. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Three capabilities were listed as un-probeable. Only one of the reasons held. Voice cloning 'needs a reference sample' — and espeak produces one, with a transcript we know exactly because we chose the sentence. Audio generation was 'minutes of GPU': asking for 2 seconds instead of the 10s default proves the model renders audio for a fifth of the cost. Video was 'real money': the smallest clip a model can make — 9 frames, 256x256, 4 steps — proves the path without paying for a film. 'Expensive' was an argument for asking for less, not for proving nothing. Generated media has no ground truth, so these check that media came back at all; a 200 carrying nothing is still a failure. Face swap stays reachability-only and now says why: a drawn face does not survive face detection, and a real person's photo is not ours to use. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
OCR engines are picked by name (paddle|doctr|surya) rather than registered like a model, so nothing could place one individually and the test for them failed with "not in models.json" before it ever reached a pod. The only way to send OCR to a pod was a capability remote — a production routing switch, and the mechanism that once sent live embedding traffic to a rented GPU. An ocr_models section maps to the ocr capability, so an engine gets the same per-model placement every other model type already has, and the harness creates a temporary entry for one when the catalogue has none. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
--extra-index-url only ADDS an index; pip then picks the highest version across all of them, and PyPI's default torch is built for CUDA 13. So the pin never applied: every capability image shipped a CUDA-13 torch and failed on any machine with an older driver — 'The NVIDIA driver on your system is too old (found version 12080)'. RunPod's fleet is mixed, so the same GPU model passed or failed depending on which machine the pod landed on, which is exactly why this looked like flakiness for so long. torch, torchvision and torchaudio now install from the CUDA index alone, via --index-url, before anything else; installed later from PyPI they would pull a matching CUDA-13 torch back in. cu128 resolves to torch 2.11 — the version the local install already runs — and runs on the driver that failed. Both Dockerfiles print the resolved torch build at each layer, so a silent substitution cannot happen again. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The test run synthesises a spoken sentence to probe an STT model with real audio rather than a reachability check. espeak was never in the image, so the probe fell back to "configured, not verified" — correct behaviour (the gap is ours, not the model's) and therefore invisible: no error, no failing test. Found by inspecting a built image. Added to the full build and to the thin update layer, so it is present whichever path produced the running image. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
A pod could not say WHICH build it was running, which is exactly the question a cached ':latest' raises — an images pod reported a dependency missing that had already been published under that tag. The build now bakes IMAGE_TAG in, so the boot record names the image. /boot also showed only 'lifespan start': the seed registration and the end of startup were plain prints. They are phases now, so one request covers the whole boot including how many seeded models registered and whether that step failed. Co-Authored-By:
-