images: a light core, so a specialised image carries ONE GPU stack, not two

RunPod refuses images above a size line — measured: 10.1 GB boots, 14.5 GB is
'Exited by Runpod' two seconds after rent on every machine, no other reason
given. Container disk (tried 80 GB), layer format and the full image config were
each ruled out; size was the only difference. All five specialised images were
above the line.

They were above it for one reason: each was built on the full core, so it
carried 7 GB of CUDA, torch and triton the main venv never used on that pod,
beside the venv that did the work. The speaker image was 14.8 GB, of which
pyannote's own venv was 5.6.

The light core is coderai's Python dependencies with no GPU stack: 1.2 GB.
codai.api.app imports with no torch present (verified first), so the main
interpreter still boots the API, serves /healthz and /boot, and drives the
subprocess worker. A profile marked <name>.light builds on it, its main venv
carries only what the API layer needs, and the build FAILS if a torch reaches
that venv — because that is exactly the leak that would put the image back over
the line.

speaker: 14,796 MB -> 7,077 MB, complete, nothing on a volume.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY

packaging/runpod/Dockerfile.capability

@@ -44,7 +44,13 @@ RUN --mount=type=cache,target=/root/.cache/pip set -eux; \
         apt-get update && apt-get install -y --no-install-recommends $deps; \
     fi; \
     python -m pip install --extra-index-url ${TORCH_INDEX} -r /tmp/profiles/${PROFILE}.txt; \
+    if [ -f "/tmp/profiles/${PROFILE}.light" ]; then \
+        if python -c "import torch" 2>/dev/null; then \
+            echo "ERROR: ${PROFILE} is a light-core image but torch reached its main venv" >&2; exit 1; \
+        fi; echo "main venv: no torch (light core, as intended)"; \
+    else \
         python -c "import torch; print('torch', torch.__version__, 'cuda', torch.version.cuda)"; \
+    fi; \
     if [ -f "/tmp/profiles/${PROFILE}.nodeps" ]; then \
         python -m pip install --no-deps -r "/tmp/profiles/${PROFILE}.nodeps"; \
         for pkg in $(grep -vE '^\s*(#|$)' "/tmp/profiles/${PROFILE}.nodeps" | sed 's/[<>=!].*//'); do \

packaging/runpod/Dockerfile.capability-base-light

+# The LIGHT core: coderai's Python dependencies with no GPU stack at all.
+#
+# For images whose capability runs in a venv of its own. RunPod refuses images
+# above a size threshold — measured: a 10.1 GB image boots, a 14.5 GB one is
+# "Exited by Runpod" two seconds after rent, on every machine, with no other
+# reason given (container disk, layer format and image config were each ruled
+# out; size was the only difference). The specialised images were all above the
+# line because each carried the FULL core — 7 GB of CUDA, torch and triton that
+# the main venv never used on that pod — beside the venv that did the work.
+#
+# Starting here, the capability's own venv is the only GPU stack in the image.
+# The main interpreter still boots the API, serves /healthz and /boot, and
+# drives the subprocess worker: codai.api.app imports with no torch present,
+# verified before this file was written.
+#
+#   docker build -f packaging/runpod/Dockerfile.capability-base-light \
+#       -t coderai-capability-base-light:latest .
+
+ARG BASE=python:3.12-slim
+FROM ${BASE}
+
+ENV PYTHONUNBUFFERED=1 \
+    HF_HOME=/cache/huggingface \
+    CODERAI_HOME=/config
+
+# ffmpeg for every audio/video path; git because a few ML packages still want it
+# at install time; libgl/libglib for opencv.
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        ffmpeg git curl ca-certificates libgl1 libglib2.0-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /opt/coderai/app
+
+COPY packaging/runpod/profiles/core-light.txt /tmp/core.txt
+# No torch here, on purpose — and the build FAILS if one arrives anyway, because
+# a transitive dependency pulling torch in would silently put the image right
+# back over the size line it exists to stay under.
+RUN --mount=type=cache,target=/root/.cache/pip \
+    python -m pip install --upgrade pip \
+    && python -m pip install -r /tmp/core.txt \
+    && if python -c "import torch" 2>/dev/null; then \
+           echo "ERROR: torch leaked into the light core" >&2; exit 1; \
+       fi \
+    && echo "light core: no torch, as intended" \
+    && rm -f /tmp/core.txt

packaging/runpod/build_capability_image.sh

@@ -44,10 +44,21 @@ fi
 # images share those layers by construction — the registry stores and transfers
 # the core once instead of nine near-identical ~7 GB copies. Rebuild it with
 # REBUILD_CORE=1 (needed when core.txt changes).
-CORE_BASE="${CORE_BASE:-coderai-capability-base:latest}"
+# A profile that runs its capability in a venv of its own starts from the LIGHT
+# core — coderai's deps and no GPU stack — marked by a `<profile>.light` file.
+# RunPod refuses images above a size line (10.1 GB boots, 14.5 GB is refused),
+# and a specialised image on the full core carried 7 GB of torch and CUDA its
+# main venv never touched. See Dockerfile.capability-base-light.
+if [[ -f "packaging/runpod/profiles/${PROFILE}.light" ]]; then
+    CORE_BASE="${CORE_BASE:-coderai-capability-base-light:latest}"
+    CORE_DOCKERFILE=packaging/runpod/Dockerfile.capability-base-light
+else
+    CORE_BASE="${CORE_BASE:-coderai-capability-base:latest}"
+    CORE_DOCKERFILE=packaging/runpod/Dockerfile.capability-base
+fi
 if [[ "${REBUILD_CORE:-0}" == "1" ]] || ! docker image inspect "$CORE_BASE" >/dev/null 2>&1; then
     echo "==> building the shared core $CORE_BASE"
-    DOCKER_BUILDKIT=1 docker build -f packaging/runpod/Dockerfile.capability-base \
+    DOCKER_BUILDKIT=1 docker build -f "$CORE_DOCKERFILE" \
         -t "$CORE_BASE" . || { echo "core build failed" >&2; exit 1; }
 else
     echo "==> reusing the shared core $CORE_BASE"

packaging/runpod/profiles/core-light.txt

+# The core WITHOUT the GPU stack — for images whose capability runs in a venv
+# of its own.
+#
+# RunPod refuses images above a size threshold with a two-second "Exited by
+# Runpod" and no other reason: a 14.8 GB speaker image was rejected on three
+# machines while 10.1 GB images boot fine (disk, layer format and image config
+# were each ruled out; size is the only difference). The five specialised images
+# were all above the line because each carried the full core — 7 GB of CUDA and
+# torch the main venv never uses on that pod — beside the venv that actually
+# does the work.
+#
+# So a specialised image starts from THIS: coderai's Python dependencies and
+# nothing GPU-side. The capability's own venv brings the one torch it needs.
+# The main interpreter can still boot the API, serve /healthz and /boot, and
+# drive the subprocess worker — none of that needs torch.
+# Core — what every capability pod needs to boot coderai and answer /healthz.
+# The routers import lazily, so a pod only needs the libraries its own capability
+# actually touches; this file is the floor under all of them.
+# torch lives here rather than in each profile so it becomes ONE shared layer:
+# nine images that each built their own copy would duplicate ~5 GB nine times,
+# on disk and on every push. Profiles add torchvision/torchaudio as they need.
+
+fastapi>=0.104.0
+uvicorn[standard]>=0.24.0
+pydantic>=2.5.0
+python-multipart>=0.0.6
+requests>=2.31.0
+psutil>=5.9.0
+setproctitle>=1.1
+Pillow>=10.0.0
+numpy>=1.24.0
+safetensors>=0.4.0
+huggingface_hub>=0.25.0
+
+# Imported at MODULE level somewhere in the graph `import codai.api.app` walks,
+# so every pod needs them regardless of profile — app.py mounts every router, and
+# a few of those touch these at import time rather than lazily:
+#   cv2, PIL   codai/api/faceswap.py
+#   soundfile  codai/api/voice_convert.py
+#   litellm    codai/models/parser.py
+#   websockets codai/broker/client.py
+# Verified by a static walk of the import graph, and enforced by the Dockerfile's
+# `import codai.api.app` build step. torch/transformers are NOT here: those are
+# imported lazily, so they stay per-profile.
+opencv-python-headless>=4.8.0
+soundfile>=0.12.0
+litellm>=1.40.0
+websockets>=12.0
+
+# Quantization. models.json carries quant settings for EVERY model type, not just
+# LLMs, and diffusers/transformers refuse to load a quantized checkpoint without
+# this — a rented images pod failed with "No package metadata was found for
+# bitsandbytes" after a 10-minute SDXL download. Shared here rather than per
+# profile: one layer, downloaded once, reused by all ten images.
+
+# torchcodec's CUDA build links against NVIDIA Performance Primitives
+# (libnppicc.so.12). torch does not use NPP, so nothing else pulls it in, and
+# torchcodec — which arrives as a transitive dependency of f5-tts — then fails
+# to load with a message that blames FFmpeg and the torch version. It is neither.

packaging/runpod/profiles/ocr-paddle.light

+# This profile's capability runs in its own venv: build on the light core.

packaging/runpod/profiles/ocr-paddle.txt

-# /v1/ocr with PaddleOCR + PP-Structure (layout, tables, reading order).
-#
-# paddlepaddle-gpu ships its OWN CUDA runtime, not just its own torch — the one
-# stack here that cannot share anything with the main venv.
-transformers>=4.35.0
+# Main-venv deps for a LIGHT-core image: only what the API layer needs to
+# accept the request and hand it to the subprocess worker. No torch, no
+# torchaudio, no transformers here — those live in the capability's own venv
+# (the .venv-*.txt beside this file), and on the light core a torch arriving in
+# the main venv fails the build on purpose: it is the 7 GB that put these images
+# over RunPod's size line in the first place.
+# /v1/ocr with PaddleOCR, in the paddleocr venv. pypdfium2 rasterises PDFs API-side.
 pypdfium2>=4.20.0
-python-doctr>=0.8.0
 jsonschema>=4.0.0

packaging/runpod/profiles/speaker.light

+# This profile's capability runs in its own venv: build on the light core.

packaging/runpod/profiles/speaker.txt

-# /v1/audio/diarization, /v1/audio/speaker* — pyannote + voiceprints.
-#
-# A SEPARATE image because pyannote.audio 3.3.2 calls
-# torchaudio.AudioMetaData, which torchaudio 2.11 removed. It is not a
-# conservative pin like audiocraft's was — the attribute is genuinely gone — so
-# it needs an interpreter on an older torch, and that means its own venv (see
-# speaker.venv-pyannote.txt) and its own image rather than 6 GB added to the
-# stt image that most users pull for transcription alone.
-torchaudio
-transformers>=4.35.0
+# Main-venv deps for a LIGHT-core image: only what the API layer needs to
+# accept the request and hand it to the subprocess worker. No torch, no
+# torchaudio, no transformers here — those live in the capability's own venv
+# (the .venv-*.txt beside this file), and on the light core a torch arriving in
+# the main venv fails the build on purpose: it is the 7 GB that put these images
+# over RunPod's size line in the first place.
+# /v1/audio/diarization, /v1/audio/speaker* — pyannote + speechbrain, both in the pyannote venv.
 soundfile>=0.12.0
 scipy>=1.11.0
-speechbrain>=1.0.0

packaging/runpod/profiles/speaker.venv-pyannote.txt

@@ -7,3 +7,5 @@ torchaudio==2.6.0+cu124
 pyannote.audio==3.3.2
 huggingface_hub<1.0
 soundfile>=0.12.0
+# speechbrain (speaker ID) lives here too: this venv is where the torch is.
+speechbrain>=1.0.0

packaging/runpod/profiles/stt-crisper.light

+# This profile's capability runs in its own venv: build on the light core.

packaging/runpod/profiles/stt-crisper.txt

-# /v1/audio/transcriptions with CrisperWhisper (verbatim, disfluency-aware).
-#
-# It pins transformers==4.46.3 AND torch==2.5.1 — a conflict on both axes with
-# the main images, so it cannot share one.
-torchaudio
+# Main-venv deps for a LIGHT-core image: only what the API layer needs to
+# accept the request and hand it to the subprocess worker. No torch, no
+# torchaudio, no transformers here — those live in the capability's own venv
+# (the .venv-*.txt beside this file), and on the light core a torch arriving in
+# the main venv fails the build on purpose: it is the 7 GB that put these images
+# over RunPod's size line in the first place.
+# /v1/audio/transcriptions with CrisperWhisper, in the transformers venv.
 soundfile>=0.12.0
 scipy>=1.11.0

packaging/runpod/profiles/stt-nemo.light

+# This profile's capability runs in its own venv: build on the light core.

packaging/runpod/profiles/stt-nemo.txt

-# /v1/audio/transcriptions with NVIDIA NeMo Canary.
-#
-# nemo_toolkit[asr] brings a large stack and its own torch expectations. It gets
-# an image rather than 6 GB on the stt image that most transcription uses.
-torchaudio
-transformers>=4.35.0
+# Main-venv deps for a LIGHT-core image: only what the API layer needs to
+# accept the request and hand it to the subprocess worker. No torch, no
+# torchaudio, no transformers here — those live in the capability's own venv
+# (the .venv-*.txt beside this file), and on the light core a torch arriving in
+# the main venv fails the build on purpose: it is the 7 GB that put these images
+# over RunPod's size line in the first place.
+# /v1/audio/transcriptions with NVIDIA NeMo Canary, in the nemo venv.
 soundfile>=0.12.0
 scipy>=1.11.0

packaging/runpod/profiles/tts-xtts.light

+# This profile's capability runs in its own venv: build on the light core.

packaging/runpod/profiles/tts-xtts.txt

-# /v1/audio/speech and /v1/audio/clone with coqui XTTS-v2.
-#
-# coqui-tts imports isin_mps_friendly from transformers.pytorch_utils, removed
-# in transformers 5.x. The main images run 5.17, and pinning 4.x there would
-# break every other capability — so XTTS gets an image whose venv can be on 4.x.
-torchaudio
+# Main-venv deps for a LIGHT-core image: only what the API layer needs to
+# accept the request and hand it to the subprocess worker. No torch, no
+# torchaudio, no transformers here — those live in the capability's own venv
+# (the .venv-*.txt beside this file), and on the light core a torch arriving in
+# the main venv fails the build on purpose: it is the 7 GB that put these images
+# over RunPod's size line in the first place.
+# /v1/audio/speech and /v1/audio/clone with coqui XTTS-v2, in the TTS venv.
 soundfile>=0.12.0
 scipy>=1.11.0