docker/backend: graceful llama-cpp load + additive GPU modes + libcuda...

docker/backend: graceful llama-cpp load + additive GPU modes + libcuda mapping; admin GGUF batch/slots tuning

Backend robustness:
- vulkan.py catches Exception (not just ImportError) around the llama_cpp
  import: a CUDA-built llama-cpp missing libcuda.so.1 raised RuntimeError/OSError
  that crash-looped the whole server. Now it logs a warning and marks the
  Vulkan/GGUF backend unavailable; CUDA/CPU/ds4 keep working.
- detect_available_backends() reads LLAMA_CPP_AVAILABLE instead of re-importing
  (which re-raised the same error).

Docker launcher (run_oci.sh):
- GPU backends are now additive: --nvidia --vulkan enables both (maps libcuda via
  --gpus all AND /dev/dri). Added --all and --with-libcuda[=PATH].
- --vulkan auto bind-mounts the host's libcuda.so.1 (the bundled llama-cpp is a
  CUDA build), so Vulkan GGUF loads without full --gpus all. Banner shows mode set
  and libcuda status.

Dist bundle:
- New uninstall.sh (removes runner + optional image), wired into make_dist_bundle.
- install.sh + uninstall.sh print what they'll do and confirm before proceeding,
  bypassable with --yes/-y.

Admin GGUF tuning:
- Expose n_batch / n_ubatch / n_seq_max (llama.cpp -b/-ub/-np) in the model config
  UI and apply them in the Vulkan backend to shrink VRAM at the ceiling; n_seq_max
  gated on llama-cpp-python support.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

codai/admin/routes.py

@@ -2747,7 +2747,8 @@ async def api_model_configure(request: Request, username: str = Depends(require_
                 "max_vram", "sdcpp_flash_attn", "sdcpp_diffusion_flash_attn", "vae_tiling",
                 "component_quantization", "output_crf", "force_vram_update",
                 "balanced_gpu_percent", "acceleration",
-                "cache_type_k", "cache_type_v", "kv_offload", "turboquant", "engine", "engine_fallback",
+                "cache_type_k", "cache_type_v", "kv_offload", "n_batch", "n_ubatch", "n_seq_max",
+                "turboquant", "engine", "engine_fallback",
                 "quant_backend", "kv_cache_budget_mb", "kv_cache_slots", "mmproj",
                 "auto_compact", "auto_compact_pct", "auto_compact_strategy",
                 "auto_compact_model", "suppress_reasoning"):

codai/admin/templates/models.html

@@ -648,6 +648,16 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
           <label class="form-label">Context size</label>
           <input type="number" id="cfg-n-ctx" class="form-input" min="128" step="128" value="2048">
         </div>
+        <div class="form-row" style="margin:0" id="cfg-n-batch-row">
+          <label class="form-label">Batch size <span class="muted">(GGUF; llama.cpp -b)</span></label>
+          <input type="number" id="cfg-n-batch" class="form-input" min="1" step="1" placeholder="auto (512)">
+          <span class="form-hint" style="font-size:11px">Prompt-ingestion batch. Lowering it (e.g. <b>256</b>) shrinks the compute buffer so a big-context model fits at the VRAM ceiling — slower prefill. Blank = default (512).</span>
+        </div>
+        <div class="form-row" style="margin:0" id="cfg-n-seq-max-row">
+          <label class="form-label">Parallel slots <span class="muted">(GGUF; llama.cpp -np / n_seq_max)</span></label>
+          <input type="number" id="cfg-n-seq-max" class="form-input" min="1" step="1" placeholder="auto (1)">
+          <span class="form-hint" style="font-size:11px">Concurrent sequence slots. Each slot reserves its own KV/compute VRAM; keep at <b>1</b> to avoid reserving for unused parallelism. Blank = default (1). Ignored by older llama-cpp-python builds.</span>
+        </div>
         <div class="form-row" style="margin:0" id="cfg-kv-k-row">
           <label class="form-label">KV cache — Keys <span class="muted">(GGUF text; shrinks KV VRAM)</span></label>
           <select id="cfg-cache-type-k" class="form-input">
@@ -3218,6 +3228,8 @@ function openCfgModal(idx, cfgIdx){
   document.getElementById('cfg-force-vram-update').checked = !!s.force_vram_update;
   document.getElementById('cfg-gpu-layers').value = s.n_gpu_layers !== undefined ? s.n_gpu_layers : -1;
   document.getElementById('cfg-n-ctx').value = nCtxForEst;
+  document.getElementById('cfg-n-batch').value = s.n_batch != null ? s.n_batch : '';
+  document.getElementById('cfg-n-seq-max').value = s.n_seq_max != null ? s.n_seq_max : '';
   document.getElementById('cfg-cache-type-k').value = s.cache_type_k || '';
   document.getElementById('cfg-cache-type-v').value = s.cache_type_v || '';
   _populateMmprojSelect(m, s);
@@ -3626,6 +3638,8 @@ async function saveModelConfig(){
     preload_all_instances: document.getElementById('cfg-preload-all-instances').checked,
     n_gpu_layers:      parseInt(document.getElementById('cfg-gpu-layers').value) || -1,
     n_ctx:             parseInt(document.getElementById('cfg-n-ctx').value) || 2048,
+    n_batch:           parseInt(document.getElementById('cfg-n-batch').value) || null,
+    n_seq_max:         parseInt(document.getElementById('cfg-n-seq-max').value) || null,
     cache_type_k:      document.getElementById('cfg-cache-type-k').value || null,
     cache_type_v:      document.getElementById('cfg-cache-type-v').value || null,
     mmproj:            document.getElementById('cfg-mmproj').value || null,

codai/backends/__init__.py

@@ -33,12 +33,13 @@ def detect_available_backends():
     except ImportError:
         pass
     
-    # Check for llama-cpp-python (Vulkan)
-    try:
-        import llama_cpp
+    # Check for llama-cpp-python (Vulkan / GGUF). Use the flag computed at import
+    # time so a llama-cpp build that fails its shared-library load (e.g. a CUDA
+    # build missing libcuda.so.1) is reported unavailable instead of re-raising
+    # the RuntimeError/OSError here and aborting backend detection.
+    from codai.backends.vulkan import LLAMA_CPP_AVAILABLE
+    if LLAMA_CPP_AVAILABLE:
         backends['vulkan'] = True
-    except ImportError:
-        pass
     
     return backends
 

codai/backends/vulkan.py

@@ -97,11 +97,35 @@ try:
     from llama_cpp.llama_chat_format import ChatFormatterResponse
     import llama_cpp as _llama_cpp
     LLAMA_CPP_AVAILABLE = True
-except ImportError:
+except Exception as _llama_import_err:
+    # Catch more than ImportError: a llama-cpp-python built against CUDA raises
+    # RuntimeError/OSError (e.g. "libcuda.so.1: cannot open shared object file")
+    # when the NVIDIA driver libs aren't present — as in Vulkan/CPU-only runs.
+    # That must NOT crash the whole server import chain; the Vulkan backend just
+    # becomes unavailable and other backends (CUDA/CPU/ds4) keep working.
     LLAMA_CPP_AVAILABLE = False
     Llama = None
     ChatFormatterResponse = None
     _llama_cpp = None
+    if not isinstance(_llama_import_err, ImportError):
+        import logging as _logging
+        _logging.getLogger(__name__).warning(
+            "llama-cpp-python present but failed to load (%s); Vulkan/GGUF "
+            "backend disabled. If you expect GPU GGUF, ensure the matching GPU "
+            "runtime libs are mapped into this environment.", _llama_import_err)
+
+
+def _llama_accepts(param: str) -> bool:
+    """True when the installed llama-cpp-python ``Llama`` constructor accepts a
+    given keyword. Used to gate kwargs (e.g. ``n_seq_max``) that only some builds
+    expose, so passing them never raises ``TypeError`` on older bindings."""
+    if Llama is None:
+        return False
+    try:
+        import inspect
+        return param in inspect.signature(Llama.__init__).parameters
+    except (TypeError, ValueError):
+        return False
 
 
 # Friendly KV-cache quant names → llama.cpp GGML type. q8_0 is near-lossless and
@@ -951,6 +975,44 @@ class VulkanBackend(ModelBackend):
             print("  KV cache: offload_kqv=False — KV held in host RAM (saves VRAM, "
                   "slower decode)")
 
+        # Batch size (llama.cpp -b / n_batch) and physical micro-batch (-ub /
+        # n_ubatch). The compute/graph buffer reserved for prompt ingestion scales
+        # with the micro-batch, so lowering it shrinks a large VRAM allocation (the
+        # buffer that ggml_gallocr fails to reserve when a big-context model is right
+        # at the VRAM ceiling) at the cost of slower prefill. llama.cpp clamps
+        # n_ubatch to <= n_batch, so setting n_batch alone also caps the micro-batch.
+        _n_batch = kwargs.get('n_batch', _raw_cfg.get('n_batch'))
+        if _n_batch:
+            try:
+                llama_kwargs['n_batch'] = int(_n_batch)
+                print(f"  batch        : n_batch={int(_n_batch)} (smaller prompt-ingest buffer)")
+            except (TypeError, ValueError):
+                pass
+        _n_ubatch = kwargs.get('n_ubatch', _raw_cfg.get('n_ubatch'))
+        if _n_ubatch:
+            try:
+                llama_kwargs['n_ubatch'] = int(_n_ubatch)
+            except (TypeError, ValueError):
+                pass
+
+        # Parallel sequence slots (llama.cpp -np / n_seq_max). Each slot reserves its
+        # own share of KV + compute VRAM; keeping it at 1 avoids reserving VRAM for
+        # concurrent sequences we don't serve. Only newer llama-cpp-python builds
+        # accept this kwarg, so pass it only when the constructor supports it.
+        _n_seq = kwargs.get('n_seq_max', _raw_cfg.get('n_seq_max'))
+        if _n_seq:
+            try:
+                _n_seq = int(_n_seq)
+            except (TypeError, ValueError):
+                _n_seq = None
+            if _n_seq:
+                if _llama_accepts('n_seq_max'):
+                    llama_kwargs['n_seq_max'] = _n_seq
+                    print(f"  slots        : n_seq_max={_n_seq} (fewer parallel-slot VRAM reserves)")
+                else:
+                    print(f"  slots        : n_seq_max={_n_seq} requested but llama-cpp-python "
+                          f"{getattr(_llama_cpp, '__version__', '?')} doesn't expose it — ignoring")
+
         # Multimodal projector (mmproj): pairs a CLIP/vision projector GGUF with
         # this text model so it can accept images — the llama.cpp `--mmproj`
         # equivalent, which adds vision capability (e.g. gemma). Uses llama.cpp's

packaging/linux/dist-bundle/install.sh

@@ -8,7 +8,7 @@
 #        - user  -> ~/.local/usr/bin/coderai-docker  (and ensures it's on PATH,
 #                   adding it to ~/.bashrc if missing).
 #
-# Usage:  ./install.sh
+# Usage:  ./install.sh [--yes]
 # Env:    CONTAINER_ENGINE=docker|podman   (default docker)
 set -euo pipefail
 
@@ -16,10 +16,29 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 IMAGE_TAR="${IMAGE_TAR:-$HERE/coderai-dist.tar.gz}"
 RUNNER_SRC="${RUNNER_SRC:-$HERE/coderai-docker}"
 ENGINE="${CONTAINER_ENGINE:-docker}"
+ASSUME_YES=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -y|--yes) ASSUME_YES=1; shift ;;
+    -h|--help) sed -n '2,12p' "$0"; exit 0 ;;
+    *) printf 'Error: unknown option: %s\n' "$1" >&2; exit 2 ;;
+  esac
+done
 
 say(){ printf '%s\n' "$*"; }
 die(){ printf 'Error: %s\n' "$*" >&2; exit 1; }
 
+# Tell the user what this will do, and confirm before proceeding.
+if [ "$(id -u)" -eq 0 ]; then _bin="/usr/local/bin"; else _bin="$HOME/.local/usr/bin"; fi
+say "This will load the CoderAI image into $ENGINE and install the 'coderai-docker'"
+say "runner to $_bin. Your runtime data and config are not touched."
+if [ "$ASSUME_YES" -ne 1 ]; then
+  printf 'Proceed? [y/N] '
+  read -r reply </dev/tty || reply=""
+  case "$reply" in y|Y|yes|YES) ;; *) die "aborted by user." ;; esac
+fi
+
 command -v "$ENGINE" >/dev/null 2>&1 || die "'$ENGINE' not found in PATH — install Docker (or set CONTAINER_ENGINE=podman) first."
 [ -f "$IMAGE_TAR" ]  || die "image tarball not found: $IMAGE_TAR"
 [ -f "$RUNNER_SRC" ] || die "runner script not found: $RUNNER_SRC"

packaging/linux/dist-bundle/uninstall.sh

+#!/usr/bin/env bash
+# CoderAI Docker distribution uninstaller — reverses install.sh. It:
+#   1. Removes the `coderai-docker` runner from both the root and user install
+#      dirs (/usr/local/bin and ~/.local/usr/bin).
+#   2. Removes the loaded image (unless --keep-image).
+#   3. Points out the PATH line install.sh may have added to ~/.bashrc.
+#
+# It does NOT touch your runtime data (coderai-runtime/, ~/.coderai) — those are
+# yours; delete them by hand if you want them gone.
+#
+# Usage:  ./uninstall.sh [--keep-image] [--image TAG] [--yes]
+# Env:    CONTAINER_ENGINE=docker|podman   (default docker)
+#         OCI_IMAGE=TAG                     image tag to remove (default coderai:dist)
+set -euo pipefail
+
+ENGINE="${CONTAINER_ENGINE:-docker}"
+IMAGE="${OCI_IMAGE:-coderai:dist}"
+KEEP_IMAGE=0
+ASSUME_YES=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --keep-image) KEEP_IMAGE=1; shift ;;
+    --image) [[ $# -ge 2 ]] || { echo "Error: --image requires a tag" >&2; exit 2; }; IMAGE="$2"; shift 2 ;;
+    -y|--yes) ASSUME_YES=1; shift ;;
+    -h|--help) sed -n '2,12p' "$0"; exit 0 ;;
+    *) echo "Error: unknown option: $1" >&2; exit 2 ;;
+  esac
+done
+
+say(){ printf '%s\n' "$*"; }
+die(){ printf 'Error: %s\n' "$*" >&2; exit 1; }
+
+# Tell the user what this will do, and confirm before proceeding.
+say "This will remove the 'coderai-docker' runner and$([ "$KEEP_IMAGE" -eq 1 ] && echo ' keep' || echo ' (after a prompt) remove') the CoderAI image"
+say "'$IMAGE' from $ENGINE. Your runtime data and config are not touched."
+if [ "$ASSUME_YES" -ne 1 ]; then
+  printf 'Proceed? [y/N] '
+  read -r reply </dev/tty || reply=""
+  case "$reply" in y|Y|yes|YES) ;; *) die "aborted by user." ;; esac
+fi
+
+# 1. Remove the runner from every place install.sh may have put it.
+removed_any=0
+for d in /usr/local/bin "$HOME/.local/usr/bin"; do
+  f="$d/coderai-docker"
+  [ -e "$f" ] || continue
+  if [ -w "$d" ]; then
+    rm -f "$f" && { say "[uninstall] removed runner: $f"; removed_any=1; }
+  elif command -v sudo >/dev/null 2>&1; then
+    sudo rm -f "$f" && { say "[uninstall] removed runner (sudo): $f"; removed_any=1; }
+  else
+    say "[uninstall] cannot remove $f (no write permission and no sudo) — remove it manually."
+  fi
+done
+[ "$removed_any" -eq 1 ] || say "[uninstall] no coderai-docker runner found in the standard locations."
+
+# 2. Remove the image unless asked to keep it.
+if [ "$KEEP_IMAGE" -eq 0 ]; then
+  DK=("$ENGINE")
+  if ! "$ENGINE" info >/dev/null 2>&1; then
+    if command -v sudo >/dev/null 2>&1; then DK=(sudo "$ENGINE"); fi
+  fi
+  if "${DK[@]}" image inspect "$IMAGE" >/dev/null 2>&1; then
+    if [ "$ASSUME_YES" -ne 1 ]; then
+      printf '[uninstall] remove image "%s"? [y/N] ' "$IMAGE"
+      read -r reply </dev/tty || reply=""
+      case "$reply" in y|Y|yes|YES) ;; *) say "[uninstall] keeping image $IMAGE."; KEEP_IMAGE=1 ;; esac
+    fi
+    if [ "$KEEP_IMAGE" -eq 0 ]; then
+      "${DK[@]}" image rm "$IMAGE" && say "[uninstall] removed image: $IMAGE"
+    fi
+  else
+    say "[uninstall] image '$IMAGE' not present (use --image TAG if you tagged it differently)."
+  fi
+else
+  say "[uninstall] keeping image (--keep-image)."
+fi
+
+# 3. Note the PATH line install.sh may have appended (we don't edit ~/.bashrc).
+RC="${HOME}/.bashrc"
+if [ -f "$RC" ] && grep -Fqs "Added by the CoderAI Docker installer" "$RC"; then
+  say ""
+  say "[uninstall] $RC still has the PATH line install.sh added. Remove it if you like:"
+  say "    # Added by the CoderAI Docker installer"
+  say "    export PATH=\"\$HOME/.local/usr/bin:\$PATH\""
+fi
+
+say ""
+say "Done. Runtime data (coderai-runtime/, ~/.coderai) was left untouched."

packaging/linux/make_dist_bundle.sh

@@ -4,6 +4,7 @@
 #
 #   <NAME>/
 #     install.sh            loads the image + installs the runner (see dist-bundle/)
+#     uninstall.sh          removes the runner + (optionally) the image
 #     coderai-docker        the run wrapper (run_oci.sh, image tag pinned)
 #     coderai-dist.tar.gz   the image (gzip-compressed `docker save`)
 #     README.txt / README.md
@@ -47,7 +48,8 @@ mkdir -p "$STAGE"
 ln "$IMAGE_TAR" "$STAGE/coderai-dist.tar.gz" 2>/dev/null \
   || cp "$IMAGE_TAR" "$STAGE/coderai-dist.tar.gz"
 install -m 0755 "$HERE/run_oci.sh"            "$STAGE/coderai-docker"
-install -m 0755 "$HERE/dist-bundle/install.sh" "$STAGE/install.sh"
+install -m 0755 "$HERE/dist-bundle/install.sh"   "$STAGE/install.sh"
+install -m 0755 "$HERE/dist-bundle/uninstall.sh" "$STAGE/uninstall.sh"
 cp "$HERE/dist-bundle/README.txt"             "$STAGE/README.txt"
 cp "$HERE/dist-bundle/README.md"              "$STAGE/README.md"
 

packaging/linux/run_oci.sh

@@ -10,7 +10,14 @@ fi
 
 ENGINE="${CONTAINER_ENGINE:-docker}"
 IMAGE_TAG="${OCI_IMAGE:-coderai:dist}"
-MODE="cpu"
+# Selected GPU backends. ADDITIVE: --nvidia --vulkan enables BOTH, so the
+# container gets the NVIDIA driver libs (libcuda.so.1 — needed even by a
+# CUDA-built llama-cpp running under Vulkan) AND /dev/dri. CPU always works.
+declare -A MODES=()
+# Bind-mount the host's libcuda.so.1 into the container (for Vulkan/CPU runs of a
+# CUDA-built llama-cpp on a host that has the driver but where you don't want the
+# full --gpus all). "auto" = detect via ldconfig; or an explicit path.
+WITH_LIBCUDA=""
 PORT="${CODERAI_PORT:-8776}"
 DATA_ROOT="$PWD/coderai-runtime"
 DETACH=0
@@ -51,9 +58,18 @@ Usage:
 Options:
   --docker            Use docker (default).
   --podman            Use podman.
-  --cpu               CPU-only run mode (default).
-  --nvidia            NVIDIA CUDA mode; adds --gpus all for Docker.
-  --vulkan            Vulkan mode; adds --device /dev/dri.
+  --cpu               Enable the CPU backend (always available; default if none).
+  --nvidia            Enable NVIDIA CUDA; adds --gpus all for Docker (maps the
+                      driver incl. libcuda.so.1).
+  --vulkan            Enable Vulkan; adds --device /dev/dri and auto bind-mounts
+                      the host's libcuda.so.1 (the bundled llama-cpp is a CUDA
+                      build). --nvidia and --vulkan are ADDITIVE — pass both to
+                      enable both backends in one container.
+  --all               Enable all GPU backends (nvidia + vulkan).
+  --with-libcuda[=P]  Bind-mount libcuda.so.1 into the container so a CUDA-built
+                      llama-cpp loads under --vulkan/--cpu on a driver-equipped
+                      host. P is an explicit path; default auto-detects via
+                      ldconfig. (Implied automatically when --nvidia is set.)
   -p, --port PORT     Host port to expose (default: 8776).
   --data-dir PATH     Directory for config/models/cache (default: ./coderai-runtime).
   --name NAME         Container name (default: coderai).
@@ -92,9 +108,12 @@ while [[ $# -gt 0 ]]; do
   case "$1" in
     --docker) ENGINE=docker; shift ;;
     --podman) ENGINE=podman; shift ;;
-    --cpu) MODE=cpu; shift ;;
-    --nvidia|--cuda) MODE=nvidia; shift ;;
-    --vulkan) MODE=vulkan; shift ;;
+    --cpu) MODES[cpu]=1; shift ;;
+    --nvidia|--cuda) MODES[nvidia]=1; shift ;;
+    --vulkan) MODES[vulkan]=1; shift ;;
+    --all) MODES[nvidia]=1; MODES[vulkan]=1; shift ;;
+    --with-libcuda) WITH_LIBCUDA="auto"; shift ;;
+    --with-libcuda=*) WITH_LIBCUDA="${1#*=}"; shift ;;
     -p|--port)
       [[ $# -ge 2 ]] || { echo "Error: $1 requires a port" >&2; exit 2; }
       PORT="$2"; shift 2 ;;
@@ -145,19 +164,53 @@ if [[ "$DETACH" == "1" ]]; then
   args+=(-d)
 fi
 
-case "$MODE" in
-  nvidia)
-    if [[ "$ENGINE" == "docker" ]]; then
-      args+=(--gpus all)
-    else
-      args+=(--hooks-dir=/usr/share/containers/oci/hooks.d)
-    fi
-    ;;
-  vulkan)
-    args+=(--device /dev/dri)
-    ;;
-  cpu) ;;
-esac
+# Default to CPU-only when no GPU backend was requested.
+if [[ "${#MODES[@]}" -eq 0 ]]; then
+  MODES[cpu]=1
+fi
+
+if [[ -n "${MODES[nvidia]:-}" ]]; then
+  if [[ "$ENGINE" == "docker" ]]; then
+    args+=(--gpus all)
+  else
+    args+=(--hooks-dir=/usr/share/containers/oci/hooks.d)
+  fi
+fi
+if [[ -n "${MODES[vulkan]:-}" ]]; then
+  args+=(--device /dev/dri)
+  # The bundled llama-cpp is a CUDA build, so Vulkan GGUF still needs libcuda.so.1.
+  # Auto-map it from the host (unless --nvidia already maps the whole driver, or
+  # the user gave an explicit --with-libcuda path).
+  [[ -z "$WITH_LIBCUDA" ]] && WITH_LIBCUDA="auto"
+fi
+
+# libcuda.so.1: the bundled llama-cpp-python is a CUDA build, so it needs the
+# NVIDIA userspace driver lib even for Vulkan/CPU GGUF. --nvidia maps the whole
+# driver via --gpus all already; --vulkan auto-enables a libcuda bind-mount (set
+# just above); otherwise bind-mount just libcuda when asked via --with-libcuda,
+# so a CUDA llama-cpp at least loads. Misses degrade gracefully now: the server
+# starts and the Vulkan/GGUF backend is simply reported unavailable.
+LIBCUDA_NOTE="none"
+if [[ -n "${MODES[nvidia]:-}" ]]; then
+  LIBCUDA_NOTE="via --gpus all (driver mapped)"
+elif [[ -n "$WITH_LIBCUDA" ]]; then
+  libcuda_path=""
+  if [[ "$WITH_LIBCUDA" == "auto" ]]; then
+    libcuda_path="$(ldconfig -p 2>/dev/null | awk '/libcuda\.so\.1/ {print $NF; exit}')"
+    [[ -n "$libcuda_path" ]] || for c in /usr/lib/x86_64-linux-gnu/libcuda.so.1 /usr/lib/libcuda.so.1 /usr/lib64/libcuda.so.1; do
+      [[ -e "$c" ]] && { libcuda_path="$c"; break; }
+    done
+  else
+    libcuda_path="$WITH_LIBCUDA"
+  fi
+  if [[ -n "$libcuda_path" && -e "$libcuda_path" ]]; then
+    args+=(-v "$libcuda_path:/usr/lib/x86_64-linux-gnu/libcuda.so.1:ro")
+    LIBCUDA_NOTE="$libcuda_path → /usr/lib/x86_64-linux-gnu/libcuda.so.1"
+  else
+    echo "Warning: --with-libcuda requested but libcuda.so.1 not found${WITH_LIBCUDA:+ ($WITH_LIBCUDA)}; skipping" >&2
+    LIBCUDA_NOTE="requested but not found"
+  fi
+fi
 
 volume_suffix=""
 if [[ "$ENGINE" == "podman" ]]; then
@@ -242,7 +295,8 @@ cat <<EOF
 Starting CoderAI OCI container
   engine:  $ENGINE
   image:   $IMAGE_TAG
-  mode:    $MODE
+  mode:    $(echo "${!MODES[@]}" | tr ' ' '+' | tr 'A-Z' 'a-z')
+  libcuda: $LIBCUDA_NOTE
   url:     http://127.0.0.1:$PORT/admin
   data:    $DATA_ROOT
   config:  $CONFIG_NOTE