oci: expose the NVIDIA GPU to Vulkan (NVIDIA_DRIVER_CAPABILITIES) for docker --nvidia

The nvidia container runtime only injects the NVIDIA Vulkan ICD when the
'graphics' driver capability is requested. --nvidia ran with the runtime default
(compute,utility), so inside the container Vulkan could see only the AMD card
(RADV) and llvmpipe — never the NVIDIA GPU. A Vulkan-only GGUF embedder
(dinov2-embed, built with GGML_VULKAN) therefore could not run on the NVIDIA card
at all; it was pinned to the AMD RX 580 or forced to CPU.

Add `-e NVIDIA_DRIVER_CAPABILITIES=all` (honouring any caller-exported value) to
the docker --nvidia args so the NVIDIA Vulkan ICD is injected and Vulkan
enumerates the NVIDIA GPU. Enables running the GGUF dinov2 embedder on the 3090
via Vulkan under the nvidia-gguf engine (device pinned via GGML_VK_VISIBLE_DEVICES
/ VK_ICD_FILENAMES; the build hardcodes ggml_backend_vk_init(0)). Takes effect on
the next container (re)launch.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
parent 643fa155
......@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here.
__version__ = "0.1.63"
__version__ = "0.1.64"
# Configure the CUDA caching allocator BEFORE torch is imported anywhere.
# expandable_segments lets the allocator return freed pages to the driver even
......
......@@ -481,6 +481,13 @@ fi
if [[ -n "${MODES[nvidia]:-}" ]]; then
if [[ "$ENGINE" == "docker" ]]; then
args+=(--gpus all)
# Expose the NVIDIA GPU to Vulkan, not just CUDA. The nvidia container runtime
# only injects the NVIDIA Vulkan ICD when the 'graphics' capability is requested;
# the default (compute,utility) leaves Vulkan blind to the NVIDIA card, so a
# Vulkan GGUF (e.g. the dinov2-embed embedder) can only reach the AMD card. 'all'
# adds graphics/display so Vulkan enumerates the NVIDIA GPU too. Honour any value
# the caller already exported.
args+=(-e "NVIDIA_DRIVER_CAPABILITIES=${NVIDIA_DRIVER_CAPABILITIES:-all}")
# Some hosts set `no-cgroups = true` in /etc/nvidia-container-runtime/config.toml.
# Then --gpus all injects the device nodes + driver libs but does NOT add them to
# the container's device-cgroup allowlist, so the kernel blocks GPU access and
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment