coderai-llama came out at 11.6 GB — the CUDA wheel for five GPU
generations on top of the full core — over RunPod's size line. The only
thing the CUDA GGUF backend needed torch for was three questions about
the card (is there one, total memory, free memory); gpu_probe answers
them from torch when present and NVML otherwise. The image drops to the
light core plus NVIDIA's runtime, cuBLAS and NCCL wheels.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY