runpod: guarantee a host node on GPU-less / unknown-backend deployments

For a coderai instance that only orchestrates RunPod (no local models), the node
that hosts the proxy must advertise the runpod capability. The no-GPU path already
builds its engine with backend "auto" (which has it), but an explicit engine_specs
backend:"cpu" and the unknown-backend fallback did not — those would 503.

- _DEFAULT_CAPS gains "cpu": {transformers, gguf, k3, runpod}.
- Unknown-backend fallback now {transformers, gguf, runpod} (runpod needs no local
  hardware, so every node can host it).

Verified by simulating pick_engine for a runpod-pinned model on auto/cpu/nvidia
nodes — all route instead of 503.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
parent 256900b7
......@@ -52,6 +52,9 @@ _DEFAULT_CAPS = {
"vulkan": {"gguf", "whisper", "k3", "runpod"},
"opencl": {"gguf", "whisper", "k3", "runpod"},
"auto": {"transformers", "gguf", "whisper", "ds4", "colibri", "k3", "kt", "runpod"},
# An explicit CPU node (engine_specs backend:"cpu"): no GPU work, but it can
# still serve CPU llama.cpp/transformers and orchestrate RunPod.
"cpu": {"transformers", "gguf", "k3", "runpod"},
}
......@@ -103,7 +106,10 @@ class Engine:
if not self.name:
self.name = f"engine#{self.id}"
if not self.capabilities:
self.capabilities = set(_DEFAULT_CAPS.get(self.backend, {"transformers", "gguf"}))
# Unknown backend: assume the local basics plus runpod (which needs no
# local hardware), so a RunPod-only deployment always has a host node.
self.capabilities = set(_DEFAULT_CAPS.get(
self.backend, {"transformers", "gguf", "runpod"}))
def can_serve(self, required_cap: Optional[str]) -> bool:
# The system worker (cache/downloads) never serves inference, so it must
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment