Skip to content
Commit cc6db025 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

router: pin video generation to a capable engine (never the gguf-only sibling)



Two bugs let a torch video pipeline run on the nvidia-gguf engine (caps={"gguf"})
when the nvidia engine was briefly down mid-restart:

1. _INFERENCE_PATHS listed "/v1/videos/generations" (plural) but the endpoint is
   "/v1/video/generations" (singular, video.py:3104). So video requests failed
   is_inference_path(), skipped all capability-aware routing, and fell through to
   registry.primary() — which returns the first HEALTHY engine when the primary
   (nvidia) is unhealthy, i.e. the gguf sibling. Fixed the path.

2. pick_engine() step 5 fell back to a capability-BLIND least_loaded(None) pick,
   so a typed request could still land on an engine lacking the capability. Now a
   typed request (transformers/gguf/whisper) only ever picks a capable engine —
   preferring the primary when it can serve the cap (request queues / caller
   retries), else 503 — instead of mis-routing to an incompatible engine.

Net: video generation (cap=transformers) routes to the nvidia engine only; if it
is busy it queues there, if it is restarting the caller retries — it never runs a
torch pipeline on the gguf-only engine. Bumps to 0.1.30.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
parent ff39ee42
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment