• Stefy Lanza (nextime / spora )'s avatar
    router: pin video generation to a capable engine (never the gguf-only sibling) · cc6db025
    Stefy Lanza (nextime / spora ) authored
    Two bugs let a torch video pipeline run on the nvidia-gguf engine (caps={"gguf"})
    when the nvidia engine was briefly down mid-restart:
    
    1. _INFERENCE_PATHS listed "/v1/videos/generations" (plural) but the endpoint is
       "/v1/video/generations" (singular, video.py:3104). So video requests failed
       is_inference_path(), skipped all capability-aware routing, and fell through to
       registry.primary() — which returns the first HEALTHY engine when the primary
       (nvidia) is unhealthy, i.e. the gguf sibling. Fixed the path.
    
    2. pick_engine() step 5 fell back to a capability-BLIND least_loaded(None) pick,
       so a typed request could still land on an engine lacking the capability. Now a
       typed request (transformers/gguf/whisper) only ever picks a capable engine —
       preferring the primary when it can serve the cap (request queues / caller
       retries), else 503 — instead of mis-routing to an incompatible engine.
    
    Net: video generation (cap=transformers) routes to the nvidia engine only; if it
    is busy it queues there, if it is restarting the caller retries — it never runs a
    torch pipeline on the gguf-only engine. Bumps to 0.1.30.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    cc6db025
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...