• Stefy Lanza (nextime / spora )'s avatar
    router: pin video generation to a capable engine (never the gguf-only sibling) · cc6db025
    Stefy Lanza (nextime / spora ) authored
    Two bugs let a torch video pipeline run on the nvidia-gguf engine (caps={"gguf"})
    when the nvidia engine was briefly down mid-restart:
    
    1. _INFERENCE_PATHS listed "/v1/videos/generations" (plural) but the endpoint is
       "/v1/video/generations" (singular, video.py:3104). So video requests failed
       is_inference_path(), skipped all capability-aware routing, and fell through to
       registry.primary() — which returns the first HEALTHY engine when the primary
       (nvidia) is unhealthy, i.e. the gguf sibling. Fixed the path.
    
    2. pick_engine() step 5 fell back to a capability-BLIND least_loaded(None) pick,
       so a typed request could still land on an engine lacking the capability. Now a
       typed request (transformers/gguf/whisper) only ever picks a capable engine —
       preferring the primary when it can serve the cap (request queues / caller
       retries), else 503 — instead of mis-routing to an incompatible engine.
    
    Net: video generation (cap=transformers) routes to the nvidia engine only; if it
    is busy it queues there, if it is restarting the caller retries — it never runs a
    torch pipeline on the gguf-only engine. Bumps to 0.1.30.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    cc6db025
Name
Last commit
Last update
..
__init__.py Loading commit data...
admin_data.py Loading commit data...
app.py Loading commit data...
assignment.py Loading commit data...
engine_supervisor.py Loading commit data...
gpu_detect.py Loading commit data...
registry.py Loading commit data...
reqqueue.py Loading commit data...
router.py Loading commit data...
ui_pages.py Loading commit data...