-
Stefy Lanza (nextime / spora ) authored
Embeddings bypassed the front request queue (relayed unqueued) and the per-model lease scheduler, so the only guard was ratelimit.py's per-IP fixed window. A single authenticated client could pipeline unlimited embedding requests straight at the GPU — which on the Polaris/RX580 Vulkan backend wedged the SDMA ring (ring timeout -> GPU reset -> device lost from bus). Add _embed_admission() around create_embeddings: bounded concurrency, backlog shedding (429 when in-flight >= concurrency + backlog), and optional pacing between GPU starts. Tunable via CODERAI_EMBED_MAX_CONCURRENCY (2), CODERAI_EMBED_MAX_BACKLOG (32), CODERAI_EMBED_MIN_INTERVAL_MS (0). Bump 0.1.79. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0196Gm4PNkcuybDj9yfcz3CR
bbac56b1