• Stefy Lanza (nextime / spora )'s avatar
    embeddings: per-model throttle + default 200ms pacing · dc6026ef
    Stefy Lanza (nextime / spora ) authored
    The RX 580 (Polaris/Vulkan) keeps timing out its GPU ring under relentless
    back-to-back embedding load — a global throttle can't distinguish a fragile card
    from a robust one. Make the admission gate PER-MODEL: each model id gets its own
    semaphore + backlog counter + pacing, resolved from the models.json entry
    (embed_max_concurrency / embed_max_backlog / embed_min_interval_ms), falling back
    to the CODERAI_EMBED_* env vars, then defaults. Default min interval is now
    200ms (was 0) so GPU starts are paced out of the box, giving the ring breathing
    room; a robust CUDA model can set it to 0 per-model. Bump 0.1.82.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0196Gm4PNkcuybDj9yfcz3CR
    dc6026ef
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...