-
Stefy Lanza (nextime / spora ) authored
New server.engine_request_min_interval_ms (engine name → ms, 0/unset = off). _rate_gate spaces inference dispatch STARTS to an engine by at least the interval, capping request rate and inserting idle time between GPU submissions — a stability lever for a marginal card (e.g. RX 580) that wedges under sustained back-to-back Vulkan compute. Wired into all three inference dispatch paths after the swap-gate; the request itself runs unthrottled, only the start cadence is gated. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
19241dc1
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| frontproxy | ||
| models | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| main.py | ||
| platform_paths.py | ||
| system_app.py |