-
Stefy Lanza (nextime / spora ) authored
Reworked engine_request_min_interval_ms from start-spacing to a proper post-completion gap: _rate_acquire holds a per-engine lock for the whole request, _rate_release frees it only `interval` ms AFTER completion (via loop.call_later, non-blocking) — so consecutive requests to the engine are ALWAYS separated by at least that idle GPU time regardless of request duration. Wired acquire/release into all 3 inference dispatch paths with release in every finally/early-return. _rate_acquire refreshes config on mtime change so a value saved in the web UI applies to the next request. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
086c6543
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| admin | ||
| api | ||
| backends | ||
| broker | ||
| frontproxy | ||
| models | ||
| openai | ||
| pydantic | ||
| queue | ||
| tasks | ||
| __init__.py | ||
| cli.py | ||
| config.py | ||
| main.py | ||
| platform_paths.py | ||
| system_app.py |