-
Stefy Lanza (nextime / spora ) authored
#1 Front-side in-flight task tracking: Engine.enter_request now records per-request metadata (model/kind/path/started_at) in engine.active; _merge_engine_tasks injects synthetic "running" task entries for in-flight requests not already reported by the engine (deduped by engine+model). Both the direct proxy and the broker route register/clear it. So the Tasks page shows work the front dispatched even when the engine is too GIL-busy generating to answer its own /admin/api/tasks. (Combines with the last-good cache.) #3 Thermal stopping-criteria cadence: the per-token callback held the GIL every token, starving admin handlers. Now it does real work every 50 tokens for text, scaling to every 100 when generation is fast (>50 tok/s) — far less GIL contention, negligible thermal-drift risk between checks. #2 (no change needed): generation already runs OFF the event loop via _aiter_blocking (asyncio.to_thread per token step) for streaming and asyncio.to_thread(manager.generate) for non-streaming. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
afefee53