-
Stefy Lanza (nextime / spora ) authored
The engine-side "Waiting for model reply..." content injection ran inside the GIL-blocked engine handler during model load, so it never streamed live -- it only got concatenated into the final reply and polluted the output. Removed it. Load-status now lives in the responsive front-proxy/broker layer: - SSE: broker_execute_stream yields a NON-content chunk (empty delta.content + x_queue_info status) during each not-ready wait (engine down / starting / model loading), so a watching client sees progress without reply pollution. - out-of-band: the existing broker `pending` keepalives are kept. Both are gated by a single flag, resolved by _load_status_enabled(model): - global models.load_status_updates (default True) - per-model "load_status_updates" override in models.json (unset = inherit) The same gate drives the SSE side and, via client.status_gate, the out-of-band `pending` keepalives. Unknown model defaults to on so the relay deadline stays protected. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
bb8ebcf0