• Stefy Lanza (nextime / spora )'s avatar
    broker/text: move model-load status out of the engine; gate by load_status_updates · bb8ebcf0
    Stefy Lanza (nextime / spora ) authored
    The engine-side "Waiting for model reply..." content injection ran inside the
    GIL-blocked engine handler during model load, so it never streamed live -- it
    only got concatenated into the final reply and polluted the output. Removed it.
    
    Load-status now lives in the responsive front-proxy/broker layer:
    - SSE: broker_execute_stream yields a NON-content chunk (empty delta.content +
      x_queue_info status) during each not-ready wait (engine down / starting /
      model loading), so a watching client sees progress without reply pollution.
    - out-of-band: the existing broker `pending` keepalives are kept.
    
    Both are gated by a single flag, resolved by _load_status_enabled(model):
    - global models.load_status_updates (default True)
    - per-model "load_status_updates" override in models.json (unset = inherit)
    The same gate drives the SSE side and, via client.status_gate, the out-of-band
    `pending` keepalives. Unknown model defaults to on so the relay deadline stays
    protected.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    bb8ebcf0
Name
Last commit
Last update
..
__init__.py Loading commit data...
asgi_bridge.py Loading commit data...
capabilities.py Loading commit data...
client.py Loading commit data...
config.py Loading commit data...
dispatcher.py Loading commit data...
models.py Loading commit data...
service.py Loading commit data...
streaming.py Loading commit data...