• Stefy Lanza (nextime / spora )'s avatar
    broker/text: move model-load status out of the engine; gate by load_status_updates · bb8ebcf0
    Stefy Lanza (nextime / spora ) authored
    The engine-side "Waiting for model reply..." content injection ran inside the
    GIL-blocked engine handler during model load, so it never streamed live -- it
    only got concatenated into the final reply and polluted the output. Removed it.
    
    Load-status now lives in the responsive front-proxy/broker layer:
    - SSE: broker_execute_stream yields a NON-content chunk (empty delta.content +
      x_queue_info status) during each not-ready wait (engine down / starting /
      model loading), so a watching client sees progress without reply pollution.
    - out-of-band: the existing broker `pending` keepalives are kept.
    
    Both are gated by a single flag, resolved by _load_status_enabled(model):
    - global models.load_status_updates (default True)
    - per-model "load_status_updates" override in models.json (unset = inherit)
    The same gate drives the SSE side and, via client.status_gate, the out-of-band
    `pending` keepalives. Unknown model defaults to on so the relay deadline stays
    protected.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    bb8ebcf0
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...