• Stefy Lanza (nextime / spora )'s avatar
    coderai broker: actually stream responses (don't wait for done before yielding) · 3c7fc7c2
    Stefy Lanza (nextime / spora ) authored
    Streaming was scaffolded (stream_queue, _publish_stream_response,
    wait_for_stream_event, _iter_broker_stream_chunks) but send_request always awaited
    the terminal future and then popped the pending entry in finally — so by the time
    _broker_stream began consuming, the stream was over and its queue was gone. Result:
    the whole reply arrived at once.
    
    Fixes:
    - send_request: for streaming requests, return a {event:"stream_start"} immediately
      and do NOT pop _pending in finally — the consumer drives it. Pop on error.
    - _publish_stream_response: queue EVERY event (chunks AND the terminal done) so the
      consumer's loop receives the end marker; still resolves the future for safety.
    - finish_stream(): new — removes the streaming request's _pending once drained.
    - wait_for_stream_event: simplified to a queue.get (dropped the event_log path).
    - providers/coderai._iter_broker_stream_chunks: wrap in try/finally that calls
      finish_stream(request_id) on completion/error; treat stream_start as a no-data
      continue.
    
    Pairs with the coderai-side change that emits chunk/done envelopes.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    3c7fc7c2
Name
Last commit
Last update
aisbf Loading commit data...
config Loading commit data...
docs Loading commit data...
static Loading commit data...
templates Loading commit data...
tests Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
API_EXAMPLES.md Loading commit data...
BUILD_DEPLOY.md Loading commit data...
CHANGELOG.md Loading commit data...
DEBUG_GUIDE.md Loading commit data...
DOCUMENTATION.md Loading commit data...
ENDPOINTS.md Loading commit data...
KILO_OAUTH2.md Loading commit data...
LICENSE.txt Loading commit data...
MANIFEST.in Loading commit data...
MISSING_ENDPOINTS.md Loading commit data...
PYPI.md Loading commit data...
QUICK_START_PAYMENT.md Loading commit data...
README.md Loading commit data...
RUNPOD_IMPLEMENTATION_PLAN.md Loading commit data...
SUBSCRIPTION_PRICING_FEATURE.md Loading commit data...
TODO.md Loading commit data...
TORRC_SETUP.md Loading commit data...
aisbf.sh Loading commit data...
build.sh Loading commit data...
chatgpt-api-implementation-guide.md Loading commit data...
clean.sh Loading commit data...
cli.py Loading commit data...
codex-oauth-implementation-guide.md Loading commit data...
main.py Loading commit data...
main.py.bak Loading commit data...
minio-archive-plan.md Loading commit data...
pyproject.toml Loading commit data...
requirements.txt Loading commit data...
screenshot.png Loading commit data...
setup.py Loading commit data...
start_proxy.sh Loading commit data...
verify_release.sh Loading commit data...