• Stefy Lanza (nextime / spora )'s avatar
    Streaming support: anthropic + ollama, and true incremental streaming for google · 42e06a0c
    Stefy Lanza (nextime / spora ) authored
    - anthropic: previously ignored the stream flag entirely (always non-stream).
      Add _handle_streaming_request converting Anthropic SSE events into
      OpenAI-compatible chunks (text deltas, tool_use/input_json deltas, usage).
    - ollama: previously hard-coded stream=False. Add incremental streaming over
      /api/generate yielding OpenAI chunks.
    - google: was buffering the entire response before yielding. Stream chunks
      incrementally as they arrive when no tools are requested (the common case);
      keep the buffer-and-parse path only when tools are present, since the
      text-encoded tool-call detection needs the complete response.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    42e06a0c
Name
Last commit
Last update
aisbf Loading commit data...
config Loading commit data...
docs Loading commit data...
static Loading commit data...
templates Loading commit data...
tests Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
API_EXAMPLES.md Loading commit data...
BUILD_DEPLOY.md Loading commit data...
CHANGELOG.md Loading commit data...
DEBUG_GUIDE.md Loading commit data...
DOCUMENTATION.md Loading commit data...
ENDPOINTS.md Loading commit data...
KILO_OAUTH2.md Loading commit data...
LICENSE.txt Loading commit data...
MANIFEST.in Loading commit data...
MISSING_ENDPOINTS.md Loading commit data...
PYPI.md Loading commit data...
QUICK_START_PAYMENT.md Loading commit data...
README.md Loading commit data...
RUNPOD_IMPLEMENTATION_PLAN.md Loading commit data...
SUBSCRIPTION_PRICING_FEATURE.md Loading commit data...
TODO.md Loading commit data...
TORRC_SETUP.md Loading commit data...
aisbf.sh Loading commit data...
build.sh Loading commit data...
chatgpt-api-implementation-guide.md Loading commit data...
clean.sh Loading commit data...
cli.py Loading commit data...
codex-oauth-implementation-guide.md Loading commit data...
main.py Loading commit data...
main.py.bak Loading commit data...
minio-archive-plan.md Loading commit data...
pyproject.toml Loading commit data...
requirements.txt Loading commit data...
screenshot.png Loading commit data...
setup.py Loading commit data...
start_proxy.sh Loading commit data...
verify_release.sh Loading commit data...