• Stefy Lanza (nextime / spora )'s avatar
    a remote's tool call reaches the client (v0.4.60) · 417328c7
    Stefy Lanza (nextime / spora ) authored
    Every tool-using request to a pod-served model answered an EMPTY assistant
    message: content "", finish_reason "stop", no tool_calls. Asked directly the
    pod answered the call it had generated; through the orchestrator the same
    request answered nothing. Both the LoRA (diogene-alpha) and the base model
    were affected, so no adapter was involved.
    
    A proxy backend's generate_chat returns a STRING. The remote server owns the
    chat template, so it is the side that recognises a tool call and reports it
    structurally -- `{"content": null, "tool_calls": [...]}`. `message.content or
    ""` dropped that on the floor; the caller then had an empty string, found no
    call to parse in it, and answered the client with an empty message.
    
    The calls now travel beside the text, where usage and logprobs already do:
    get_last_tool_calls() on the backend, forwarded by the manager, preferred over
    re-parsing text in both the streamed and buffered chat paths. A backend that
    generates locally still keeps its call inside the text and answers None, so
    that path is untouched.
    
    Streaming needed the same: a streamed call arrives as deltas with no content
    at all, so merge_tool_call_deltas folds the fragments -- keyed by the call's
    index, never by arrival order, since concatenating the arguments of the wrong
    call yields valid JSON for a call nobody made.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    417328c7
Name
Last commit
Last update
..
__init__.py Loading commit data...
base.py Loading commit data...
colibri.py Loading commit data...
colibri_families.py Loading commit data...
cuda.py Loading commit data...
ds4.py Loading commit data...
ggml_rpc.py Loading commit data...
gpu_probe.py Loading commit data...
k3.py Loading commit data...
ktransformers.py Loading commit data...
overrides.py Loading commit data...
remote_openai.py Loading commit data...
runpod.py Loading commit data...
vllm.py Loading commit data...
vulkan.py Loading commit data...