say why generation stopped, instead of always "stop" (v0.4.83)

Measured against the production pod: a reasoning model asked for 64 tokens
spent all 64 inside its thought, returned NO content, and the response said
finish_reason "stop".

    max_tokens=64    finish=stop  content=0 chars ''             reasoning=262
    max_tokens=1500  finish=stop  content=13 chars 'XMU-5635-FB' reasoning=448

Two layers were each sure the other knew better. The proxy hardcoded
finish_reason = "stop" and never read choices[0].finish_reason from the remote;
OpenAIFormatter then hardcoded it again, so even a caller that worked the
reason out had it overwritten on the way to the client. vLLM had been saying
"length" all along.

It matters because "length" is the one reason a client can act on. An answer
cut off at the limit is half a sentence, and told it stopped normally the client
has nothing to retry on and no sign anything is missing. On a reasoning model
the thought is generated FIRST and spends the same budget, so a tight limit
returns an EMPTY answer that claims to have finished of its own accord -- which
is how this surfaced, as a retrieval test scoring zero for reasons that had
nothing to do with what it was measuring.

The remote's word is now carried through the per-call sink beside reasoning,
tool_calls and logprobs (get_last_finish_reason, buffered and streamed), the
formatter accepts the caller's reason and keeps "stop" only as the default, and
a tool call still wins -- that describes the answer's shape, and a truncation
after the call was formed must not relabel it. Fixed at all five response
exits, not just the one it was found at: streamed chat (both terminal-chunk
paths), buffered chat, and streamed and buffered /v1/completions.

A local backend is never told a reason, so the budget decides: generation that
used every token it was allowed was cut off, not finished. Drawn only when a
limit was actually set and reached -- never from a token count alone, or every
long completion would be mislabelled.

Tests verified against the reverted files: assert None == 'length' for the
backends, TypeError: unexpected keyword 'finish_reason' for the formatter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.4.82"
+__version__ = "0.4.83"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/text.py

@@ -281,6 +281,39 @@ def log_request_exchange(request):
         print(f"[debug-requests] failed to log request: {e}", flush=True)
 
 
+def _finish_reason(manager, completion_tokens, max_tokens, default="stop"):
+    """Why generation ended: what the backend says, else what the budget shows.
+
+    Hardcoding "stop" here made every answer look complete. It is wrong in the
+    one case a client needs to detect: an answer cut off because it hit
+    max_tokens is half a sentence, and reported as "stop" it is indistinguishable
+    from a finished one -- nothing to retry on, no sign anything is missing. On a
+    reasoning model the thought is generated FIRST and spends the same budget, so
+    a tight limit returns an EMPTY answer that still claims to have stopped of
+    its own accord (observed: 64 tokens asked of a reasoning model, 262
+    characters of thought, no content, finish_reason "stop").
+
+    A proxy backend is TOLD the reason by the remote, which is authoritative and
+    is used as given. A local backend says nothing, so the budget answers
+    instead: generation that used every token it was allowed was cut off, not
+    finished. The inference is only drawn when a limit was actually set and
+    reached -- never guessed from a token count alone.
+    """
+    try:
+        reported = (manager.get_last_finish_reason()
+                    if hasattr(manager, "get_last_finish_reason") else None)
+    except Exception:
+        reported = None
+    if reported:
+        return str(reported)
+    try:
+        if max_tokens and completion_tokens and int(completion_tokens) >= int(max_tokens):
+            return "length"
+    except (TypeError, ValueError):
+        pass
+    return default
+
+
 def log_response_exchange(content, tool_calls=None, finish_reason=None,
                           streamed=False, stage="pre-format"):
     """Dump the assistant message coderai *extracted* (content + tool_calls) when
@@ -3382,11 +3415,12 @@ async def stream_chat_response(
                     "completion_tokens": completion_tokens,
                     "total_tokens": prompt_tokens + completion_tokens,
                 }
-                log_response_exchange(generated_text, finish_reason="stop",
+                _fr = _finish_reason(current_manager, completion_tokens, max_tokens)
+                log_response_exchange(generated_text, finish_reason=_fr,
                                       streamed=True, stage="pre-format extracted")
-                log_response_exchange(client_sent_content, finish_reason="stop",
+                log_response_exchange(client_sent_content, finish_reason=_fr,
                                       streamed=True, stage="post-format sent to client")
-                final_chunk = formatter.format_litellm_chunk("", is_final=True, usage=usage_details, context_size=context_size)
+                final_chunk = formatter.format_litellm_chunk("", is_final=True, usage=usage_details, context_size=context_size, finish_reason=_fr)
                 log_response_payload(final_chunk, streamed=True)
                 yield f"data: {json.dumps(final_chunk)}\n\n"
         else:
@@ -3417,6 +3451,7 @@ async def stream_chat_response(
             # accumulated them, so the final chunk can carry the whole list.
             _lp = (current_manager.get_last_logprobs()
                    if hasattr(current_manager, "get_last_logprobs") else None)
+            _fr = _finish_reason(current_manager, completion_tokens, max_tokens)
             final_chunk = {
                 "id": completion_id,
                 "object": "chat.completion.chunk",
@@ -3424,9 +3459,9 @@ async def stream_chat_response(
                 "model": model_name,
                 "choices": [{
                     "index": 0,
-                    "finish_reason": "stop",
+                    "finish_reason": _fr,
                     "logprobs": _lp,
-                    "native_finish_reason": "stop",
+                    "native_finish_reason": _fr,
                 }],
                 "usage": {
                     "prompt_tokens": prompt_tokens,
@@ -3448,9 +3483,9 @@ async def stream_chat_response(
                 },
                 "system_fingerprint": None,
             }
-            log_response_exchange(generated_text, finish_reason="stop",
+            log_response_exchange(generated_text, finish_reason=_fr,
                                   streamed=True, stage="pre-format extracted")
-            log_response_exchange(client_sent_content, finish_reason="stop",
+            log_response_exchange(client_sent_content, finish_reason=_fr,
                                   streamed=True, stage="post-format sent to client")
             log_response_payload(final_chunk, streamed=True)
             yield f"data: {json.dumps(final_chunk)}\n\n"
@@ -3591,6 +3626,8 @@ async def generate_chat_response(
             response_message["reasoning"] = reasoning_text
             response_message["reasoning_content"] = reasoning_text
 
+        # Provisional: the real reason needs the token counts, which are read
+        # below. A tool call overrides it either way.
         finish_reason = "stop"
 
         # Check for tool calls
@@ -3658,6 +3695,9 @@ async def generate_chat_response(
         completion_tokens = last_usage.get('completion_tokens') or (
             len(generated_text.split()) if generated_text else 0)
         cached_tokens = last_usage.get('cached_tokens', 0)
+        if finish_reason != "tool_calls":
+            finish_reason = _finish_reason(current_manager, completion_tokens,
+                                           max_tokens)
 
         # Store in prompt cache manager for future prefix matching
         prompt_cache_manager.store(messages, _model_key_for_cache,
@@ -3682,6 +3722,10 @@ async def generate_chat_response(
             # it was separated and then thrown away, so a client saw neither
             # the thinking nor any sign that there had been any.
             reasoning=response_message.get("reasoning_content"),
+            # Without this the formatter assumed "stop" and the reason worked
+            # out above -- "length" from a remote that cut the answer off at
+            # the token limit -- was computed and then discarded.
+            finish_reason=finish_reason,
         )
         # Patch in the real cached_tokens value
         if formatted_response and 'usage' in formatted_response:
@@ -3904,7 +3948,8 @@ async def stream_completion_response(
                 "text": "",
                 "index": 0,
                 "logprobs": None,
-                "finish_reason": "stop",
+                "finish_reason": _finish_reason(current_manager, completion_tokens,
+                                                max_tokens),
             }],
             "usage": {
                 "prompt_tokens": prompt_tokens,
@@ -3965,7 +4010,8 @@ async def generate_completion_response(
                 "text": generated_text,
                 "index": 0,
                 "logprobs": None,
-                "finish_reason": "stop",
+                "finish_reason": _finish_reason(current_manager, completion_tokens,
+                                                max_tokens),
             }],
             "usage": {
                 "prompt_tokens": prompt_tokens,

codai/backends/base.py

@@ -139,6 +139,22 @@ class ModelBackend(ABC):
         """
         return None
 
+    def get_last_finish_reason(self):
+        """Why the remote stopped generating, or None if it did not say.
+
+        ``generate_chat`` returns text, and text cannot say whether it ENDED or
+        merely ran out of room. A local backend is asked to generate and the
+        caller knows the budget it set, but a proxy backend is told the answer
+        by the remote in ``choices[0].finish_reason`` -- and dropping it is not
+        a cosmetic loss. "length" means the answer is cut off mid-sentence;
+        reported as "stop" it is indistinguishable from a complete one, so a
+        client has nothing to retry on and no way to know it is reading half a
+        sentence. On a reasoning model it is worse: the thought is generated
+        first and spends the budget, so a tight limit yields an EMPTY answer
+        that still claims to have stopped naturally.
+        """
+        return None
+
     def get_last_tool_calls(self):
         """The tool calls the backend itself reported for the last call, or None.
 

codai/backends/runpod.py

@@ -118,6 +118,7 @@ class RunpodBackend(ModelBackend):
         self._last_logprobs: Optional[Dict] = None
         self._last_tool_calls: Optional[List[Dict]] = None
         self._last_reasoning: Optional[str] = None
+        self._last_finish_reason: Optional[str] = None
 
     # ------------------------------------------------------------------ #
     # lifecycle
@@ -217,6 +218,10 @@ class RunpodBackend(ModelBackend):
         """
         return sink_get("tool_calls", self._last_tool_calls)
 
+    def get_last_finish_reason(self):
+        """Why the REMOTE stopped generating. See ModelBackend for why it matters."""
+        return sink_get("finish_reason", self._last_finish_reason)
+
     def get_last_reasoning(self):
         """The thought the POD separated out of the last answer, or None.
 
@@ -496,11 +501,13 @@ class RunpodBackend(ModelBackend):
         self._last_logprobs = None
         self._last_tool_calls = None
         self._last_reasoning = None
+        self._last_finish_reason = None
         # The per-call store is cleared with them. A store is opened per request,
         # but a caller that reuses one context for several calls (a pipeline, a
         # test) would otherwise read the PREVIOUS call's tool call -- the exact
         # staleness the attribute resets exist to prevent.
-        for _k in ("usage", "logprobs", "tool_calls", "reasoning"):
+        for _k in ("usage", "logprobs", "tool_calls", "reasoning",
+                  "finish_reason"):
             sink_put(_k, None)
         try:
             for attempt in (1, 2):
@@ -556,7 +563,9 @@ class RunpodBackend(ModelBackend):
                     # can reset between the answer and the caller's read.
                     sink_put("logprobs", self._last_logprobs)
                     sink_put("tool_calls", self._last_tool_calls)
+                    self._last_finish_reason = choice.get("finish_reason") or None
                     sink_put("reasoning", self._last_reasoning)
+                    sink_put("finish_reason", self._last_finish_reason)
                     return message.get("content") or ""
                 finally:
                     release()
@@ -573,11 +582,13 @@ class RunpodBackend(ModelBackend):
         self._last_logprobs = None
         self._last_tool_calls = None
         self._last_reasoning = None
+        self._last_finish_reason = None
         # The per-call store is cleared with them. A store is opened per request,
         # but a caller that reuses one context for several calls (a pipeline, a
         # test) would otherwise read the PREVIOUS call's tool call -- the exact
         # staleness the attribute resets exist to prevent.
-        for _k in ("usage", "logprobs", "tool_calls", "reasoning"):
+        for _k in ("usage", "logprobs", "tool_calls", "reasoning",
+                  "finish_reason"):
             sink_put(_k, None)
         try:
             for attempt in (1, 2):
@@ -656,7 +667,8 @@ class RunpodBackend(ModelBackend):
         # runs on a thread and a thread inherits no context variables. Keeping
         # them per stream also stops two concurrent streams on this one backend
         # object from appending their thoughts to each other.
-        _acc = {"reasoning": None, "tool_calls": None, "logprobs": None}
+        _acc = {"reasoning": None, "tool_calls": None, "logprobs": None,
+                "finish_reason": None}
         _sink = call_sink()
 
         def _publish(key, value):
@@ -730,7 +742,10 @@ class RunpodBackend(ModelBackend):
                 _merged = {**_cur, "content": list(_cur.get("content") or [])
                            + list(_lp["content"])}
                 _publish("logprobs", _merged)
-            return bool(choice.get("finish_reason"))
+            _fr = choice.get("finish_reason")
+            if _fr:
+                _publish("finish_reason", _fr)
+            return bool(_fr)
 
         def _worker():
             import requests

codai/backends/vllm.py

@@ -62,6 +62,7 @@ class VllmBackend(ModelBackend):
         self._last_logprobs: Optional[Dict] = None
         self._last_tool_calls: Optional[List[Dict]] = None
         self._last_reasoning: Optional[str] = None
+        self._last_finish_reason: Optional[str] = None
 
     # ------------------------------------------------------------------ #
     # lifecycle
@@ -197,6 +198,10 @@ class VllmBackend(ModelBackend):
         """
         return sink_get("tool_calls", self._last_tool_calls)
 
+    def get_last_finish_reason(self):
+        """Why the REMOTE stopped generating. See ModelBackend for why it matters."""
+        return sink_get("finish_reason", self._last_finish_reason)
+
     def get_last_reasoning(self):
         """The thought the SERVER separated out of the last answer, or None.
 
@@ -412,11 +417,13 @@ class VllmBackend(ModelBackend):
         self._last_logprobs = None
         self._last_tool_calls = None
         self._last_reasoning = None
+        self._last_finish_reason = None
         # The per-call store is cleared with them. A store is opened per request,
         # but a caller that reuses one context for several calls (a pipeline, a
         # test) would otherwise read the PREVIOUS call's tool call -- the exact
         # staleness the attribute resets exist to prevent.
-        for _k in ("usage", "logprobs", "tool_calls", "reasoning"):
+        for _k in ("usage", "logprobs", "tool_calls", "reasoning",
+                  "finish_reason"):
             sink_put(_k, None)
         try:
             payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, False,
@@ -446,7 +453,9 @@ class VllmBackend(ModelBackend):
             # between the answer and the caller's read.
             sink_put("logprobs", self._last_logprobs)
             sink_put("tool_calls", self._last_tool_calls)
+            self._last_finish_reason = choice.get("finish_reason") or None
             sink_put("reasoning", self._last_reasoning)
+            sink_put("finish_reason", self._last_finish_reason)
             return message.get("content") or ""
         finally:
             self._exit_request()
@@ -460,11 +469,13 @@ class VllmBackend(ModelBackend):
         self._last_logprobs = None
         self._last_tool_calls = None
         self._last_reasoning = None
+        self._last_finish_reason = None
         # The per-call store is cleared with them. A store is opened per request,
         # but a caller that reuses one context for several calls (a pipeline, a
         # test) would otherwise read the PREVIOUS call's tool call -- the exact
         # staleness the attribute resets exist to prevent.
-        for _k in ("usage", "logprobs", "tool_calls", "reasoning"):
+        for _k in ("usage", "logprobs", "tool_calls", "reasoning",
+                  "finish_reason"):
             sink_put(_k, None)
         try:
             payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, True,
@@ -514,7 +525,8 @@ class VllmBackend(ModelBackend):
         # runs on a thread and a thread inherits no context variables. Keeping
         # them per stream also stops two concurrent streams on this one backend
         # object from appending their thoughts to each other.
-        _acc = {"reasoning": None, "tool_calls": None, "logprobs": None}
+        _acc = {"reasoning": None, "tool_calls": None, "logprobs": None,
+                "finish_reason": None}
         _sink = call_sink()
 
         def _publish(key, value):
@@ -587,7 +599,10 @@ class VllmBackend(ModelBackend):
                 _merged = {**_cur, "content": list(_cur.get("content") or [])
                            + list(_lp["content"])}
                 _publish("logprobs", _merged)
-            return bool(choice.get("finish_reason"))
+            _fr = choice.get("finish_reason")
+            if _fr:
+                _publish("finish_reason", _fr)
+            return bool(_fr)
 
         def _worker():
             import requests

codai/models/parser.py

@@ -1966,8 +1966,17 @@ class OpenAIFormatter:
         self.model_name = model_name
         self.id = f"chatcmpl-{uuid.uuid4()}"
 
-    def format_full(self, text, prompt_tokens, completion_tokens, tool_calls=None, reasoning=None, context_size=None):
-        """Standard Response (Non-Streaming)"""
+    def format_full(self, text, prompt_tokens, completion_tokens, tool_calls=None,
+                    reasoning=None, context_size=None, finish_reason=None):
+        """Standard Response (Non-Streaming)
+
+        ``finish_reason`` is the caller's: it knows whether the backend reported
+        one ("length" from a remote that cut the answer off at the token limit)
+        and this formatter does not. Omitted, it keeps the old behaviour, which
+        is the only safe default for the callers that pass nothing -- but a
+        caller that DOES know and is overruled here had its answer labelled
+        complete when it was truncated, which is the whole bug.
+        """
         if LITELLM_AVAILABLE and all([ModelResponse, Choices, Message, Usage]):
             try:
                 usage_dict = {
@@ -1981,7 +1990,8 @@ class OpenAIFormatter:
                     object="chat.completion",
                     created=int(time.time()),
                     choices=[Choices(
-                        finish_reason="tool_calls" if tool_calls else "stop",
+                        finish_reason=("tool_calls" if tool_calls
+                                       else (finish_reason or "stop")),
                         index=0,
                         message=Message(content=text if not tool_calls else None, role="assistant", tool_calls=tool_calls)
                     )],
@@ -2017,7 +2027,8 @@ class OpenAIFormatter:
         choice = {
             "index": 0,
             "message": message,
-            "finish_reason": "tool_calls" if tool_calls else "stop",
+            "finish_reason": ("tool_calls" if tool_calls
+                              else (finish_reason or "stop")),
         }
         
         usage = {
@@ -2041,8 +2052,13 @@ class OpenAIFormatter:
             },
         }
 
-    def format_chunk(self, delta_text, is_final=False, usage=None, context_size=None):
-        """Streaming Chunk (Used in a Generator)"""
+    def format_chunk(self, delta_text, is_final=False, usage=None, context_size=None,
+                     finish_reason=None):
+        """Streaming Chunk (Used in a Generator)
+
+        Only the final chunk carries a reason; see format_full for why the
+        caller's value wins over the "stop" assumed here.
+        """
         if LITELLM_AVAILABLE and all([ChatCompletionChunk, StreamingChoices, Delta, (Usage if usage else True)]):
             try:
                 return ChatCompletionChunk(
@@ -2051,7 +2067,7 @@ class OpenAIFormatter:
                     object="chat.completion.chunk",
                     created=int(time.time()),
                     choices=[StreamingChoices(
-                        finish_reason="stop" if is_final else None,
+                        finish_reason=((finish_reason or "stop") if is_final else None),
                         index=0,
                         delta=Delta(content=delta_text, role="assistant")
                     )],
@@ -2069,7 +2085,7 @@ class OpenAIFormatter:
         choice = {
             "index": 0,
             "delta": delta,
-            "finish_reason": "stop" if is_final else None,
+            "finish_reason": (finish_reason or "stop") if is_final else None,
         }
         
         chunk = {
@@ -2087,12 +2103,15 @@ class OpenAIFormatter:
             
         return chunk
 
-    def format_final_chunk(self, usage: dict = None, context_size: int = None) -> dict:
+    def format_final_chunk(self, usage: dict = None, context_size: int = None,
+                           finish_reason: str = None) -> dict:
         """Format the final streaming chunk with usage information."""
-        return self.format_chunk("", is_final=True, usage=usage, context_size=context_size)
+        return self.format_chunk("", is_final=True, usage=usage,
+                                 context_size=context_size,
+                                 finish_reason=finish_reason)
 
     # Backward compatibility methods
-    def format_litellm_full(self, text: str, prompt_tokens: int, completion_tokens: int, tool_calls=None, context_size=None, reasoning=None) -> dict:
+    def format_litellm_full(self, text: str, prompt_tokens: int, completion_tokens: int, tool_calls=None, context_size=None, reasoning=None, finish_reason=None) -> dict:
         """Backward compatibility method - calls format_full.
 
         ``reasoning`` was missing here, so a thought the caller had carefully
@@ -2100,11 +2119,13 @@ class OpenAIFormatter:
         saw neither the thinking nor any sign that there had been any.
         """
         return self.format_full(text, prompt_tokens, completion_tokens, tool_calls,
-                                reasoning=reasoning, context_size=context_size)
+                                reasoning=reasoning, context_size=context_size,
+                                finish_reason=finish_reason)
 
-    def format_litellm_chunk(self, delta_text: str, is_final: bool = False, usage: dict = None, context_size: int = None) -> dict:
+    def format_litellm_chunk(self, delta_text: str, is_final: bool = False, usage: dict = None, context_size: int = None, finish_reason: str = None) -> dict:
         """Backward compatibility method - calls format_chunk."""
-        return self.format_chunk(delta_text, is_final, usage, context_size)
+        return self.format_chunk(delta_text, is_final, usage, context_size,
+                                 finish_reason=finish_reason)
 
 
 # =============================================================================

tests/test_finish_reason_is_the_real_one.py

+"""An answer cut off at the token limit must not report that it stopped.
+
+2026-10-23, measured against the production pod: a reasoning model asked for 64
+tokens spent all 64 inside its thought, returned NO content, and the response
+said `finish_reason: "stop"`.
+
+    max_tokens=64    finish=stop  content=0 chars ''  reasoning=262 chars
+    max_tokens=1500  finish=stop  content=13 chars 'XMU-5635-FB'  reasoning=448
+
+Two layers were each sure the other knew better. The proxy hardcoded
+`finish_reason = "stop"` and never read what the pod reported; the formatter
+then hardcoded it again, so even a caller that worked the reason out had it
+overwritten on the way to the client. vLLM had said "length" all along.
+
+Why it matters more than a label: "length" is the one reason a client can act
+on. An answer truncated mid-sentence is incomplete, and told it stopped
+normally the client has nothing to retry on and no sign anything is missing. On
+a reasoning model the thought is generated FIRST and spends the same budget, so
+a tight limit yields an empty answer that claims to have finished -- which is
+how this was found, as a needle test scoring 0% for reasons that had nothing to
+do with what it was measuring.
+"""
+import asyncio
+import json
+
+import pytest
+
+from codai.api.text import _finish_reason
+from codai.backends.base import ModelBackend, new_call_sink
+from codai.backends.runpod import RunpodBackend
+from codai.backends.vllm import VllmBackend
+from codai.models.parser import OpenAIFormatter
+
+
+# ----------------------------------------------------------- the pod's answer #
+
+def _answer(finish_reason, content="half a sentence that stops mid-", reasoning=None):
+    msg = {"role": "assistant", "content": content}
+    if reasoning:
+        msg["reasoning_content"] = reasoning
+    return {"choices": [{"index": 0, "message": msg,
+                         "finish_reason": finish_reason}],
+            "usage": {"prompt_tokens": 7855, "completion_tokens": 64}}
+
+
+class _Resp:
+    status_code = 200
+    ok = True
+    headers = {}
+    url = "https://pod:1/v1/chat/completions"
+
+    def __init__(self, payload):
+        self._payload = payload
+        self.text = json.dumps(payload)
+
+    def json(self):
+        return self._payload
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        return False
+
+
+def _sse(*objs):
+    body = ("\n".join("data: " + json.dumps(o) for o in objs)
+            + "\ndata: [DONE]\n").encode()
+
+    class _R(_Resp):
+        def __init__(self):
+            self.text = body.decode()
+
+        def iter_content(self, chunk_size=8192):
+            return [body]
+
+        def json(self):
+            return {}
+    return _R()
+
+
+def _runpod(monkeypatch, response):
+    b = RunpodBackend.__new__(RunpodBackend)
+    b._model_id = b._served_name = "qwen38"
+    b._ctx = 122880
+    b._last_usage = {}
+    b._last_logprobs = b._last_tool_calls = b._last_reasoning = None
+    b._last_finish_reason = None
+    b._url, b._headers, b._pool, b._mode = "https://pod:1/v1", {}, None, "serverless"
+    monkeypatch.setattr("codai.api.pod_http.post", lambda *a, **k: response)
+    return b
+
+
+def _vllm(monkeypatch, response):
+    b = VllmBackend.__new__(VllmBackend)
+    b._model_id = b._served_name = "qwen38"
+    b._ctx = 122880
+    b._last_usage = {}
+    b._last_logprobs = b._last_tool_calls = b._last_reasoning = None
+    b._last_finish_reason = None
+    b._url = "http://127.0.0.1:41721"
+    monkeypatch.setattr("requests.post", lambda *a, **k: response)
+    return b
+
+
+@pytest.mark.parametrize("make", [_runpod, _vllm])
+def test_the_remote_said_length_and_the_proxy_keeps_it(monkeypatch, make):
+    """THE BUG: the pod's own word for it was read and thrown away."""
+    b = make(monkeypatch, _Resp(_answer("length")))
+    new_call_sink()
+    b.generate_chat([{"role": "user", "content": "hi"}], max_tokens=64)
+    assert b.get_last_finish_reason() == "length"
+
+
+@pytest.mark.parametrize("make", [_runpod, _vllm])
+def test_a_normal_answer_still_says_stop(monkeypatch, make):
+    b = make(monkeypatch, _Resp(_answer("stop", content="done.")))
+    new_call_sink()
+    b.generate_chat([{"role": "user", "content": "hi"}], max_tokens=1500)
+    assert b.get_last_finish_reason() == "stop"
+
+
+@pytest.mark.parametrize("make", [_runpod, _vllm])
+def test_the_streamed_path_carries_it_too(monkeypatch, make):
+    """The streamed answer is the one production actually uses."""
+    b = make(monkeypatch, _sse(
+        {"choices": [{"index": 0, "delta": {"content": "half a "}}]},
+        {"choices": [{"index": 0, "delta": {"content": "sentence that"}},
+                     ]},
+        {"choices": [{"index": 0, "delta": {}, "finish_reason": "length"}]}))
+    new_call_sink()
+
+    async def _run():
+        return [c async for c in b.generate_chat_stream(
+            [{"role": "user", "content": "hi"}], max_tokens=64)]
+    asyncio.run(_run())
+    assert b.get_last_finish_reason() == "length"
+
+
+def test_a_backend_that_does_not_know_says_nothing():
+    """A local backend generates text and is not told a reason; None lets the
+    caller fall back to the budget rather than invent authority it lacks."""
+    assert ModelBackend.get_last_finish_reason(object()) is None
+
+
+# --------------------------------------------------------------- the decision #
+
+class _Mgr:
+    def __init__(self, reported=None, raises=False):
+        self._reported, self._raises = reported, raises
+
+    def get_last_finish_reason(self):
+        if self._raises:
+            raise RuntimeError("backend is gone")
+        return self._reported
+
+
+def test_what_the_backend_reports_wins():
+    assert _finish_reason(_Mgr("length"), 10, 1500) == "length"
+    assert _finish_reason(_Mgr("stop"), 1500, 1500) == "stop"
+
+
+def test_without_a_report_the_budget_decides():
+    """A local backend that used every token it was allowed was cut off."""
+    assert _finish_reason(_Mgr(), 64, 64) == "length"
+    assert _finish_reason(_Mgr(), 65, 64) == "length"
+    assert _finish_reason(_Mgr(), 63, 64) == "stop"
+
+
+def test_nothing_is_inferred_without_a_limit():
+    """A long answer is not a truncated one; with no limit set there is no
+    evidence either way, and guessing would mislabel every long completion."""
+    assert _finish_reason(_Mgr(), 100000, None) == "stop"
+    assert _finish_reason(_Mgr(), 0, 0) == "stop"
+
+
+def test_a_broken_backend_does_not_break_the_answer():
+    assert _finish_reason(_Mgr(raises=True), 10, 1500) == "stop"
+    assert _finish_reason(object(), 10, 1500) == "stop"
+
+
+# ---------------------------------------------------------------- the formatter #
+
+def test_the_formatter_no_longer_overwrites_the_reason():
+    """THE SECOND HALF OF THE BUG: the caller's reason was computed, passed in,
+    and replaced with "stop" here."""
+    out = OpenAIFormatter("qwen38").format_full(
+        "half a sentence that stops mid-", 7855, 64, finish_reason="length")
+    assert out["choices"][0]["finish_reason"] == "length"
+
+
+def test_the_formatter_still_defaults_to_stop():
+    out = OpenAIFormatter("qwen38").format_full("done.", 10, 3)
+    assert out["choices"][0]["finish_reason"] == "stop"
+
+
+def test_a_tool_call_is_still_a_tool_call():
+    """tool_calls describes the answer's SHAPE and must not be overruled by a
+    truncation that happened after the call was fully formed."""
+    calls = [{"id": "c1", "type": "function",
+              "function": {"name": "f", "arguments": "{}"}}]
+    out = OpenAIFormatter("qwen38").format_full(
+        "", 10, 3, tool_calls=calls, finish_reason="length")
+    assert out["choices"][0]["finish_reason"] == "tool_calls"
+
+
+def test_the_final_stream_chunk_carries_it():
+    out = OpenAIFormatter("qwen38").format_chunk(
+        "", is_final=True, usage={"prompt_tokens": 1, "completion_tokens": 64,
+                                  "total_tokens": 65}, finish_reason="length")
+    assert out["choices"][0]["finish_reason"] == "length"
+    mid = OpenAIFormatter("qwen38").format_chunk("x", is_final=False,
+                                                 finish_reason="length")
+    assert mid["choices"][0]["finish_reason"] is None, "only the last chunk says why"
+
+
+# ------------------------------------------------------- the production shape #
+
+@pytest.mark.parametrize("make", [_runpod, _vllm])
+def test_the_empty_thinking_answer_is_reported_as_truncated(monkeypatch, make):
+    """What was actually observed: all 64 tokens spent in the thought, no
+    content, and the client told the model had stopped on its own."""
+    b = make(monkeypatch, _Resp(_answer("length", content="",
+                                        reasoning="x" * 262)))
+    new_call_sink()
+    content = b.generate_chat([{"role": "user", "content": "find the code"}],
+                              max_tokens=64)
+    assert content == ""
+    assert b.get_last_finish_reason() == "length"
+    out = OpenAIFormatter("qwen38").format_full(
+        content, 7855, 64, reasoning=b.get_last_reasoning(),
+        finish_reason=b.get_last_finish_reason())
+    choice = out["choices"][0]
+    assert choice["finish_reason"] == "length", (
+        "an empty answer that spent its whole budget thinking must say why")
+    assert (choice["message"].get("reasoning_content") or "").startswith("x")