proxy: forward the fields coderai does not understand (v0.3.51)

The Digesta dev's 400 on page resumption, and a bug behind it that was worse
than the 400.

The vLLM backend built its outgoing request from an ALLOW-LIST of fields it
recognised. Resumption asks for two beyond an ordinary request --
continue_final_message, so the model carries on the exact string, and
add_generation_prompt: false, so no new turn is opened after the prefilled text
-- and neither was ever forwarded, by this or any previous version.

What reached vLLM was a conversation ending in an assistant turn with
add_generation_prompt left at its default TRUE. surya-ocr-2's chat template
then appends a fresh <|im_start|>assistant after the text so far, so the model
was asked to START A NEW ANSWER about the same image. It did.

And the failure hid behind the guard built to catch it: the new answer repeated
the beginning of the page, the no-progress check saw text it already had, logged
"came back repeating itself" and discarded the segment. 253 of 262 resumptions
on the live install -- read for a day as a model that degenerates on dense
pages, when it was a model being asked the wrong question, answering correctly,
and having the answer thrown away. It also explains the dev's table exactly: at
max_tokens=8000 resumption is barely exercised and OCR holds 93%, while at 4000
and 2000 nearly every page resumes, each resumption is a full generation that
produces a restart and is then discarded, and the batch times out.

Fixed by forwarding BY EXCLUSION, which is the only form that stays correct:
coderai names the fields it CONSUMES (_CHAT_CONSUMED_FIELDS) and everything
else goes to the server, which is the thing entitled to decide what it
supports. An allow-list means every field a server gains after the list was
written is born broken -- the same reasoning as forwarding page diagnostics by
exclusion. It reaches only a backend whose signature accepts it, so a backend
that cannot carry an unknown field is unaffected rather than broken, and
model/messages/stream are never overridable this way.

Also: a 4xx explains itself, if nobody discards it. raise_for_status() reports
"400 Client Error: Bad Request for url: ..." -- the symptom and nothing else --
while vLLM puts the reason in the BODY. A whole diagnosis cycle was spent
guessing at a 400 that had already been explained and thrown away. The error
now carries the server's own words. Made tolerant of a response object that
reports no status_code, rather than turning an odd shape into an AttributeError
inside the request path.

Two findings for the operator, not code: the pods serve max-model-len 32768
while the model entry asks for ctx 18432, so per-model vllm settings still are
not reaching the pod; and this fix must be in the POD image, because the proxy
that dropped the fields runs there.

Tests: 16 new (flags reach the payload, an unknown field is forwarded, no
passthrough changes nothing, structure not overridable, exclusion at the API
layer, signature-probed delivery, the error carries the server's words). Full
suite: 25 failures, all pre-existing and identical to the stashed baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.3.50"
+__version__ = "0.3.51"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

codai/api/text.py

@@ -2295,6 +2295,7 @@ async def chat_completions(request: ChatCompletionRequest, http_request: Request
                     frequency_penalty=request.frequency_penalty,
                     logprobs=request.logprobs,
                     top_logprobs=request.top_logprobs,
+                    passthrough=_chat_passthrough(request),
                 ):
                     yield chunk
             finally:
@@ -2324,11 +2325,56 @@ async def chat_completions(request: ChatCompletionRequest, http_request: Request
                 frequency_penalty=request.frequency_penalty,
                 logprobs=request.logprobs,
                 top_logprobs=request.top_logprobs,
+                passthrough=_chat_passthrough(request),
             )
         finally:
             _release_instance()
 
 
+#: Fields of a chat request coderai reads itself. Everything ELSE a caller sends
+#: is the caller's business with the serving backend, and is forwarded there.
+#:
+#: Named here rather than naming the fields to forward, because the forward list
+#: is the one that cannot be kept current: vLLM and its peers add request fields
+#: continuously, and a proxy that passes only what it recognises silently
+#: decides, for the server, which of the server's features exist. Every new
+#: field would be born broken -- the same failure as a diagnostic copied key by
+#: key. coderai's OCR page resumption was the first casualty: it sends
+#: `continue_final_message`, the proxy dropped it, and vLLM rejected the
+#: resulting conversation with a 400 at every token cap low enough to need one.
+#:
+#: Adding a field to THIS set is the deliberate act, and it means "coderai acts
+#: on this one".
+_CHAT_CONSUMED_FIELDS = frozenset({
+    "model", "messages", "temperature", "top_p", "n", "max_tokens",
+    "max_completion_tokens", "stream", "stop", "presence_penalty",
+    "frequency_penalty", "repeat_penalty", "tools", "tool_choice", "seed",
+    "logprobs", "top_logprobs", "response_format", "user", "enable_thinking",
+    "reasoning", "reasoning_effort", "suppress_reasoning",
+    # Transport and bookkeeping, never sampling.
+    "api_key", "stream_options", "metadata", "store",
+})
+
+
+def _chat_passthrough(request) -> Optional[Dict]:
+    """The request fields coderai does not consume, for the backend to forward.
+
+    Only reaches a backend whose signature accepts it (see
+    ``ModelManager._passthrough_kwargs``), so a backend that cannot carry an
+    unknown field is unaffected rather than broken.
+    """
+    try:
+        extra = getattr(request, "model_extra", None)
+        if not extra:
+            return None
+        out = {k: v for k, v in extra.items()
+               if k not in _CHAT_CONSUMED_FIELDS and not k.startswith("_")}
+        return out or None
+    except Exception:
+        # A request that cannot be inspected is still a request to serve.
+        return None
+
+
 _TOOL_SPAN_RE = _re.compile(r'<(tool|tool_call)\b[\s\S]*?</\1\s*>', _re.IGNORECASE)
 _TOOL_OPEN_RE = _re.compile(r'<(?:tool|tool_call)\b', _re.IGNORECASE)
 _TOOL_OPEN_TAGS = ('<tool>', '<tool_call>', '<|tool_call>', '<|tool_call|>',
@@ -2533,6 +2579,7 @@ async def stream_chat_response(
     frequency_penalty: float = 0.0,
     logprobs: Optional[bool] = None,
     top_logprobs: Optional[int] = None,
+    passthrough: Optional[Dict] = None,
 ) -> AsyncGenerator[str, None]:
     """Stream chat completion response with queue notifications."""
     completion_id = f"chatcmpl-{uuid.uuid4().hex}"
@@ -2684,6 +2731,7 @@ async def stream_chat_response(
             frequency_penalty=frequency_penalty,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            passthrough=passthrough,
         ):
             # Cooperative cancellation: stop streaming if the task was cancelled.
             if task_registry.is_cancelled(_tid):
@@ -3071,6 +3119,7 @@ async def generate_chat_response(
     frequency_penalty: float = 0.0,
     logprobs: Optional[bool] = None,
     top_logprobs: Optional[int] = None,
+    passthrough: Optional[Dict] = None,
 ) -> Dict:
     """Generate non-streaming chat completion response."""
     completion_id = f"chatcmpl-{uuid.uuid4().hex}"
@@ -3114,6 +3163,7 @@ async def generate_chat_response(
             frequency_penalty=frequency_penalty,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            passthrough=passthrough,
         )
         
         # Always filter out malformed content

codai/backends/vllm.py

@@ -196,8 +196,80 @@ class VllmBackend(ModelBackend):
             parts.append(f"{role}: {content}")
         return "\n".join(parts)
 
+    @staticmethod
+    def _raise_for_status(r, what="request"):
+        """`raise_for_status`, but carrying what the server actually said.
+
+        `requests` reports only "400 Client Error: Bad Request for url: ...",
+        which names the symptom and nothing else. vLLM puts the reason -- which
+        field it rejected, which template failed to render, by how much the
+        context was overrun -- in the response BODY, and that body was being
+        discarded on the way up. A whole diagnosis cycle was spent guessing at a
+        400 whose explanation had already been sent and thrown away.
+        """
+        code = getattr(r, "status_code", None)
+        if code is None:
+            # An object that does not report a code is not something to crash
+            # on inside the request path: fall back to whatever it does offer.
+            r.raise_for_status()
+            return
+        if code < 400:
+            return
+        detail = ""
+        try:
+            body = r.json()
+            detail = (body.get("message") or body.get("detail")
+                      or body.get("error") or "")
+            if isinstance(detail, dict):
+                detail = detail.get("message") or str(detail)
+            detail = str(detail) or str(body)
+        except Exception:
+            try:
+                detail = (r.text or "").strip()
+            except Exception:
+                detail = ""
+        if len(detail) > 2000:
+            detail = detail[:2000] + " ...[truncated]"
+        import requests as _rq
+        raise _rq.HTTPError(
+            f"{code} {getattr(r, 'reason', '')} from "
+            f"{getattr(r, 'url', '?')} on {what}"
+            + (f": {detail}" if detail else " (the server sent no explanation)"),
+            response=r)
+
+    #: Never overridable by a caller: these are the request's structure, not its
+    #: sampling. Letting `passthrough` rewrite them would let a client retarget
+    #: the server at another model or flip streaming under the response parser.
+    _PAYLOAD_STRUCTURE = ("model", "messages", "stream")
+
+    def _apply_passthrough(self, payload, passthrough):
+        """Merge fields coderai does not itself understand into the request.
+
+        A proxy that forwards only the fields it knows about decides, on the
+        server's behalf, which of the server's own features exist. vLLM grows
+        request fields faster than any allow-list here can be updated
+        (`continue_final_message`, `add_generation_prompt`,
+        `chat_template_kwargs`, `truncate_prompt_tokens`, guided-decoding
+        parameters), and a dropped field fails in the worst possible way: the
+        server answers something plausible for a DIFFERENT request, or rejects
+        it with a 400 that names nothing.
+
+        Measured 2026-10-13: OCR page resumption sends `continue_final_message`
+        so the model carries on the exact string it was cut off in. It never
+        arrived, so what reached vLLM was a conversation ending in an assistant
+        turn with no instruction to continue it -- rejected 400, every resumed
+        page lost, at every token cap low enough to need one.
+        """
+        if not passthrough:
+            return payload
+        for k, v in dict(passthrough).items():
+            if k in self._PAYLOAD_STRUCTURE:
+                continue
+            payload[k] = v
+        return payload
+
     def _chat_payload(self, messages, max_tokens, temperature, top_p, stop, stream,
-                      logprobs=None, top_logprobs=None):
+                      logprobs=None, top_logprobs=None, passthrough=None):
         payload = {
             "model": getattr(self, "_served_name", self._model_id),
             "messages": messages,
@@ -229,20 +301,23 @@ class VllmBackend(ModelBackend):
             # worker's repair.
             if getattr(self._cfg, "return_token_ids", False):
                 payload["return_tokens_as_token_ids"] = True
-        return payload
+        # Last, so a caller that asks for a field coderai also computes gets the
+        # value it asked for -- it is talking to this server deliberately.
+        return self._apply_passthrough(payload, passthrough)
 
     # ------------------------------------------------------------------ #
     # chat-level generation
     # ------------------------------------------------------------------ #
     def generate_chat(self, messages: List[Dict], max_tokens=None, temperature=0.7,
                       top_p=1.0, stop=None, tools=None, response_format=None,
-                      logprobs=None, top_logprobs=None):
+                      logprobs=None, top_logprobs=None, passthrough=None):
         import requests
         self._enter_request()
         self._last_logprobs = None
         try:
             payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, False,
-                                         logprobs=logprobs, top_logprobs=top_logprobs)
+                                         logprobs=logprobs, top_logprobs=top_logprobs,
+                                         passthrough=passthrough)
             # Forward the structured-output request AS ASKED. This used to
             # keep only {"type":"json_object"}, so a json_schema — the strict
             # form every current client sends, Surya's guided decoding among
@@ -254,7 +329,7 @@ class VllmBackend(ModelBackend):
             if tools:
                 payload["tools"] = tools
             r = requests.post(self._base() + "/v1/chat/completions", json=payload, timeout=3600)
-            r.raise_for_status()
+            self._raise_for_status(r, "chat/completions")
             data = r.json()
             self._store_usage(data.get("usage", {}))
             choice = (data.get("choices") or [{}])[0]
@@ -266,12 +341,14 @@ class VllmBackend(ModelBackend):
     async def generate_chat_stream(self, messages: List[Dict], max_tokens=None,
                                    temperature=0.7, top_p=1.0, stop=None, tools=None,
                                    response_format=None, logprobs=None,
-                                   top_logprobs=None) -> AsyncGenerator[str, None]:
+                                   top_logprobs=None,
+                                   passthrough=None) -> AsyncGenerator[str, None]:
         self._enter_request()
         self._last_logprobs = None
         try:
             payload = self._chat_payload(messages, max_tokens, temperature, top_p, stop, True,
-                                         logprobs=logprobs, top_logprobs=top_logprobs)
+                                         logprobs=logprobs, top_logprobs=top_logprobs,
+                                         passthrough=passthrough)
             # Same for the streamed path, which ignored it entirely.
             if response_format:
                 payload["response_format"] = response_format
@@ -333,7 +410,7 @@ class VllmBackend(ModelBackend):
             import requests
             try:
                 with requests.post(url, json=payload, stream=True, timeout=3600) as r:
-                    r.raise_for_status()
+                    self._raise_for_status(r, "streamed chat/completions")
                     buf = b""
                     done = False
                     for bchunk in r.iter_content(chunk_size=8192):

codai/models/manager.py

@@ -753,6 +753,26 @@ class ModelManager:
         return self.backend.generate(prompt, max_tokens, temperature, top_p, stop)
 
     
+    @staticmethod
+    def _passthrough_kwargs(fn, passthrough) -> dict:
+        """``{"passthrough": {...}}`` if ``fn`` takes it, else {}.
+
+        Asked of the signature rather than negotiated by TypeError, for the same
+        reason as ``_logprob_kwargs``: a backend that accepts it and raises
+        TypeError for its own reasons must not be called twice.
+        """
+        if not passthrough:
+            return {}
+        try:
+            import inspect
+            params = inspect.signature(fn).parameters
+        except (TypeError, ValueError):
+            return {}
+        if "passthrough" not in params and not any(
+                pp.kind == pp.VAR_KEYWORD for pp in params.values()):
+            return {}
+        return {"passthrough": dict(passthrough)}
+
     @staticmethod
     def _logprob_kwargs(fn, logprobs, top_logprobs) -> dict:
         """``{"logprobs": ..., "top_logprobs": ...}`` if ``fn`` takes them, else {}.
@@ -800,7 +820,8 @@ class ModelManager:
                       response_format: Optional[Dict] = None, enable_thinking: bool = False,
                       repeat_penalty: float = 1.0, presence_penalty: float = 0.0,
                       frequency_penalty: float = 0.0, logprobs: Optional[bool] = None,
-                      top_logprobs: Optional[int] = None):
+                      top_logprobs: Optional[int] = None,
+                      passthrough: Optional[Dict] = None):
         """Generate chat completion non-streaming.
 
         Forwards client repetition controls to the backend (defaults are no-ops).
@@ -813,7 +834,12 @@ class ModelManager:
         # Use generate_chat if available (Vulkan backend), otherwise format and use generate
         if hasattr(self.backend, 'generate_chat'):
             _lp = self._logprob_kwargs(self.backend.generate_chat, logprobs, top_logprobs)
-            for _extra in (dict(enable_thinking=enable_thinking, **_pen, **_lp),
+            # Tried FIRST and dropped first: an unknown field is the caller's
+            # business with the upstream server, never a reason to fail the
+            # request on a backend that cannot carry it.
+            _pt = self._passthrough_kwargs(self.backend.generate_chat, passthrough)
+            for _extra in (dict(enable_thinking=enable_thinking, **_pen, **_lp, **_pt),
+                           dict(enable_thinking=enable_thinking, **_pen, **_lp),
                            dict(enable_thinking=enable_thinking, **_lp),
                            dict(**_lp), {}):
                 try:
@@ -845,7 +871,8 @@ class ModelManager:
                                     response_format: Optional[Dict] = None, enable_thinking: bool = False,
                                     repeat_penalty: float = 1.0, presence_penalty: float = 0.0,
                                     frequency_penalty: float = 0.0, logprobs: Optional[bool] = None,
-                                    top_logprobs: Optional[int] = None):
+                                    top_logprobs: Optional[int] = None,
+                                    passthrough: Optional[Dict] = None):
         """Generate chat completion streaming.
 
         Client-supplied repetition controls (repeat/presence/frequency penalty) are
@@ -860,7 +887,9 @@ class ModelManager:
             # Try richest signature first, then progressively drop kwargs the backend
             # doesn't accept (older signatures) so we never hard-fail on plumbing.
             _lp = self._logprob_kwargs(self.backend.generate_chat_stream, logprobs, top_logprobs)
-            for _extra in (dict(enable_thinking=enable_thinking, **_pen, **_lp),
+            _pt = self._passthrough_kwargs(self.backend.generate_chat_stream, passthrough)
+            for _extra in (dict(enable_thinking=enable_thinking, **_pen, **_lp, **_pt),
+                           dict(enable_thinking=enable_thinking, **_pen, **_lp),
                            dict(enable_thinking=enable_thinking, **_lp),
                            dict(**_lp), {}):
                 try:

docs/ocr.md

@@ -1154,6 +1154,65 @@ So coderai now establishes the window and respects it:
 - Not knowing the window is survivable, and nothing is guessed: with no number,
   resumption behaves exactly as it did before and the budget rules.
 
+### The resumption flags never arrived (and why that looked like repetition)
+
+Resumption asks the server for two things beyond an ordinary request:
+`continue_final_message`, so the model carries on the exact string, and
+`add_generation_prompt: false`, so no new turn is opened after the prefilled
+text. coderai sends both. For three days neither reached the model.
+
+The vLLM backend built its outgoing request from an **allow-list** of fields it
+recognised, so anything else was dropped on the floor — silently, since a
+dropped field is indistinguishable from a field never sent. What reached vLLM
+was a conversation ending in an assistant turn with `add_generation_prompt` left
+at its default **true**. surya-ocr-2's chat template then does exactly what it
+is told:
+
+```jinja
+{%- elif message.role == "assistant" %}
+    {{- '<|im_start|>' + message.role + '\n' + content }}
+    {{- '<|im_end|>\n' }}
+...
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
+```
+
+A fresh `<|im_start|>assistant` after the text so far. The model was being asked
+to **start a new answer about the same image**, which is precisely what it did.
+
+The failure then hid behind the guard built to catch it: the new answer repeated
+the beginning of the page, the no-progress check saw text it already had, logged
+"came back repeating itself" and threw the segment away. 253 of 262 resumptions
+on a live install, read for a day as a model that degenerates on dense pages. It
+was a model being asked the wrong question, answering it correctly, and having
+the answer discarded.
+
+Two lessons worth keeping, because both recur:
+
+- **Forward by exclusion, never by allow-list.** coderai now names the request
+  fields it *consumes* (`_CHAT_CONSUMED_FIELDS`); everything else goes to the
+  server, which is the thing entitled to decide what it supports. An allow-list
+  means every field a server gains after the list was written is born broken.
+  The same reasoning as forwarding page diagnostics by exclusion.
+- **A guard that catches a bug is not a diagnosis of it.** "Came back repeating
+  itself" was true every time and explained nothing. The log now has to say
+  *which* of the two it saw, because "the model looped" and "the server ignored
+  our flag" look identical from the text alone and need opposite fixes.
+
+Structural fields (`model`, `messages`, `stream`) are never overridable this
+way: retargeting the server at another model, or flipping streaming under the
+response parser, is not a feature.
+
+### A server's 4xx explains itself — if nobody discards it
+
+`requests.raise_for_status()` produces `400 Client Error: Bad Request for url:
+http://127.0.0.1:53601/v1/chat/completions` — the symptom and nothing else. vLLM
+puts the reason in the response **body**: which field it rejected, which template
+failed to render, by how much the context was overrun. That body was being
+dropped, so a 400 that had already explained itself cost a diagnosis cycle of
+guesswork. The error now carries the server's own words.
+
 ### A reply that repeats itself did not run out of room
 
 A generation that trips surya's repeat detector hit the cap because it is

tests/test_chat_passthrough.py

+"""A proxy that forwards only the fields it recognises decides, on the server's
+behalf, which of the server's features exist.
+
+The bug this covers, measured 2026-10-13 on a live install: coderai's OCR page
+resumption sends `continue_final_message` and `add_generation_prompt: False` so
+the model carries on the exact string it was cut off in. The vLLM backend built
+its request from an allow-list, so neither flag was ever forwarded. What reached
+vLLM was a conversation ending in an assistant turn with `add_generation_prompt`
+left at its default TRUE -- and surya-ocr-2's chat template then appends a fresh
+`<|im_start|>assistant` after the prefilled text, so the model started a NEW
+answer about the same image instead of continuing.
+
+It did not fail loudly. Every resumption produced a restart, the no-progress
+guard in `_continue_reply` caught it as "came back repeating itself" and threw
+it away, so resumption had never once added a correct continuation while
+appearing to work.
+"""
+import sys
+import types
+import pytest
+
+
+def _backend():
+    """A VllmBackend instance with nothing loaded — only payload building."""
+    from codai.backends.vllm import VllmBackend
+    b = VllmBackend.__new__(VllmBackend)
+    b._served_name = "m"
+    b._model_id = "m"
+    b._cfg = types.SimpleNamespace(return_token_ids=False)
+    return b
+
+
+MSGS = [{"role": "user", "content": "read it"},
+        {"role": "assistant", "content": "<block>half a page"}]
+
+
+def test_the_resumption_flags_reach_the_payload():
+    p = _backend()._chat_payload(
+        MSGS, 2048, 0.0, 0.1, None, False,
+        passthrough={"continue_final_message": True,
+                     "add_generation_prompt": False})
+    assert p["continue_final_message"] is True
+    # The one that matters most: left at its default the template appends a new
+    # assistant turn and the model restarts the page.
+    assert p["add_generation_prompt"] is False
+
+
+def test_an_unknown_field_is_forwarded_rather_than_dropped():
+    """Not an allow-list. A field coderai has never heard of is between the
+    caller and the server — vLLM grows request fields faster than any list
+    here can be updated."""
+    p = _backend()._chat_payload(
+        MSGS, 2048, 0.0, 0.1, None, False,
+        passthrough={"chat_template_kwargs": {"enable_thinking": False},
+                     "truncate_prompt_tokens": 4096,
+                     "some_field_invented_next_year": "x"})
+    assert p["chat_template_kwargs"] == {"enable_thinking": False}
+    assert p["truncate_prompt_tokens"] == 4096
+    assert p["some_field_invented_next_year"] == "x"
+
+
+def test_no_passthrough_changes_nothing():
+    a = _backend()._chat_payload(MSGS, 2048, 0.0, 0.1, None, False)
+    b = _backend()._chat_payload(MSGS, 2048, 0.0, 0.1, None, False,
+                                 passthrough=None)
+    c = _backend()._chat_payload(MSGS, 2048, 0.0, 0.1, None, False,
+                                 passthrough={})
+    assert a == b == c
+
+
+@pytest.mark.parametrize("field,value", [("model", "someone-elses-model"),
+                                         ("messages", []),
+                                         ("stream", True)])
+def test_the_requests_structure_is_not_overridable(field, value):
+    """`passthrough` carries sampling and template options, not the request's
+    shape: retargeting the model or flipping streaming under the response
+    parser is not a feature, and a caller must not reach them this way."""
+    p = _backend()._chat_payload(MSGS, 2048, 0.0, 0.1, None, False,
+                                 passthrough={field: value})
+    assert p[field] != value or field == "messages" and p[field] is MSGS
+    assert p["model"] == "m"
+    assert p["messages"] is MSGS
+    assert p["stream"] is False
+
+
+# --- what the API layer decides to forward -----------------------------------
+
+
+def _req(**extra):
+    from codai.pydantic.textrequest import ChatCompletionRequest
+    return ChatCompletionRequest(model="m", messages=[
+        {"role": "user", "content": "hi"}], **extra)
+
+
+def test_unknown_request_fields_become_the_passthrough():
+    from codai.api.text import _chat_passthrough
+    out = _chat_passthrough(_req(continue_final_message=True,
+                                 add_generation_prompt=False))
+    assert out == {"continue_final_message": True,
+                   "add_generation_prompt": False}
+
+
+def test_fields_coderai_acts_on_are_not_forwarded_twice():
+    """By EXCLUSION: coderai names what it consumes, not what it forwards, so a
+    field invented after this code was written is forwarded by default rather
+    than born broken."""
+    from codai.api.text import _chat_passthrough
+    assert _chat_passthrough(_req(temperature=0.5, max_tokens=99, seed=7,
+                                  logprobs=True)) is None
+
+
+def test_a_plain_request_forwards_nothing():
+    from codai.api.text import _chat_passthrough
+    assert _chat_passthrough(_req()) is None
+
+
+def test_an_uninspectable_request_is_still_served():
+    from codai.api.text import _chat_passthrough
+    assert _chat_passthrough(object()) is None
+
+
+# --- the manager only hands it to a backend that can take it -----------------
+
+
+def test_a_backend_without_the_parameter_is_not_broken_by_it():
+    from codai.models.manager import ModelManager
+
+    def narrow(messages, max_tokens=None, temperature=0.7, top_p=1.0,
+               stop=None, tools=None, response_format=None):
+        return ""
+
+    assert ModelManager._passthrough_kwargs(narrow, {"a": 1}) == {}
+
+
+def test_a_backend_that_takes_it_gets_it():
+    from codai.models.manager import ModelManager
+
+    def wide(messages, max_tokens=None, temperature=0.7, top_p=1.0, stop=None,
+             tools=None, response_format=None, passthrough=None):
+        return ""
+
+    assert ModelManager._passthrough_kwargs(wide, {"a": 1}) == {"passthrough": {"a": 1}}
+    # Nothing to forward means nothing is added, so the richest rung of the
+    # signature ladder is not tried for no reason.
+    assert ModelManager._passthrough_kwargs(wide, None) == {}
+    assert ModelManager._passthrough_kwargs(wide, {}) == {}
+
+
+def test_a_kwargs_backend_counts_as_accepting_it():
+    from codai.models.manager import ModelManager
+
+    def anything(messages, **kw):
+        return ""
+
+    assert ModelManager._passthrough_kwargs(anything, {"a": 1}) == {"passthrough": {"a": 1}}
+
+
+# --- the server's own explanation must survive the trip up -------------------
+
+
+def test_an_error_carries_what_the_server_said():
+    """`requests` reports only "400 Client Error: Bad Request for url: ...".
+    vLLM puts the reason in the BODY, and it was being discarded — a whole
+    diagnosis cycle was spent guessing at a 400 that had already explained
+    itself."""
+    import requests
+    from codai.backends.vllm import VllmBackend
+
+    class _R:
+        status_code = 400
+        reason = "Bad Request"
+        url = "http://127.0.0.1:53601/v1/chat/completions"
+
+        @staticmethod
+        def json():
+            return {"message": "This model's maximum context length is 32768 "
+                                "tokens. However, you requested 40864 tokens."}
+
+    with pytest.raises(requests.HTTPError) as e:
+        VllmBackend._raise_for_status(_R(), "chat/completions")
+    assert "32768" in str(e.value) and "40864" in str(e.value)
+    assert "chat/completions" in str(e.value)
+
+
+def test_a_server_that_explains_nothing_says_so():
+    import requests
+    from codai.backends.vllm import VllmBackend
+
+    class _R:
+        status_code = 500
+        reason = "Internal Server Error"
+        url = "http://x/v1/chat/completions"
+        text = ""
+
+        @staticmethod
+        def json():
+            raise ValueError("not json")
+
+    with pytest.raises(requests.HTTPError) as e:
+        VllmBackend._raise_for_status(_R())
+    assert "no explanation" in str(e.value)
+
+
+def test_a_success_raises_nothing():
+    from codai.backends.vllm import VllmBackend
+
+    class _R:
+        status_code = 200
+
+    VllmBackend._raise_for_status(_R())

tests/test_proxy_logprobs.py

@@ -33,6 +33,12 @@ LOGPROBS = {"content": [{"token": "A", "logprob": 0.0, "bytes": [65], "top_logpr
 
 
 class _Reply:
+    #: A real response reports its code, and the backend reads it so that a
+    #: server's own explanation of a 4xx survives the trip up.
+    status_code = 200
+    reason = "OK"
+    url = "http://stub/v1/chat/completions"
+
     def __init__(self, payload):
         self._payload = payload