broker: log inbound max_tokens at the relay boundary (locate where it's dropped)

A remote client (Hermes) reportedly sends max_tokens=512, but requests arrive at
the engine with no max_tokens (defaulting to 2048). Our broker client preserves
whatever payload it's handed, so log the inbound max_tokens / max_completion_tokens
and the body keys when a brokered request arrives — this shows definitively whether
the value survives the aisbf relay into our payload or is dropped upstream
(remote), so we fix the right layer. Diagnostic only.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
parent 62dde8c2
......@@ -378,6 +378,35 @@ class BrokerClient:
message.get("client_id") or self.session_metadata.get("client_id"),
endpoint_path, method, stream,
)
# Diagnostic: surface the inbound max_tokens so we can tell whether a
# remote client's value (e.g. Hermes) survives the aisbf relay into our
# payload, or is dropped before it reaches us. Handles the chat body
# being the payload directly or nested under body / body_base64.
try:
_b = payload
if isinstance(payload, dict):
if isinstance(payload.get("body"), dict):
_b = payload["body"]
elif isinstance(payload.get("body"), str):
try:
_b = json.loads(payload["body"])
except Exception:
_b = {}
elif payload.get("body_base64"):
import base64 as _b64
_b = json.loads(_b64.b64decode(
payload["body_base64"]).decode("utf-8", "replace") or "{}")
_mt = _b.get("max_tokens") if isinstance(_b, dict) else None
_mct = _b.get("max_completion_tokens") if isinstance(_b, dict) else None
logger.info(
"CoderAI broker inbound request_id=%s max_tokens=%s "
"max_completion_tokens=%s body_keys=%s",
request_id, _mt, _mct,
sorted(_b.keys()) if isinstance(_b, dict) else type(_b).__name__,
)
except Exception as _probe_exc:
logger.info("CoderAI broker inbound max_tokens probe failed: %s",
_probe_exc)
# Send an immediate "pending" acknowledgment so the broker side extends
# its deadline, then keep sending periodic keepalives while the request
# runs (e.g. during model download). The keepalive task is cancelled
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment