Even after the readiness retries, a brokered stream could still reach the client
empty: when the model falls back to CPU offload after a VRAM OOM, the load + first
prefill can take tens of seconds during which the engine sends no SSE bytes, so the
relay/client times out and shows an empty/errored reply.

Keep the stream alive end-to-end: await the engine response and read its stream via
a background task, pulling with a timeout. On any gap (long load wait, or slow
first token) emit a visible "the model is thinking..." chunk once - a reasoning_content
delta when thinking is enabled (enable_thinking/thinking, or a known reasoning model,
unless reasoning_effort==none), otherwise a plain content message - then cheap
SSE-comment pings to hold the connection without further polluting the reply. Fast
responses see no gap and thus no keepalive. Slowness is fine; an empty reply is not.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce