models: two defects the first real test run exposed
Running the new test against the live instance found both immediately,
which is the point of it.
A FORCED REMOTE TEST COULD PASS LOCALLY. /v1/chat/completions is
deliberately outside the gateway (RemoteOpenAIBackend owns it), so the
x-coderai-placement header is ignored on that path: a text model with no
remote configuration was served HERE while the result reported
where=remote, ok=true. A test that passes in the wrong place is worse
than no test. Forcing remote is now verified before anything is
dispatched, and refuses with what is actually missing.
A 200 WITH NO OUTPUT WAS A PASS. The text probe allowed 8 tokens, which
a reasoning model spends thinking — Qwen3.5-9B answered with
completion_tokens=0, finish_reason=stop, and the test called it ok. The
budget is now 64, and an empty result is a failure carrying the numbers
that explain it, for text, embeddings, images and rerank alike.
Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
Loading
Please register or sign in to comment