• Stefy Lanza (nextime / spora )'s avatar
    models: two defects the first real test run exposed · ef60591a
    Stefy Lanza (nextime / spora ) authored
    Running the new test against the live instance found both immediately,
    which is the point of it.
    
    A FORCED REMOTE TEST COULD PASS LOCALLY. /v1/chat/completions is
    deliberately outside the gateway (RemoteOpenAIBackend owns it), so the
    x-coderai-placement header is ignored on that path: a text model with no
    remote configuration was served HERE while the result reported
    where=remote, ok=true. A test that passes in the wrong place is worse
    than no test. Forcing remote is now verified before anything is
    dispatched, and refuses with what is actually missing.
    
    A 200 WITH NO OUTPUT WAS A PASS. The text probe allowed 8 tokens, which
    a reasoning model spends thinking — Qwen3.5-9B answered with
    completion_tokens=0, finish_reason=stop, and the test called it ok. The
    budget is now 64, and an empty result is a failure carrying the numbers
    that explain it, for text, embeddings, images and rerank alike.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    ef60591a
test_remote_execution.py 52.4 KB