• Stefy Lanza (nextime / spora )'s avatar
    test-run: stop three probes from failing models for the harness's own mistakes · ae0f967e
    Stefy Lanza (nextime / spora ) authored
    The first RunPod pass over every capability produced four failures, three of
    which were ours:
    
    The TTS probe sent voice "default", which no shipped model has — Kokoro loaded
    fine on a rented pod and then answered "Voice default not found in available
    voices". TTSRequest already defaults to a real voice, so the probe stops
    inventing one.
    
    OCR engines are chosen by name (paddle|doctr|surya) and are not models.json
    entries, so the test refused with "'surya' is not in the model list". _describe
    now recognises them.
    
    For a text model the refusal offered "a remote for its text capability" as a
    fix. It is not one: /v1/chat/completions is outside the gateway, so that config
    routes nothing and the test would then report a remote pass for a request served
    locally — the exact failure the forced-remote check exists to prevent.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    ae0f967e
model_test.py 21.5 KB