• Stefy Lanza (nextime / spora )'s avatar
    test-run: place ONE model on a pod, and prove the engine took the config · 36f53303
    Stefy Lanza (nextime / spora ) authored
    Testing an image model by setting remotes.endpoints["images"] moves every
    request of that kind. An embeddings remote left on from an earlier test did
    exactly that on a live box: production embedding traffic went to a rented pod
    while the local GPU sat idle, and the operator noticed because the card was
    quiet. tools/runpod_model_test.py writes a runpod block on the single model
    entry instead — per-model placement already beats the capability map, which is
    what lets two models of the same kind sit in different places.
    
    Writing the config is also not enough. The front pushes a reload and the engine
    takes it only when idle, so a harness that writes and immediately tests is
    testing the PREVIOUS configuration. That produced a whole pass of false "nothing
    configures this to run remotely" failures, and could just as easily have
    reported a remote pass for a request served locally. GET /v1/models/test/state
    answers for the process that serves the request; the harness polls it until the
    placement it wrote is visible and refuses to test if it never is.
    
    The new endpoint is in _SKIP_PATHS: forwarded, it would report a pod's view of a
    config the pod does not have.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    36f53303
test_remote_execution.py 65.3 KB