Skip to content
Commit 36f53303 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

test-run: place ONE model on a pod, and prove the engine took the config



Testing an image model by setting remotes.endpoints["images"] moves every
request of that kind. An embeddings remote left on from an earlier test did
exactly that on a live box: production embedding traffic went to a rented pod
while the local GPU sat idle, and the operator noticed because the card was
quiet. tools/runpod_model_test.py writes a runpod block on the single model
entry instead — per-model placement already beats the capability map, which is
what lets two models of the same kind sit in different places.

Writing the config is also not enough. The front pushes a reload and the engine
takes it only when idle, so a harness that writes and immediately tests is
testing the PREVIOUS configuration. That produced a whole pass of false "nothing
configures this to run remotely" failures, and could just as easily have
reported a remote pass for a request served locally. GET /v1/models/test/state
answers for the process that serves the request; the harness polls it until the
placement it wrote is visible and refuses to test if it never is.

The new endpoint is in _SKIP_PATHS: forwarded, it would report a pod's view of a
config the pod does not have.

Co-Authored-By: default avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
parent ae0f967e
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment