-
Stefy Lanza (nextime / spora ) authored
Testing an image model by setting remotes.endpoints["images"] moves every request of that kind. An embeddings remote left on from an earlier test did exactly that on a live box: production embedding traffic went to a rented pod while the local GPU sat idle, and the operator noticed because the card was quiet. tools/runpod_model_test.py writes a runpod block on the single model entry instead — per-model placement already beats the capability map, which is what lets two models of the same kind sit in different places. Writing the config is also not enough. The front pushes a reload and the engine takes it only when idle, so a harness that writes and immediately tests is testing the PREVIOUS configuration. That produced a whole pass of false "nothing configures this to run remotely" failures, and could just as easily have reported a remote pass for a request served locally. GET /v1/models/test/state answers for the process that serves the request; the harness polls it until the placement it wrote is visible and refuses to test if it never is. The new endpoint is in _SKIP_PATHS: forwarded, it would report a pod's view of a config the pod does not have. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
36f53303