-
Stefy Lanza (nextime / spora ) authored
collect_models unioned only registry.healthy() engines and re-fetched each engine's /v1/models live. An engine loading a model is GIL-blocked and misses the 2s health poll, so it goes "unhealthy" and ALL its models — including a freshly-added one — drop out of the aggregated /v1/models until the load finishes. A client (e.g. the kilo model script) polling during a load then sees models vanish. Cache each engine's last-good /v1/models and, when it's transiently unhealthy/unreachable, serve that cached list instead of dropping it. The models are still assigned to the engine and will serve once it's free. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com>
1c57629b