• Stefy Lanza (nextime / spora )'s avatar
    runpod: two pods for a warm floor of one (v0.2.95) · 5cec0702
    Stefy Lanza (nextime / spora ) authored
    Watched it happen on the production orchestrator: configuring qwen38-awq warm
    during office hours provisioned TWO A40s a second apart.
    
    A pod does not enter self.pods until it answers its health probe, which is
    minutes away. ensure_ready() never set the _provisioning flag, and the warm
    top-up in maintain() never read it — so the 15 s scaler saw "0 healthy, floor 1"
    while a pod was already booting and rented another. It stopped at two only
    because _provision_one blocks the scaler thread for the whole boot; a
    non-blocking provision would have added one every 15 seconds.
    
    The demand path (_should_grow) has always checked the flag. Now all three agree:
    one on the way is one pod, not none, including one a sibling engine sharing the
    pool is starting.
    
    Also: /v1/runpod/scale answered with the internal pool key ("pool:digesta-qwen38")
    rather than the model the client asked about. Several models share a pool, and
    that string is not something the caller sent or can look up — so the answer
    names the model and reports the pool beside it.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    5cec0702
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...