-
Stefy Lanza (nextime / spora ) authored
Live testing showed RunPod sometimes ACCEPTS a pod create but the machine never starts (port never exposed) — 2 of 4 pods hung this way, burning the full 900s wait. Provisioning is now resilient and diagnosable: - Boot failure retries on the NEXT ranked GPU candidate (MAX_PROVISION_ATTEMPTS=3) instead of failing the request; _create_with_fallback returns the next index so each attempt uses a different machine/type. - Timeouts split + tightened: port-exposed 300s (was 900 — a pod that hasn't exposed its port in 5min never will), vLLM /v1/models health 600s (model load). Cuts worst-case billing on a bad machine from ~15min to ~5min per attempt. - Self-diagnosing: on any boot failure the pod's container/vLLM log is fetched and printed to coderai's log before teardown (the user's #1 cause is a vLLM OOM / launch error, only visible in that log), plus the console link. - Pod id + console URL are now logged immediately after create (previously only at "ready", so a stuck pod was untraceable). - Booting pods appear in pods_status/stats page as state="provisioning" with their console link, so a slow boot is visible live. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
0c8417fa
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| static | ||
| templates | ||
| __init__.py | ||
| auth.py | ||
| download_worker.py | ||
| routes.py |