Skip to content
Commit 256900b7 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

runpod: per-model boot/load timeouts (big models need longer)



The 300s boot cap was a hard-coded constant; large models need much more time for
the image pull + weight download. Both budgets are now per-model config:

- boot_timeout_s (default 300) — until the pod exposes its port; a stuck machine
  is still dropped fast and the next candidate tried.
- load_timeout_s (default 600) — until vLLM answers /v1/models; raise for big models.

Parsed in RunpodModelConfig (min 30s), used by _provision_one in place of the class
constants (which remain the fallback), whitelisted in api_model_configure, and
exposed on the model page with hints.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
parent d9e49200
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment