-
Stefy Lanza (nextime / spora ) authored
Completes the RunPod backend for testing. Pods pool (runpod_worker.py): - RunpodPodPool: provision via _select_gpu (cheaper=lowest $/hr, faster=SECURE+ throughput, across allowed pools, spot-aware) → create_pod → wait_ready → vLLM health-check; least-loaded per-request acquire, provision-on-demand up to max_pods; background scaler thread tears down idle pods to min after idle_timeout_s, keeps min warm, reaps. get_pod_pool registry + pods_status(). - Budget enforcement at provision: global $/hr rate cap + per-model & global cumulative spend caps (rolling period) via the ledger; refuses over budget. Cost ledger (runpod_ledger.py): persistent per-model + global spend, trailing rolling windows (hour/day/week/month), totals() for stats. Final pod cost booked on teardown; live pod cost added on top for enforcement/stats (no double count). Backend (backends/runpod.py): pods mode wired to the poo...
7649c9c2Stefy Lanza (nextime / spora ) authoredCompletes the RunPod backend for testing. Pods pool (runpod_worker.py): - RunpodPodPool: provision via _select_gpu (cheaper=lowest $/hr, faster=SECURE+ throughput, across allowed pools, spot-aware) → create_pod → wait_ready → vLLM health-check; least-loaded per-request acquire, provision-on-demand up to max_pods; background scaler thread tears down idle pods to min after idle_timeout_s, keeps min warm, reaps. get_pod_pool registry + pods_status(). - Budget enforcement at provision: global $/hr rate cap + per-model & global cumulative spend caps (rolling period) via the ledger; refuses over budget. Cost ledger (runpod_ledger.py): persistent per-model + global spend, trailing rolling windows (hour/day/week/month), totals() for stats. Final pod cost booked on teardown; live pod cost added on top for enforcement/stats (no double count). Backend (backends/runpod.py): pods mode wired to the poo...
Loading