• Stefy Lanza (nextime / spora )'s avatar
    runpod: make the next pod failure diagnosable from the answer · 7c78c5fb
    Stefy Lanza (nextime / spora ) authored
    The last three pod failures were each diagnosed the same way: find the
    pod URL in the engine log, ask the pod by hand what it serves, work
    backwards. That is a diagnosis loop measured in minutes of log
    archaeology per attempt.
    
    Boot is now reported in PHASES, because "it took 150s" is not a
    diagnosis: port open after Ns (image pull + container start), serving
    after N more. The two have different fixes — a bigger boot_timeout_s
    versus a bigger load_timeout_s.
    
    The moment a pod is up, its /v1/models is logged. An empty catalogue now
    says so outright — "pod serves NO models — it will refuse every request"
    — instead of surfacing minutes later as "Model 'x' is not available. Use
    one of: " with nothing after the colon.
    
    And a test run's answer carries the pod diagnostics: which pods are live,
    their GPU, uptime, spend so far, URL, and the models each one lists. The
    fact that mattered in every failure so far was the catalogue, and it is
    now in the response rather than three commands away.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    7c78c5fb
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
colibri_worker.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lipsync.py Loading commit data...
log.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
model_test.py Loading commit data...
models_transfer.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...