• Stefy Lanza (nextime / spora )'s avatar
    runpod: fix pods /v1 path; add stale-pod reaper, multi-GPU fallback, pod logs · b911310a
    Stefy Lanza (nextime / spora ) authored
    Live-tested against a real RunPod account (Secure RTX 4090, Qwen2.5-0.5B):
    provision → boot → vLLM /v1/models health → idle auto-teardown + ledger ($0.021)
    all confirmed. Serverless + spillover proxy paths confirmed (reach RunPod, correct
    URL/auth). Fixes/additions from that run:
    
    - FIX: pods mode posted to <proxy>/chat/completions (missing /v1) → 404. The pod's
      vLLM serves under /v1 (health check hit /v1/models=200), so RunpodBackend now
      appends /v1 to the pod proxy URL (serverless base already ends in /openai/v1).
    
    - Stale-pod REAPER (safety): runpod_worker.reap_orphans() lists account pods tagged
      coderai-<deployment_id>-* and terminates any NOT tracked by a live pool (orphans
      from a crash/restart) — never touches in-flight provisions or another deployment's
      pods. Runs every ~30s in the scaler loop AND once at startup; started on the
      primary engine (where pods are hosted) via start_runpod_maintenance(). New
      config.runpod.deployment_id (default "default"); pods named with the tag.
    
    - Multi-GPU capacity FALLBACK: _rank_gpus() returns ranked candidates and
      _create_with_fallback() tries the next on a RunPod "no instances available"
      capacity error instead of failing (hit a real A5000 community capacity miss).
    
    - Pod LOGS: RunpodClient.get_pod_logs() (best-effort REST) + pod_console_url deep
      link; admin GET /admin/api/runpod/pod-logs; RunPod stats page gets a per-pod
      "view logs" modal + console link. PodHandle/pods_status carry console_url.
    
    - Settings: deployment_id field.
    
    Built coderai:base=5cffe23d1d20. Relaunch to activate (running image had the /v1
    bug + no reaper). After relaunch, a pods chat returns a real completion.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
    b911310a
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...