• Stefy Lanza (nextime / spora )'s avatar
    runpod(phase2): serverless request path + per-model config + no-local-download · 17604ab4
    Stefy Lanza (nextime / spora ) authored
    End-to-end serverless: a model tagged backend:runpod (mode:serverless) proxies
    OpenAI chat/completions to a RunPod serverless endpoint. Runs on a remote GPU,
    holds NO local weights.
    
    - codai/api/runpod_worker.py: RunpodModelConfig + parse_model_runpod (mode/pools/
      criteria/gpu/price/scaling/cost/serverless), serverless_base_url + auth_headers.
    - codai/backends/runpod.py: RunpodBackend — thin OpenAI proxy (chat/stream, Bearer
      auth for serverless), uses_local_vram=False; pods/auto raise a clear "next phase"
      error for now.
    - manager.py: runpod in _ENGINE_BACKENDS + _runpod_name_claims (pin-only) +
      runpod_should_handle + load_model dispatch branch. _model_is_runpod() gate:
      set_default_model skips local download/cache for runpod-served models.
    - router.py + registry.py: runpod is an explicit-pin capability; added to every
      engine's default caps (no local VRAM → any node hosts the proxy; prefers primary).
    - admin routes: whitelist per-model `runpod` + `runpod_spillover` blocks; skip
      local-VRAM estimate for runpod models.
    - models.html: RunPod backend option + full per-model sub-form (mode w/ explainer,
      serverless endpoint, served model, pools/criteria/gpu/price/spot, min/max pods,
      idle timeout, image/disk/port/env, spend limit+period) + prefill/collect.
    
    Serverless verified wired in-image (ef1fd7a3802c). Pods pool/scaler, cost ledger,
    spillover routing, and stats UI are later phases.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
    17604ab4
Name
Last commit
Last update
..
archive.html Loading commit data...
base.html Loading commit data...
change_password.html Loading commit data...
chat.html Loading commit data...
dashboard.html Loading commit data...
login.html Loading commit data...
models.html Loading commit data...
settings.html Loading commit data...
tasks.html Loading commit data...
tokens.html Loading commit data...
users.html Loading commit data...