• Stefy Lanza (nextime / spora )'s avatar
    runpod(phase1): account config + GraphQL client + admin key/GPU-catalog plumbing · 7c057af2
    Stefy Lanza (nextime / spora ) authored
    First slice of the RunPod remote-GPU backend (rent cloud GPUs, proxy inference).
    Dormant unless enabled — no request path yet.
    
    - config.py: RunpodConfig (enabled/api_key/cloud_type/api_base/rest_base/
      serverless_base/default_gpu_type/data_center/global_max_hourly_usd); registered,
      loaded, serialized. get_active_runpod_config() in manager.py.
    - codai/api/runpod_client.py: GraphQL client — ping, list_gpu_types (+prices),
      pick_gpu (cheapest under a $/hr ceiling, VRAM filter, spot-aware), create_pod
      (on-demand + interruptible/spot), get_pod, stop/terminate, list_pods, wait_ready.
      API key via ?api_key=, masked in all error strings.
    - admin: settings GET/save runpod block (key masked, overwritten only on a real
      new value); GET /admin/api/runpod/gpu-types (key check + catalog) and
      /admin/api/runpod/status (pods + live $/hr).
    - settings.html: RunPod card (key/cloud/default-gpu/global $/hr cap/DC/advanced
      bases) + "Test key & list GPUs" button.
    
    Decisions locked: host in primary engine (Layer-2, zero local VRAM); default
    vLLM-OpenAI pod image; per-model $/hr + global budget; per-model allow_spot;
    mode pods|serverless per model. TODO (request-path phase): RunPod requests must
    bypass the GLOBAL concurrency gate only (QueueManager.max_parallel_requests).
    
    Client GraphQL field names written from docs — pending live verification with a key.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01R6iugoNeEgt9PkajKrFyq4
    7c057af2
settings.html 129 KB