• Stefy Lanza (nextime / spora )'s avatar
    read pod telemetry from the endpoint pods actually serve (v0.4.68) · a4e61707
    Stefy Lanza (nextime / spora ) authored
    v0.4.67 asked each pod for /admin/api/gpu-stats. Pods answer 404: a pod runs
    the SLIM app — /healthz, /v1/models, /internal/engine-state — and no admin API
    at all. Built on an assumption; the live check caught it. Probe the pod before
    choosing its endpoint.
    
    /internal/engine-state already carries a `vram` block: used, free and total per
    card with the GPU's name, built context-free and cached 4s there. It is what
    the front health-polls, so every pod has it and it costs the pod nothing extra.
    Verified against a live A40: 43.8/48 GB with qwen38 resident.
    
    The pod reports FREE, not used — that is what a context-free query returns — so
    used is the difference. Inverting that would show an idle pod as full.
    
    Utilisation and temperature added AT THE SOURCE: nvidia-smi now asks for
    utilization.gpu and temperature.gpu in the same call as the memory query rather
    than paying for a second one (a saturated card answers slowly enough that one
    call matters), with the pynvml equivalent, and both ride into
    engine-state's vram.devices. Absent or "[N/A]" is left out, never zeroed: a
    zero draws a bar that reads as "idle" when the truth is "not known".
    
    So VRAM works on every pod already running; utilisation needs the next pod
    image, and until then that column shows a dash — which is the degradation the
    page was built for.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    a4e61707
test_pod_gpu_telemetry.py 10.3 KB