• Stefy Lanza (nextime / spora )'s avatar
    runpod: adopt a pod only if it runs the same image on a card this pool prefers (v0.3.61) · 2e5d98a3
    Stefy Lanza (nextime / spora ) authored
    Adoption matched on the pool and a health probe, and the registry recorded
    nothing about what the pod was RUNNING. Two silent mistakes followed from that:
    
    * **the image.** The same model on a pinned tag and on `:latest` is different
      code. A pool configured for one could inherit a pod running the other, and
      nothing in the system could tell — which is precisely the confusion that cost
      a whole evening today: an OCR result was attributed to code that was not what
      the pod was running.
    * **the card.** A pod rented as a FALLBACK is on a card the configuration
      accepts only when its preferred one is unavailable. Inheriting one would serve
      a lease that asked for an A40 *and nothing else* (v0.3.54) from an RTX A6000 —
      the exact comparison that feature exists to protect. Live tonight the pool had
      both: an adopted A40 and a rented A6000 fallback for one model.
    
    So `register_pod` records the image and the card at creation (and keeps them
    across a re-registration, since an adopting owner does not know them), and
    `find_shared_pod` takes `image` and `gpus` and refuses anything that does not
    match, naming the pod and the reason. The one caller passes the image from the
    plan it would launch and the cards from `narrow_gpu_choice` — so a client
    lease's narrowing constrains adoption exactly as it constrains renting.
    
    An entry recording neither field is refused rather than trusted: refusing costs
    one extra pod once and the old one is reaped, while trusting costs a silent
    answer from the wrong code. A caller that names neither constraint keeps the old
    behaviour, so this adds constraints without adding requirements.
    
    Also here: an adopted pod is now billed from when it was CREATED, not from when
    it was taken over. `_terminate` computes the spend from `started_at`, which
    adoption set to "now" — so a pod created at 22:27 and adopted at 23:10 was
    ledgered for 16 minutes of an hour's life, and the rest was charged to nobody.
    The registry already records `created`, and only cost and uptime reporting read
    `started_at`.
    
    11 tests in tests/test_adoption_matches_image_and_card.py, one more in
    tests/test_pod_token_adoption_race.py.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
    2e5d98a3
test_adoption_matches_image_and_card.py 6.02 KB