• Stefy Lanza (nextime / spora )'s avatar
    runpod: send local text adapters to the pod that can receive them · a746e0fe
    Stefy Lanza (nextime / spora ) authored
    A text LoRA that exists only on this machine had nowhere to go: vLLM and
    llama.cpp resolve adapters at launch and have no endpoint to upload one
    to, so the previous commit could only report and skip it.
    
    Such a model is now routed to a coderai TEXT pod — the one pod image that
    can be sent an adapter — and the transfer is arranged around the fact
    that a pod is created before anything can be sent to it:
    
      1. the pod is told the adapter's sha256 id, computed HERE, so both
         sides agree on the name before the pod exists;
      2. the bytes go to the pod's content-addressed blob store before the
         first request, once per pod (a repeat costs one existence check);
      3. the pod resolves sha256:<hex> back to a file — text_loras.local_path
         now understands the content-addressed forms — and applies it.
    
    A published adapter still goes to vLLM, which fetches it itself: cheaper
    and faster, so publishing an adapter you use often remains the better
    path, and the log says so. A PEFT adapter DIRECTORY is still refused with
    a reason — the blob store holds single files.
    
    packaging/runpod/profiles/text.txt adds the text profile (transformers,
    peft, bitsandbytes for QLoRA bases). It has to be built and pushed before
    use; the other nine are published already.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    a746e0fe
text_loras.py 7.27 KB