docs: network volumes, pools, and a recipe per kind of model

The guide told an operator which keys exist but not how to lay out a real
fleet. Adds a section covering:

- why a network volume is the single biggest lever: without one a pod
  re-downloads its weights on EVERY cold start, which is ten minutes for a
  15 GB repo and one to two hours for a 150 GB MoE, paid as rental before
  it answers anything;
- that the volume's region overrides data_center and the account setting,
  because a pod elsewhere cannot attach it — so two regions means two
  volumes;
- a suggested volume layout, and venv_on_volume;
- pooling: two models sharing `pool` share pods, but only CoderAI-engine
  models can, since a vLLM pod is launched --model X;
- worked blocks for vLLM, llama.cpp/GGUF, embeddings (warm + scheduled),
  native-engine OCR, and the big MoE engine pods, which REQUIRE a volume
  because colibri and k3 never download;
- LoRA/QLoRA: portable adapters travel as a reference and vLLM gets
  --enable-lora wired automatically, while a local-only adapter forces a
  CoderAI pod because vLLM and llama.cpp have no endpoint to receive one;
  the volume is the durable answer. Plus the two QLoRA traps — a
  quantization that must match what the adapter was trained against (it
  loads fine and silently degrades otherwise), and llama.cpp taking exactly
  one GGUF-converted adapter;
- a checklist for before you add the tenth model.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
parent 97dd707f
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment