Three things, one of which is a security fix.

POD AUTH. backends/runpod.py said "the pod's vLLM server needs no auth"
and sent no credential. A RunPod proxy URL is reachable by anyone who
learns it, so every pod we have ever started was an open GPU on the
public internet. Pods are now launched locked to a bearer token, and
blank does NOT mean open: the pool generates one per pool unless
`allow_open_pod` is set explicitly. Each server takes it its own way —
`--api-key` for vLLM and llama.cpp, CODERAI_API_TOKEN for a coderai pod,
which required teaching BearerAuthMiddleware to accept an env token since
a pod has no auth.json to seed. The health probe sends it too: a locked
pod answers 401 on /v1/models, which would otherwise look exactly like
"never became ready". All three callers (always-remote backend, capability
gateway, front spill) now carry it.

SCALING. scale_up_inflight_per_pod was parsed and stored but never used:
the pool grew only when it had NO healthy pod, so with max_pods=3 fifty
concurrent requests all piled onto pod #1. It now grows when the
least-loaded pod reaches the threshold, in the BACKGROUND — the request
that proved the pool needs to grow is served by an existing pod rather
than waiting minutes for a cold one. `max_inflight_per_pod` (0 = let the
remote's batching absorb it) makes requests wait for a slot instead of
piling on, for engines that degrade under concurrency rather than
queueing internally.

CACHE. `sticky_sessions` sends a conversation back to the pod that
already served it, so the remote's prefix/KV cache still holds its
context; round-robin would re-prefill every turn from scratch. Bounded to
the newest 4096 conversations.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY