runpod: a base and its adapter are two models, not one path (v0.3.39)

Seeds were de-duplicated by `path`. A base model and a LoRA of it share one path
by definition, so the adapter's entry was dropped and the pod was told about the
base alone — observed live, one line after the sibling seeding had correctly
found it:

    [runpod] pool 'pool:digesta-qwen38' is shared with: diogene-alpha
    [runpod] pod will be told about 1 model(s)

The dedup exists because a capability pod's list can overlap the one model a
per-model pool also seeds, which is a real case; it just has to key on what the
pod will SERVE the entry as — path plus alias, id and adapters — rather than on
the weights the entries happen to have in common.

Orchestrator-side only: the pod registers what it is handed, so no new image.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxYHWbCAjjjD7A1rhFKtaW
parent cd133742
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API # Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here. # metadata and the admin web UI read from here.
__version__ = "0.3.38" __version__ = "0.3.39"
__tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference." __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
# Configure the CUDA caching allocator BEFORE torch is imported anywhere. # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
......
...@@ -1104,13 +1104,18 @@ def _plan(engine, image, args, mcfg, api_key, entry, served, ...@@ -1104,13 +1104,18 @@ def _plan(engine, image, args, mcfg, api_key, entry, served,
if one: if one:
seeds.extend(_json.loads(one)) seeds.extend(_json.loads(one))
if seeds: if seeds:
# De-duplicate by path: a capability pod's list can overlap the one model # De-duplicate by what the pod will SERVE it as, not by path: a capability
# a per-model pool also seeds. # pod's list can overlap the one model a per-model pool also seeds, which
# is what this guards against — but a base model and a LoRA of it share
# one path by definition, so keying on path silently dropped the adapter
# and the pod was "told about 1 model(s)" when it needed two.
seen, unique = set(), [] seen, unique = set(), []
for s in seeds: for s in seeds:
if s.get("path") in seen: key = (s.get("path"), s.get("alias") or "", s.get("id") or "",
_json.dumps(s.get("loras") or [], sort_keys=True))
if key in seen:
continue continue
seen.add(s.get("path")) seen.add(key)
unique.append(s) unique.append(s)
env["CODERAI_SEED_MODELS"] = _json.dumps(unique) env["CODERAI_SEED_MODELS"] = _json.dumps(unique)
print(f"[runpod] pod will be told about {len(unique)} model(s)", flush=True) print(f"[runpod] pod will be told about {len(unique)} model(s)", flush=True)
......
...@@ -382,3 +382,20 @@ def test_the_limiter_can_be_turned_off_by_env(monkeypatch): ...@@ -382,3 +382,20 @@ def test_the_limiter_can_be_turned_off_by_env(monkeypatch):
assert ratelimit._limits_enabled() is True assert ratelimit._limits_enabled() is True
monkeypatch.setenv("CODERAI_SERVER_RATE_LIMIT_ENABLED", "0") monkeypatch.setenv("CODERAI_SERVER_RATE_LIMIT_ENABLED", "0")
assert ratelimit._limits_enabled() is False assert ratelimit._limits_enabled() is False
def test_a_base_and_its_adapter_both_reach_the_pod(monkeypatch):
"""Seeds were de-duplicated by PATH, and a base model and a LoRA of it share
one path by definition — so the adapter was dropped and the pod was told
about one model when it needed two. Observed live: "told about 1 model(s)"."""
from codai.api import runpod_worker as rw
base = {"id": "qwen38-awq", "path": "cyankiwi/Qwen3.8-27B-AWQ-INT4",
"alias": "qwen38"}
tuned = {"id": "diogene-alpha", "path": "cyankiwi/Qwen3.8-27B-AWQ-INT4",
"loras": [{"name": "diogene-alpha", "path": "sha256:" + "a" * 64}]}
mcfg = rw.parse_model_runpod({})
plan = rw._plan("vllm", "img", "", mcfg, "key", base, "qwen38-awq",
seed_entries=[tuned])
seeded = json.loads((plan.get("env") or {}).get("CODERAI_SEED_MODELS") or "[]")
assert len(seeded) == 2, seeded
assert any(m.get("loras") for m in seeded), "the adapter's entry was dropped"
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment