township: add a fighter's reference photo from a link (v0.2.47)

Uploading a photo to a fighter was never missing — the Characters page has had a
file picker and a working /profile/upload-image endpoint all along. It LOOKED
missing because the page's whole <script> block has been dead since v0.2.41
(fixed in 11b0f446), so the button did nothing at all. Anyone on 0.2.41-0.2.46
sees a picker that ignores them.

What was genuinely absent is the other half: pasting a link. That now feeds the
SAME endpoint, so a fetched picture goes through the same magic-byte check (the
extension is never trusted), the same ref_NN numbering, and the same
patch_profile sync to CoderAI. The caller gets one answer covering both sources.

I wrote a second endpoint for this first and deleted it: /profile/upload-image
already did everything except fetch, and a parallel route would have been a
second place to keep the validation, the numbering and the sync correct.

The fetch is guarded the same way as the character studio's, for the same reason
— the request is made by the tool, which can reach coderai on 127.0.0.1 and the
whole LAN:

- every hop must resolve to a public address, checked by RESOLVING the host
  rather than matching the string;
- redirects followed by hand, five hops, each re-checked (a public URL that 302s
  to 127.0.0.1 is the usual way past a one-time check);
- http/https only; 64 MB ceiling enforced while streaming;
- the content must decode as PNG/JPEG/WebP/GIF by magic bytes.

--allow-private-fetch opts out, for pulling from your own NAS.

Verified against a real profile: an uploaded JPEG became ref_01.jpg, a public
link became ref_02.png, and a link to 127.0.0.1:8776 was refused with the reason.
"synced": false is reported honestly when CoderAI is unreachable rather than
claiming the profile was updated.

131 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY

codai/__init__.py

@@ -16,7 +16,7 @@
 
 # Canonical product version for CoderAI — single source of truth. Both the API
 # metadata and the admin web UI read from here.
-__version__ = "0.2.46"
+__version__ = "0.2.47"
 __tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
 
 # Configure the CUDA caching allocator BEFORE torch is imported anywhere.

tests/test_township_page_js.py

@@ -92,3 +92,66 @@ def test_the_template_picker_is_filled_on_load(page):
     init = page.split("Restore state on page load", 1)
     assert len(init) == 2, "the page-load init block is gone"
     assert "tplRefresh();" in init[1][:600], "nothing fills the picker on load"
+
+
+# ── adding reference photos to a fighter ────────────────────────────────────
+# The page has had a file picker and a working /profile/upload-image endpoint all
+# along; what made it look missing is that the whole <script> was dead, so the
+# button did nothing. A link source now feeds the same endpoint.
+
+import json as _json
+import urllib.request as _urlreq
+
+
+def _post_form(path, fields, port=PORT):
+    """multipart/form-data without pulling in a dependency."""
+    boundary = "----codertest"
+    body = b""
+    for key, value in fields.items():
+        body += (f"--{boundary}\r\nContent-Disposition: form-data; name=\"{key}\"\r\n\r\n"
+                 f"{value}\r\n").encode()
+    body += f"--{boundary}--\r\n".encode()
+    req = _urlreq.Request(f"http://127.0.0.1:{port}{path}", data=body, method="POST",
+                          headers={"Content-Type":
+                                   f"multipart/form-data; boundary={boundary}"})
+    try:
+        with _urlreq.urlopen(req, timeout=20) as r:
+            return _json.loads(r.read())
+    except urllib.error.HTTPError as exc:
+        return _json.loads(exc.read())
+
+
+def test_the_characters_page_offers_both_sources(page):
+    """The picker was always there; the link input is the new half."""
+    import urllib.request
+    with urllib.request.urlopen(f"http://127.0.0.1:{PORT}/characters", timeout=10) as r:
+        html = r.read().decode("utf-8", "replace")
+    assert "data-upload=files" in html, "the file picker is gone"
+    assert "data-upload=url" in html, "no way to paste a link"
+    assert "/profile/upload-image" in html
+
+
+@pytest.mark.parametrize("url,why", [
+    ("http://127.0.0.1:8776/favicon.ico", "coderai's own API is on loopback"),
+    ("http://10.0.0.5/photo.jpg", "a private LAN address"),
+    ("http://169.254.169.254/latest/", "the cloud metadata address"),
+])
+def test_a_pasted_link_cannot_reach_this_machine(page, url, why):
+    """The tool makes the request, so a link is otherwise a way into the host."""
+    got = _post_form("/profile/upload-image",
+                     {"kind": "character", "name": "nobody", "url": url})
+    assert "error" in got, f"{why}: expected a refusal, got {got}"
+    assert "not a public address" in got["error"], got["error"]
+
+
+def test_a_link_that_is_not_http_is_refused(page):
+    got = _post_form("/profile/upload-image",
+                     {"kind": "character", "name": "nobody", "url": "file:///etc/passwd"})
+    assert "error" in got and "http(s)" in got["error"], got
+
+
+def test_a_bad_profile_name_cannot_escape_the_output_tree(page):
+    for name in ("../../etc", "a/b", "a\\b"):
+        got = _post_form("/profile/upload-image",
+                         {"kind": "character", "name": name, "url": "https://example.com/a.jpg"})
+        assert got.get("error") == "invalid kind/name", (name, got)

tools/gen_township_fighters.py

@@ -705,6 +705,98 @@ def _ensure_in_coderai(client, kind: str, name: str, out_dir: Path) -> bool:
         return False
 
 
+PROFILE_IMAGE_EXTS = (".png", ".jpg", ".jpeg", ".webp")
+# A link the operator pastes is fetched by THIS process, so it needs a ceiling.
+MAX_FETCH_MB = 64
+
+
+def fetch_image_url(url: str, allow_private: bool = False) -> tuple:
+    """Download a picture from a pasted link. Returns (bytes, filename).
+
+    The request is made by the tool, with the tool's network reach — on this box
+    that includes coderai on 127.0.0.1 and the whole LAN. So every hop has to
+    resolve to a public address unless the operator opted out, redirects are
+    followed by hand so each one is re-checked, and the size is capped while
+    streaming rather than trusted from a header.
+    """
+    import ipaddress
+    import mimetypes as _mt
+    import socket
+    import urllib.error
+    import urllib.request
+
+    class _NoRedirect(urllib.request.HTTPRedirectHandler):
+        def redirect_request(self, req, fp, code, msg, headers, newurl):
+            return None
+
+    def _check(parsed):
+        if parsed.scheme not in ("http", "https"):
+            raise ValueError(f"only http(s) links are supported, not {parsed.scheme!r}")
+        host = parsed.hostname
+        if not host:
+            raise ValueError("no host in the link")
+        if allow_private:
+            return
+        try:
+            infos = socket.getaddrinfo(host, None)
+        except OSError as exc:
+            raise ValueError(f"cannot resolve {host}: {exc}") from None
+        for info in infos:
+            ip = ipaddress.ip_address(info[4][0])
+            if (ip.is_private or ip.is_loopback or ip.is_link_local
+                    or ip.is_reserved or ip.is_multicast or ip.is_unspecified):
+                raise ValueError(f"{host} resolves to {ip}, which is not a public "
+                                 f"address (use --allow-private-fetch if deliberate)")
+
+    limit = int(MAX_FETCH_MB * 1024 * 1024)
+    target = (url or "").strip()
+    resp = None
+    for _hop in range(5):
+        parsed = urllib.parse.urlparse(target)
+        _check(parsed)
+        req = urllib.request.Request(target, headers={
+            "User-Agent": "coderai-township", "Accept": "image/*,*/*;q=0.5"})
+        try:
+            resp = urllib.request.build_opener(_NoRedirect).open(req, timeout=20)
+        except urllib.error.HTTPError as exc:
+            if exc.code in (301, 302, 303, 307, 308) and exc.headers.get("Location"):
+                target = urllib.parse.urljoin(target, exc.headers["Location"])
+                continue
+            raise ValueError(f"{exc.code} {exc.reason}") from None
+        except urllib.error.URLError as exc:
+            raise ValueError(f"could not fetch: {exc.reason}") from None
+        break
+    if resp is None:
+        raise ValueError("too many redirects")
+
+    with resp:
+        ctype = (resp.headers.get("Content-Type") or "").split(";")[0].strip().lower()
+        declared = int(resp.headers.get("Content-Length") or 0)
+        if declared and declared > limit:
+            raise ValueError(f"{declared/1e6:.0f} MB exceeds the {MAX_FETCH_MB} MB limit")
+        fname = os.path.basename(urllib.parse.urlparse(target).path) or "photo"
+        suffix = Path(fname).suffix.lower()
+        if suffix not in PROFILE_IMAGE_EXTS:
+            # Plenty of image URLs carry no extension; fall back to the type.
+            suffix = _mt.guess_extension(ctype) or ""
+            if suffix == ".jpe":
+                suffix = ".jpg"
+            fname = Path(fname).stem + suffix
+        if suffix not in PROFILE_IMAGE_EXTS:
+            raise ValueError(f"that link is {ctype or 'of unknown type'}, not a picture")
+        data = b""
+        while True:
+            chunk = resp.read(256 * 1024)
+            if not chunk:
+                break
+            data += chunk
+            if len(data) > limit:
+                raise ValueError(f"the picture is larger than the {MAX_FETCH_MB} MB limit")
+    if not data:
+        raise ValueError("the link returned an empty file")
+    return data, fname
+
+
 # ─────────────────────────────────────────────────────────────────────────────
 # Fatal-error detection
 # ─────────────────────────────────────────────────────────────────────────────
@@ -8478,8 +8570,10 @@ fetch('/status').then(r=>r.json()).then(d=>{{
                 f'    <label style="margin:0;font-size:.78rem">Or upload your own:</label>'
                 f'    <input type=file data-upload=files accept="image/*" multiple '
                 f'style="font-size:.76rem;width:auto;flex:1;min-width:160px">'
+                f'    <input type=url data-upload=url placeholder="…or paste a link to a photo" '
+                f'style="font-size:.76rem;width:auto;flex:1;min-width:180px">'
                 f'    <button class="btn btn-secondary btn-sm" '
-                f'onclick="uploadRefs(\'{kind}\',\'{esc(name)}\')">⬆ Upload references</button>'
+                f'onclick="uploadRefs(\'{kind}\',\'{esc(name)}\')">⬆ Add references</button>'
                 f'    <span class=pf-upload-status style="font-size:.76rem;color:var(--blue-corner)"></span>'
                 f'  </div>'
                 f'  <div class=pf-actions style="border-top:1px solid var(--line);padding-top:.6rem;margin-top:.6rem">'
@@ -8651,14 +8745,19 @@ document.addEventListener('DOMContentLoaded', resumeActiveJobs);
 async function uploadRefs(kind,name){
   const root=document.getElementById('pf-'+kind+'-'+name);
   const inp=root.querySelector('[data-upload=files]');
+  const urlEl=root.querySelector('[data-upload=url]');
+  const url=urlEl ? (urlEl.value||'').trim() : '';
   const st=root.querySelector('.pf-upload-status');
-  if(!inp||!inp.files||!inp.files.length){
-    st.style.color='var(--bad)'; st.textContent='Choose image file(s) first'; return;
+  const nFiles=(inp&&inp.files)?inp.files.length:0;
+  if(!nFiles && !url){
+    st.style.color='var(--bad)'; st.textContent='Choose image file(s) or paste a link first'; return;
   }
   const fd=new FormData();
   fd.append('kind',kind); fd.append('name',name);
-  for(const f of inp.files) fd.append('files',f);
-  st.style.color='var(--text-dim)'; st.textContent='Uploading '+inp.files.length+' file(s)…';
+  for(const f of (inp&&inp.files?inp.files:[])) fd.append('files',f);
+  if(url) fd.append('url',url);
+  st.style.color='var(--text-dim)';
+  st.textContent='Adding '+(nFiles?nFiles+' file(s)':'')+((nFiles&&url)?' + ':'')+(url?'1 link':'')+'\u2026';
   try{
     const r=await fetch('/profile/upload-image',{method:'POST',body:fd});
     const j=await r.json();
@@ -11374,6 +11473,19 @@ async function resetPrompts(ev){
                 base = out_dir / (kind + "s") / name
                 base.mkdir(parents=True, exist_ok=True)
                 added_uris, rejected = [], 0
+                fetch_error = ""
+                # A pasted link is fetched HERE and then joins the uploaded files,
+                # so it goes through the same magic-byte check, the same reference
+                # numbering and the same sync to CoderAI.
+                _url = (fields.get("url") or "").strip()
+                if _url:
+                    try:
+                        _data, _fname = fetch_image_url(
+                            _url,
+                            allow_private=getattr(default_args, "allow_private_fetch", False))
+                        files = list(files) + [{"filename": _fname, "data": _data}]
+                    except Exception as e:
+                        fetch_error = str(e)
                 for f in files:
                     data = f.get("data") or b""
                     ext, mime = _img_ext(data)
@@ -11390,7 +11502,8 @@ async function resetPrompts(ev){
                         mime, __import__("base64").b64encode(data).decode()))
                 if not added_uris:
                     self._send(400, "application/json",
-                               _j.dumps({"error": "no valid image files uploaded"}))
+                               _j.dumps({"error": fetch_error
+                                         or "no valid image files uploaded"}))
                     return
                 synced = True
                 try:
@@ -12687,6 +12800,11 @@ OUTPUT LAYOUT
                              "Without this flag the script launches a web UI instead of processing.")
     parser.add_argument("--web-port", type=int, default=7788, metavar="PORT",
                         help="Port for the web UI (default: 7788, only used without --cli-mode).")
+    parser.add_argument("--allow-private-fetch", action="store_true",
+                        help="Let 'paste a link' on the Characters page reach private/"
+                             "loopback addresses. Off by default: the tool makes that "
+                             "request itself, so a link would otherwise be a way to pull "
+                             "from 127.0.0.1 or the LAN. Turn it on for your own NAS.")
     parser.add_argument("--browser", action="store_true",
                         help="Auto-open a web browser at the UI URL on startup. Off by default "
                              "(avoids spawning a terminal text browser on headless servers).")