-
Stefy Lanza (nextime / spora ) authored
Implements the HomeHunter request (REQUEST-geoclip-embeddings.md): two model ids on the existing embeddings endpoint, both returning 512-d L2-normalised vectors in ONE shared space so a property photo can be scored directly against candidate GPS coordinates. geoclip — image encoder (CLIP ViT-L/14 + projection MLP), on GPU. Accepts the image data URI in EITHER `input` (per the request) or the `image` field (like dinov2). geoclip-location — GPS encoder (equal-earth + RFF MLP), pinned to CPU: it is a tiny MLP, so CPU is fast, frees VRAM, and is bit-exact across batch sizes (GPU reduction order otherwise perturbs a cached coordinate by ~1e-7). Batches an array of "lat,lon" strings to one vector per element, in input order. Both sides load from the SAME bundled GeoCLIP checkpoint, so the spaces coincide by construction — the mismatched-checkpoint failure the request warns about cannot happen. Loaded as separate _EmbeddingModel('geoclip', ...) instances so each is independently evictable. transformers>=5 compat: CLIPModel.get_image_features now returns a BaseModelOutputWithPooling (768-d projected features in pooler_output) instead of a bare tensor, which breaks GeoCLIP's internal mlp() call. The image path extracts the 768-d tensor itself (tolerant of both the old tensor and the new object) before the projection MLP. geoclip (+geopy, geographiclib) added to requirements.txt; its heavy deps (torch, transformers, Pillow, pandas, numpy) are already pinned, so no ML-stack churn. Verified end-to-end: 512-d L2-normalised both sides; batching + index order; same request bit-exact; both ids in GET /v1/models; image·location scoring varies by image. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoSpEthysqmseCc6Geizty
fd3221c6
This project manages its dependencies using
pip.
Learn more