Skip to content
Commit 540566b0 authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

embeddings: serialize per-model loads — concurrent bursts stacked duplicate copies



A bulk indexer's burst of first-requests all saw model_obj None and EACH
loaded its own copy of the embedder (observed pool_instances=3 for the
4B model + 6 for DINOv2 ≈ the whole card) — this duplicate stacking,
not a single load, was what exhausted VRAM and poisoned the measured
footprint (26.8 GB recorded for an 8 GB model, measured across
overlapping loads). Loads now take a per-model asyncio lock with a
re-check, so one request loads and the rest reuse the loaded model.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
parent 2ea24b0b
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment