embeddings: no orphaned dinov2-embed processes; honest tooltip footprints

An engine crash re-parented its dinov2-embed child (still holding VRAM)
and the respawned engine stacked a second copy — the child now gets
PR_SET_PDEATHSIG (dies with the engine) and stale orphans for the same
model are reaped before spawn.

Engines-card footprints: a model loaded onto an already-full card
measures a ~0 VRAM delta and reported "0.01 GB" — report
max(measured, config estimate) instead.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
parent 1920e6aa
......@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here.
__version__ = "0.1.46"
__version__ = "0.1.47"
# Configure the CUDA caching allocator BEFORE torch is imported anywhere.
# expandable_segments lets the allocator return freed pages to the driver even
......
......@@ -369,8 +369,13 @@ async def internal_engine_state():
continue
_seen_li.add(_k)
try:
_vg = (multi_model_manager._measured_vram_gb.get(_k)
or multi_model_manager._get_model_used_vram_gb(_k) or 0)
# A load that happened while the card was already full measures
# a ~0 VRAM delta (free_before ≈ free_after) — never report that
# as the model's footprint; take the max of measurement and the
# config estimate.
_meas = multi_model_manager._measured_vram_gb.get(_k) or 0
_est = multi_model_manager._get_model_used_vram_gb(_k) or 0
_vg = max(float(_meas), float(_est))
except Exception:
_vg = 0
_cfg = {}
......
......@@ -236,10 +236,29 @@ def _load_embedding_model(model_name: str, device: str, model_config: dict = Non
_ngl = cfg.get('n_gpu_layers', raw.get('n_gpu_layers', -1))
if _ngl == 0:
env['DINOV2_FORCE_CPU'] = '1'
# Reap any ORPHANED embed server for this model first: if the engine
# process crashed, its child survived re-parented (still holding VRAM)
# and the respawned engine would stack a second copy next to it.
try:
subprocess.run(['pkill', '-f', f'dinov2-embed -m {model_name}'],
timeout=10)
except Exception:
pass
def _die_with_parent():
# PR_SET_PDEATHSIG: the kernel kills the child if the engine dies,
# so a crashed engine can never leak a VRAM-holding orphan again.
try:
import ctypes
ctypes.CDLL('libc.so.6').prctl(1, 9) # (PR_SET_PDEATHSIG, SIGKILL)
except Exception:
pass
proc = subprocess.Popen(
[_bin, '-m', model_name, '-t', '8'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, env=env, text=True, bufsize=1)
stderr=subprocess.DEVNULL, env=env, text=True, bufsize=1,
preexec_fn=_die_with_parent)
# wait for the ready line (model load), skipping loader chatter
import json as _json
import time as _time
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment