• Stefy Lanza (nextime / spora )'s avatar
    stamp the LRU clock when a model is registered by raw dict write · 1b83dc27
    Stefy Lanza (nextime / spora ) authored
    `_last_used` is stamped in exactly one place: the `active_in_vram` setter,
    whose comment calls it the "single chokepoint for 'this model was just
    used'". `add_model()` goes through it. Seven sites in video.py and
    images.py wrote `multi_model_manager.models[key] = obj` directly and did
    not, so for every video pipeline and four image models:
    
      - `_lru_order()` sorts by `_last_used.get(k, 0.0)`, so the key sorted
        FIRST and was evicted ahead of genuinely older models -- permanently,
        however recently it had been used;
      - the first eviction pass skips `key == self.active_in_vram`, which an
        unstamped key never matches, so that protection never applied either.
    
    Both ladders iterate `_lru_order()`: the VRAM one and the cross-engine
    `cosite_evict`. Observed on a production run -- a co-located engine needed
    room and released the freshly-loaded video pipeline before a TTS model
    that was older but had gone through `add_model`:
    
        [cosite-evict] releasing 'video:Wan-AI/Wan2.2-TI2V-5B-Diffusers' ...
        [cosite-evict] releasing 'tts:Kokoro-82M' ...
    
    Also disambiguate the evict-debug line that led here. It read
    `getattr(model_obj, 'backend', '∅')`, which returns the default both when
    the object has no such attribute and when `model_obj` is None -- so a
    healthy diffusers pipeline (which never carries `.backend`) and a key
    that had already vanished from `self.models` printed an identical
    `backend=missing`. Only the latter is a fault. Name the two apart, report
    the object type, and compare the sentinel by identity rather than `==`,
    which a tensor-bearing object need not answer with a bool.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1b83dc27
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
acestep_worker.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
audiocraft_worker.py Loading commit data...
canary_worker.py Loading commit data...
characters.py Loading commit data...
chunking.py Loading commit data...
colibri_worker.py Loading commit data...
compose.py Loading commit data...
context_overflow.py Loading commit data...
crisperwhisper_worker.py Loading commit data...
custom_pipelines.py Loading commit data...
diarization.py Loading commit data...
ds4_kv_janitor.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
fastlog.py Loading commit data...
h3_worker.py Loading commit data...
host_worker.py Loading commit data...
hub_compat.py Loading commit data...
images.py Loading commit data...
k3_worker.py Loading commit data...
kt_worker.py Loading commit data...
lang_codes.py Loading commit data...
lipsync.py Loading commit data...
lipsync_engines.py Loading commit data...
log.py Loading commit data...
longcat_worker.py Loading commit data...
loop_watch.py Loading commit data...
lora_archs.py Loading commit data...
loras.py Loading commit data...
melotts_worker.py Loading commit data...
model_test.py Loading commit data...
models_transfer.py Loading commit data...
moderation.py Loading commit data...
mux_remote.py Loading commit data...
ocr.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
pod_http.py Loading commit data...
pod_telemetry.py Loading commit data...
pod_tls.py Loading commit data...
podlog.py Loading commit data...
prompt_cache.py Loading commit data...
pyannote_worker.py Loading commit data...
ratelimit.py Loading commit data...
remote_gateway.py Loading commit data...
request_progress.py Loading commit data...
rerank.py Loading commit data...
runpod_client.py Loading commit data...
runpod_fleet_share.py Loading commit data...
runpod_ledger.py Loading commit data...
runpod_worker.py Loading commit data...
serving_load.py Loading commit data...
spatial.py Loading commit data...
speaker_embeddings.py Loading commit data...
speaker_registry.py Loading commit data...
state.py Loading commit data...
stt_backends.py Loading commit data...
text.py Loading commit data...
thread_pool.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
uploads.py Loading commit data...
urlutils.py Loading commit data...
venv_env.py Loading commit data...
video.py Loading commit data...
vllm_worker.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...
vpr_salad_server.py Loading commit data...
xtts_worker.py Loading commit data...