• Stefy Lanza (nextime / spora )'s avatar
    stamp the LRU clock when a model is registered by raw dict write · 1b83dc27
    Stefy Lanza (nextime / spora ) authored
    `_last_used` is stamped in exactly one place: the `active_in_vram` setter,
    whose comment calls it the "single chokepoint for 'this model was just
    used'". `add_model()` goes through it. Seven sites in video.py and
    images.py wrote `multi_model_manager.models[key] = obj` directly and did
    not, so for every video pipeline and four image models:
    
      - `_lru_order()` sorts by `_last_used.get(k, 0.0)`, so the key sorted
        FIRST and was evicted ahead of genuinely older models -- permanently,
        however recently it had been used;
      - the first eviction pass skips `key == self.active_in_vram`, which an
        unstamped key never matches, so that protection never applied either.
    
    Both ladders iterate `_lru_order()`: the VRAM one and the cross-engine
    `cosite_evict`. Observed on a production run -- a co-located engine needed
    room and released the freshly-loaded video pipeline before a TTS model
    that was older but had gone through `add_model`:
    
        [cosite-evict] releasing 'video:Wan-AI/Wan2.2-TI2V-5B-Diffusers' ...
        [cosite-evict] releasing 'tts:Kokoro-82M' ...
    
    Also disambiguate the evict-debug line that led here. It read
    `getattr(model_obj, 'backend', '∅')`, which returns the default both when
    the object has no such attribute and when `model_obj` is None -- so a
    healthy diffusers pipeline (which never carries `.backend`) and a key
    that had already vanished from `self.models` printed an identical
    `backend=missing`. Only the latter is a fault. Name the two apart, report
    the object type, and compare the sentinel by identity rather than `==`,
    which a tensor-bearing object need not answer with a bool.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1b83dc27
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
cluster Loading commit data...
compose Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
ocr Loading commit data...
openai Loading commit data...
pipelines Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
util Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
hf_cache.py Loading commit data...
main.py Loading commit data...
ocr_fleet_workers.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...