• Stefy Lanza (nextime / spora )'s avatar
    stamp the LRU clock when a model is registered by raw dict write · 1b83dc27
    Stefy Lanza (nextime / spora ) authored
    `_last_used` is stamped in exactly one place: the `active_in_vram` setter,
    whose comment calls it the "single chokepoint for 'this model was just
    used'". `add_model()` goes through it. Seven sites in video.py and
    images.py wrote `multi_model_manager.models[key] = obj` directly and did
    not, so for every video pipeline and four image models:
    
      - `_lru_order()` sorts by `_last_used.get(k, 0.0)`, so the key sorted
        FIRST and was evicted ahead of genuinely older models -- permanently,
        however recently it had been used;
      - the first eviction pass skips `key == self.active_in_vram`, which an
        unstamped key never matches, so that protection never applied either.
    
    Both ladders iterate `_lru_order()`: the VRAM one and the cross-engine
    `cosite_evict`. Observed on a production run -- a co-located engine needed
    room and released the freshly-loaded video pipeline before a TTS model
    that was older but had gone through `add_model`:
    
        [cosite-evict] releasing 'video:Wan-AI/Wan2.2-TI2V-5B-Diffusers' ...
        [cosite-evict] releasing 'tts:Kokoro-82M' ...
    
    Also disambiguate the evict-debug line that led here. It read
    `getattr(model_obj, 'backend', '∅')`, which returns the default both when
    the object has no such attribute and when `model_obj` is None -- so a
    healthy diffusers pipeline (which never carries `.backend`) and a key
    that had already vanished from `self.models` printed an identical
    `backend=missing`. Only the latter is a fault. Name the two apart, report
    the object type, and compare the sentinel by identity rather than `==`,
    which a tensor-bearing object need not answer with a bool.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    1b83dc27
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-acestep.txt Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-latentsync.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-musetalk.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...