• Stefy Lanza (nextime / spora )'s avatar
    whisper: account a running runner as a loaded model for VRAM eviction · 2a214215
    Stefy Lanza (nextime / spora ) authored
    Starting a whisper-server runner loads the gguf onto the GPU, but it was
    invisible to the VRAM-eviction logic — it never evicted others to make room,
    recorded no footprint, and (lacking a cleanup()) couldn't itself be evicted.
    
    - WhisperServerManager.cleanup() -> stop(), so _evict_one/unload_model can
      free its VRAM like any other model.
    - MultiModelManager.start_whisper_server(): estimate the gguf footprint, evict
      other models if free VRAM is short, start the subprocess, and register it in
      models/models_in_vram/_measured_vram_gb (active_in_vram). It's now both a
      trigger for eviction and an eviction candidate.
    - stop_whisper_server(): stop + clear all that accounting (frees VRAM).
    - Routed every start/stop through these: on-request transcription, engine
      startup pre-load, admin model-load (Load button) and model-unload/disable.
    
    So: starting a runner = a model load (evicts as needed); unloading = frees VRAM.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    2a214215
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...