• Stefy Lanza (nextime / spora )'s avatar
    manager: thread the requested id through the TEXT load path too · 32c1d758
    Stefy Lanza (nextime / spora ) authored
    The previous two attempts wired request_model — which only video and image
    generation use. Chat completions go through get_model_for_request ->
    _load_model_by_name -> ensure_vram_for, and that path collapses the alias the same
    way, so on 0.1.93 the second lisa-32k request still logged
    
      [vram-est] key='…/gemma-4-26B-A4B-…gguf' source=measured(force) -> 51.97 GB
      Loading '…gguf': need 52.0 GB VRAM, have 4.6 GB free — evicting models
      On-request VRAM eviction: unloading active model 'lisa-32k' (measured 22.90 GB)
    
    get_model_for_request now keeps the requested id, passes it to
    _load_model_by_name, and from there to both ensure_vram_for and the
    expected_vram_gb the backend uses to size its GPU-layer split — so lisa-32k is
    sized as itself (22.9 GB) rather than as lisa (51.97 GB).
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
    32c1d758
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...