• Stefy Lanza (nextime / spora )'s avatar
    gpu: secondary-card VRAM cap (per-model + global), effective = lowest of the two · 82aec0f9
    Stefy Lanza (nextime / spora ) authored
    Add a cap on how much VRAM the auto-split may place on EACH secondary (non-main)
    card, so a slow second GPU (e.g. RX 580) stays lightly loaded and bottlenecks
    throughput less — the remainder stays on the fast card or spills to CPU.
    
    - config offload.split_secondary_cap_gb: global default, persisted + pushed live to
      every engine's global_args (each engine reads config at startup, so it's global).
    - per-model split_secondary_cap_gb overrides — but only to TIGHTEN: the effective
      cap is min(global, per-model) of whichever are set; a per-model value higher than
      the global is ignored.
    - applied in vulkan auto-split: caps each non-main device's free VRAM in the ratio
      (both vram & performance strategies), and in the auto-offload pool so the excess
      proactively spills to CPU instead of only reacting to an OOM.
    - UI: model config modal "Secondary card VRAM cap" + global Settings field.
    - added to _LOAD_AFFECTING so changes re-apply live on the next request.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    82aec0f9
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...