• Stefy Lanza (nextime / spora )'s avatar
    gpu: performance split strategy (fast card first) — per-model + global, UI-configurable · 64187292
    Stefy Lanza (nextime / spora ) authored
    The auto split is VRAM-proportional, which hands a big share to a slow second GPU
    (e.g. RX 580) so it bottlenecks throughput (Radeon ~100%, 3090 ~30%). Add a
    "performance" strategy that fills the fast lead card (main_gpu) first and spills
    only the overflow to the slower card(s), using the model's expected size — so the
    weak GPU holds the fewest layers (and none at all if the model fits on the lead
    card). "vram" (default) keeps the capacity-maximizing behavior.
    
    - config offload.split_strategy ("vram"|"performance") global default, persisted +
      pushed to live global_args; per-model split_strategy overrides it.
    - plumbed via build_runtime_kwargs + manager _cfg_or_global into vulkan auto-split.
    - model config modal: "Split strategy" select shown with the split options; global
      Settings: default split strategy select.
    - split_strategy added to the live-reload (_LOAD_AFFECTING) set so changing it
      re-applies on the next request without a restart.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    64187292
Name
Last commit
Last update
..
__init__.py Loading commit data...
base.py Loading commit data...
cuda.py Loading commit data...
ds4.py Loading commit data...
vulkan.py Loading commit data...