• Stefy Lanza (nextime / spora )'s avatar
    ui: cross-GPU split controls (global + per-model) with auto VRAM-proportional ratio · c060e6dc
    Stefy Lanza (nextime / spora ) authored
    Per-model (model config modal): the "Engine / card" selector gains an "All GPUs
    (split across every backend)" option. Choosing it sets gpu_split and reveals a
    "Weight distribution" field (tensor_split, e.g. 0.8,0.2); blank = auto. A real
    engine name still pins as before.
    
    Global (Settings): "Split models across all GPUs" checkbox + default weight
    distribution field, persisted to offload.gpu_split / offload.tensor_split and
    pushed to live global_args. Per-model "All GPUs" overrides the global default.
    
    api: settings GET/POST now expose+accept offload.gpu_split / tensor_split.
    
    backend: when gpu_split is on and no ratio is given, auto-derive tensor_split
    proportional to each device's FREE VRAM (new _per_device_free_vram_gb, llama.cpp
    device order: CUDA first, then Vulkan/AMD) — e.g. 24 GB 3090 + 8 GB RX 580 ->
    ~0.75/0.25 — so the bigger card carries more without the user computing it.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    c060e6dc
Name
Last commit
Last update
..
__init__.py Loading commit data...
base.py Loading commit data...
cuda.py Loading commit data...
ds4.py Loading commit data...
vulkan.py Loading commit data...