• Stefy Lanza (nextime / spora )'s avatar
    ui: cross-GPU split controls (global + per-model) with auto VRAM-proportional ratio · c060e6dc
    Stefy Lanza (nextime / spora ) authored
    Per-model (model config modal): the "Engine / card" selector gains an "All GPUs
    (split across every backend)" option. Choosing it sets gpu_split and reveals a
    "Weight distribution" field (tensor_split, e.g. 0.8,0.2); blank = auto. A real
    engine name still pins as before.
    
    Global (Settings): "Split models across all GPUs" checkbox + default weight
    distribution field, persisted to offload.gpu_split / offload.tensor_split and
    pushed to live global_args. Per-model "All GPUs" overrides the global default.
    
    api: settings GET/POST now expose+accept offload.gpu_split / tensor_split.
    
    backend: when gpu_split is on and no ratio is given, auto-derive tensor_split
    proportional to each device's FREE VRAM (new _per_device_free_vram_gb, llama.cpp
    device order: CUDA first, then Vulkan/AMD) — e.g. 24 GB 3090 + 8 GB RX 580 ->
    ~0.75/0.25 — so the bigger card carries more without the user computing it.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    c060e6dc
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...