-
Stefy Lanza (nextime / spora ) authored
manager (record_vram_delta): record the FULL footprint after a load — including offloaded loads (no longer skipped) — so the next load skips the OOM→move-to-RAM retry dance: * measured_vram_gb — GPU VRAM delta + reserve (GPU portion; eviction uses it) * measured_ram_gb — host-RAM delta (the CPU-offloaded layers) [new] * measured_n_gpu_layers — the layer split llama.cpp settled on [new] total need = vram + ram. All gated by force_vram_update (else only when used_vram_gb unset). New _learned_n_gpu_layers(): when n_gpu_layers is auto/-1, reuse the learned split so a reload jumps straight to the config that fit. Both load sites snapshot host RAM and read the backend's settled n_gpu_layers. frontproxy: * graceful streaming proxy: when an engine dies mid-stream (e.g. a CUDA ggml_abort SIGABRT on a VRAM-OOM decode), end the relayed stream cleanly instead of throwing an unhandled ASGI exception (noisy traceback). * task actions (cancel/interrupt/pause/resume/restart/DELETE) now route to the engine that OWNS the task — fan out to every live engine (each validates the same session cookie), first success wins — fixing 'Task not found' for tasks on a non-primary engine; front-only synthetic ids handled locally. Bump version to 0.1.7.223b46f7