• Stefy Lanza (nextime / spora )'s avatar
    ds4: configurable CUDA env knobs (expert-cache reserve + free-form extra_env) · 7fc393d4
    Stefy Lanza (nextime / spora ) authored
    ds4-server exposes several CUDA tunables only via environment, not CLI flags.
    By default ds4 reserves half the card for non-cache use and allocates the model
    weight arena in 1792 MiB chunks — both starve / OOM the streaming expert cache
    on small-weight MoE models served from SSD.
    
    Pass an explicit env to ds4-server (Popen now sets env=) with:
      - expert_cache_reserve_gb: typed knob -> DS4_CUDA_STREAMING_EXPERT_CACHE_RESERVE_GB
        (0 = leave ds4's default).
      - extra_env: free-form KEY=VALUE passthrough for the rest, e.g.
        DS4_CUDA_WEIGHT_ARENA_CHUNK_MB=512 to shrink the weight-arena chunk so it
        fits a heap fragmented by the expert cache.
    
    Both surfaced in Settings (config + admin GET/POST + UI), default to no-op so
    behaviour is unchanged unless set.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    7fc393d4
Name
Last commit
Last update
..
__init__.py Loading commit data...
_film_net.py Loading commit data...
_rife_ifnet.py Loading commit data...
app.py Loading commit data...
archive.py Loading commit data...
audio_backends.py Loading commit data...
audio_clean.py Loading commit data...
audio_gen.py Loading commit data...
audio_stems.py Loading commit data...
characters.py Loading commit data...
custom_pipelines.py Loading commit data...
ds4_worker.py Loading commit data...
embeddings.py Loading commit data...
environments.py Loading commit data...
faceswap.py Loading commit data...
images.py Loading commit data...
log.py Loading commit data...
loras.py Loading commit data...
parler_worker.py Loading commit data...
pipelines.py Loading commit data...
prompt_cache.py Loading commit data...
ratelimit.py Loading commit data...
spatial.py Loading commit data...
state.py Loading commit data...
text.py Loading commit data...
transcriptions.py Loading commit data...
tts.py Loading commit data...
tts_backends.py Loading commit data...
urlutils.py Loading commit data...
video.py Loading commit data...
voice_clone.py Loading commit data...
voice_convert.py Loading commit data...