• Stefy Lanza (nextime / spora )'s avatar
    make the cache floor configurable, and widen the cluster page too (v0.4.79) · 21fccade
    Stefy Lanza (nextime / spora ) authored
    The two numbers that decide how much KV cache the share learner must leave an
    engine were constants in the module that uses them. They are now
    `vllm.kv_target_gb` and `vllm.kv_min_concurrency`, global or per model in its
    `vllm` block, on both the Settings page and the model's vLLM section — because
    the right answer differs per model by an order of magnitude: 2 GiB is correct
    for a page-at-a-time OCR model and leaves an LLM serving a 128k window unable
    to hold a single request. Setting the count to 0 opts out and sizes by GiB
    alone, which is what an OCR engine wants.
    
    The cluster page gets the same treatment the RunPod page just had: full-width
    wrapper and the dense table class, across all nine of its tables. The widest is
    ten columns, where the default 1rem of cell padding alone was 320px. Long
    values (a node URL, a capability list) now wrap inside their cell instead of
    widening the table past the window.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    21fccade
Name
Last commit
Last update
..
favicon.ico Loading commit data...
style.css Loading commit data...