• Stefy Lanza (nextime / spora )'s avatar
    fix(ds4+config): resolve bare model ids, don't over-estimate VRAM, robust config · 8c85e16a
    Stefy Lanza (nextime / spora ) authored
    - ds4: resolve a bare/aliased model id (e.g. "Foo-ds4-Q2_K", no path/extension) to
      its configured .gguf via a config/cache-aware resolver — fixes the 503 ("no local
      deepseek4 GGUF resolved") on chat requests (only "Load now" with a full path
      worked before). Ds4Backend reuses the same resolver.
    - ds4: report a modest VRAM footprint for ds4 models (measured or ~12GB) instead of
      the 100GB+ GGUF size — ds4-server streams experts from SSD and manages its own
      memory, so the old estimate forced needless ~128GB eviction churn every request.
    - ds4: route on-disk KV checkpoints into coderai's offload directory by default
      (--kv-disk-dir <offload>/ds4-kv) unless overridden in extra_args.
    - config: tolerant load (_dc drops unknown keys) so a stale/newer config.json never
      crashes the whole load and silently resets ALL settings to defaults (the "had to
      reconfigure everything" bug). save_config + GET/POST settings carry the new ds4
      fields (model_path, auto_download, ssd_streaming).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    8c85e16a
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...