Commits · 8033f3ebafc8d73e2b575018f369a429ab8d881b · nexlab / coderai · GitLab

Sep 20, 2026
- cluster: mDNS discovery with a shared token, prefix-cache routing, Prometheus... · 0d4a338b
  Stefy Lanza (nextime / spora ) authored Sep 20, 2026
  0d4a338b
Sep 19, 2026
- cluster: nodes as engines, one GGUF over llama.cpp RPC, vLLM/SGLang... · f5ca2d82
  Stefy Lanza (nextime / spora ) authored Sep 19, 2026
  f5ca2d82
Sep 18, 2026
- runpod: drop a pod that refuses connections and retry once on a fresh one · 2315e84f
  Stefy Lanza (nextime / spora ) authored Sep 18, 2026
  2315e84f
Sep 17, 2026
- llama pod on the light core: the GGUF backend asks NVML about the card, not torch · 4866b69d
  Stefy Lanza (nextime / spora ) authored Sep 17, 2026
  4866b69d
- runpod: TLS on direct-TCP pods — both no-cutoff and encrypted · d83404ae
  Stefy Lanza (nextime / spora ) authored Sep 17, 2026
  d83404ae
Sep 14, 2026
- text: LoRA and QLoRA on text models, locally and on RunPod · 34d08890
  Stefy Lanza (nextime / spora ) authored Sep 14, 2026
  34d08890
- runpod: let several models share one pod until it saturates · 69d944a9
  Stefy Lanza (nextime / spora ) authored Sep 14, 2026
  69d944a9
Sep 13, 2026
- runpod: lock pods to a token, and scale them on real concurrency · 9ad9e19f
  Stefy Lanza (nextime / spora ) authored Sep 13, 2026
  9ad9e19f
- llm: make every text model remotizable, GGUF included · 6c6646c4
  Stefy Lanza (nextime / spora ) authored Sep 13, 2026
  6c6646c4
Sep 12, 2026
- gguf: stop reserving KV headroom that a measured footprint already includes · 2511b348
  Stefy Lanza (nextime / spora ) authored Sep 12, 2026
  2511b348
- gguf: never fall back to a 2048 context · 0ba09055
  Stefy Lanza (nextime / spora ) authored Sep 12, 2026
  0ba09055
Sep 11, 2026
- gguf: resolve split-GGUF shards and say what's missing · 12887a46
  Stefy Lanza (nextime / spora ) authored Sep 11, 2026
  12887a46
- gguf: try swa_full before quantizing, and add a finer KV ladder · 5eb92e2c
  Stefy Lanza (nextime / spora ) authored Sep 11, 2026
  5eb92e2c
- gguf: plan where the KV cache goes instead of discovering it by OOM · f440cdbb
  Stefy Lanza (nextime / spora ) authored Sep 11, 2026
  f440cdbb
Sep 06, 2026
- runpod: fix pods /v1 path; add stale-pod reaper, multi-GPU fallback, pod logs · b911310a
  Stefy Lanza (nextime / spora ) authored Sep 06, 2026
  b911310a
- runpod(phase3-5): pod pool + scaler + cost ledger, budgets, global-gate bypass, spillover, stats UI · 7649c9c2
  Stefy Lanza (nextime / spora ) authored Sep 06, 2026
  7649c9c2
- runpod(phase2): serverless request path + per-model config + no-local-download · 17604ab4
  Stefy Lanza (nextime / spora ) authored Sep 06, 2026
  17604ab4
Aug 21, 2026
- vllm: accept HF repo ids (not just local dirs) in model resolution · 714f66d0
  Stefy Lanza (nextime / spora ) authored Aug 21, 2026
  714f66d0
- vllm: serve models from the model list (nvidia/radeon-style), model_id optional · 7dee70a4
  Stefy Lanza (nextime / spora ) authored Aug 21, 2026
  7dee70a4
- Add vLLM as a first-class managed engine backend (LLM + OCR Surya2) · a5d9fe82
  Stefy Lanza (nextime / spora ) authored Aug 21, 2026
  a5d9fe82
Aug 16, 2026
- Add dedicated OCR subsystem + finalize multi-engine backends · 2485403f
  Stefy Lanza (nextime / spora ) authored Aug 16, 2026
  2485403f
Jul 29, 2026
- colibri: stop at GLM turn boundary + clean output + resolve model_id alias · 58f945de
  Stefy Lanza (nextime / spora ) authored Jul 29, 2026
  58f945de
Jul 28, 2026
- colibri: download the GLM-5.2 container from the Models interface like any other · 1b7187d4
  Stefy Lanza (nextime / spora ) authored Jul 28, 2026
  1b7187d4
- colibri: integrate GLM-5.2 native C engine (driven directly, no colibri Python) · 50f54eb3
  Stefy Lanza (nextime / spora ) authored Jul 28, 2026
  50f54eb3
Jul 22, 2026
- vulkan: vision requests must bypass the fast chat-render path · 40635282
  Stefy Lanza (nextime / spora ) authored Jul 22, 2026
  40635282
Jun 24, 2026
- nvidia HF loader: parity with hf_loading (precision, flash key, buffers, RAM cap) · 8c1ea33d
  Stefy Lanza (nextime / spora ) authored Jun 24, 2026
  8c1ea33d
- hf: resolve relative offload_dir to the configured dir; forward HF_TOKEN · fe7da690
  Stefy Lanza (nextime / spora ) authored Jun 24, 2026
  fe7da690
Jun 22, 2026
- engine: render chat prompt with a plain (cached) Jinja env — fixes ~40s per request · 080ecff7
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  080ecff7
- engine: time-split chat generation (build vs prefill) to locate pre-gen latency · e501bab0
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  e501bab0
- engine: route hot-path logs through os.write (fast_print) instead of print() · e2db1b5c
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  e2db1b5c
- vulkan: add gemma stop sequences so generation halts after the tool call · ddb48b2d
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  ddb48b2d
- gpu: performance split respects per-card VRAM cap; harden process naming · 5b4ff469
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  5b4ff469
- gpu: global VRAM cap is now per physical card (nvidia/radeon), auto-enumerated · f416117a
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  f416117a
- gpu: secondary-card VRAM cap (per-model + global), effective = lowest of the two · 82aec0f9
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  82aec0f9
- gpu: performance-split KV-aware fit + progressive CPU-offload retry on OOM · f1f11723
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  f1f11723
- gpu: performance split strategy (fast card first) — per-model + global, UI-configurable · 64187292
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  64187292
- front/engine: responsive Tasks under load — front-side in-flight tracking + thermal cadence · afefee53
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  afefee53
- gpu: exclude llvmpipe from split, mmproj-aware headroom + CPU spill when pool is full · b5acbc2c
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  b5acbc2c
- ui/gpu: explicit "Split — <card> first" options; auto = single GPU; confine non-split models · 0490e2c9
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  0490e2c9
Jun 21, 2026
- ui: cross-GPU split controls (global + per-model) with auto VRAM-proportional ratio · c060e6dc
  Stefy Lanza (nextime / spora ) authored Jun 22, 2026
  c060e6dc