• Stefy Lanza (nextime / spora )'s avatar
    front: cut engine load — front-native loaded-status, config reads to system worker · 858d393a
    Stefy Lanza (nextime / spora ) authored
    Fixes the models page's 6-7s stall and reduces engine hits generally:
    
    - loadCachedModels awaited /admin/api/models (engine) before painting the cached-
      models cards, so the fast list waited on the slow engine. /admin/api/models (and
      accel-presets/accel-loras/turboquant-info/model-enable/model-disable) are pure
      config reads/edits, so they now route to the coderai-system worker (it has
      config_manager + models.json) — fast, off the engine.
    
    - model-loaded-status is now served from the FRONT's own state: loaded comes from
      the registry (the front issues every load/unload; the supervisor keeps each
      engine's resident set fresh via the cheap engine-state poll) and running from the
      front's in-flight tracking. Per-model instance detail is synced from the engine
      ONLY when it's idle (inflight==0) and cached otherwise — no engine hit during
      generation. This is the "front maintains its own list, syncs when not under load"
      model.
    
    - The 1s Tasks poll now skips the engine entirely when the primary is mid-
      generation, serving the merged/synthesized list from cache + live in-flight
      tracking instead of burning the timeout.
    
    Verified: model-loaded-status returns front-native loaded/running with no engine
    call while busy; compiles.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    858d393a
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...