• Stefy Lanza (nextime / spora )'s avatar
    coderai-system: dedicated worker for cache scan / HF downloads / cache mgmt · b9bf7b18
    Stefy Lanza (nextime / spora ) authored
    Phase 1 of moving non-GPU work off the engine. The front now spawns and
    supervises a third process type — coderai-system — a lightweight (torch-free)
    worker that owns the slow, I/O-bound, GPU-irrelevant endpoints: cache scan
    (cached-models, cache-stats, cache delete), HF downloads (model-download,
    download-stream SSE, downloads, cancel), HF lookups (hf-search/files/model-info/
    model-files, ds4 defaults), and disk/bookkeeping (model-upload, model-free-disk,
    model-add-known, model-mark/unmark-download). The front routes those /admin/api
    paths to the worker instead of an engine, so they stay responsive during
    generation and survive engine restarts — fixing the models/archive-style hangs
    for the cache/download sections.
    
    Mechanics:
    - cli.py: --system-only flag. main.py: proc title coderai-system + run branch.
    - codai/system_app.py: slim FastAPI = admin_router only + session/config init +
      internal-auth gate + /healthz + /internal/engine-state + reload-config. Stays
      torch-free (the cache/download/HF handlers import no torch).
    - engine_supervisor: spawns one role="system" worker on the next internal port
      (_engine_cmd gains a mode arg), health-polls it, and excludes it from model
      assignment (it owns no models) while still queueing it model-json reloads.
    - registry: Engine.role; can_serve()==False for role="system" so it's never an
      inference target. Front excludes it from the engine tiles.
    - app.py: _SYSTEM_PATHS + _system_engine() + _proxy_passthrough() route the
      cache/download paths to the worker (long client handles JSON + SSE).
    - routes.get_current_user now validates ANY session/session_<port> cookie by
      signature, so front/engine/system (different ports) all accept one browser cookie.
    
    Engine still physically defines these handlers (now dead — front routes to the
    worker); stripping them is a later cleanup. Verified: slim app serves cache-stats
    with cross-port cookie + internal token (403 without), engine-state; front routing
    picks the worker and excludes it from inference/tiles.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    b9bf7b18
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...