• Stefy Lanza (nextime / spora )'s avatar
    coderai-system: dedicated worker for cache scan / HF downloads / cache mgmt · b9bf7b18
    Stefy Lanza (nextime / spora ) authored
    Phase 1 of moving non-GPU work off the engine. The front now spawns and
    supervises a third process type — coderai-system — a lightweight (torch-free)
    worker that owns the slow, I/O-bound, GPU-irrelevant endpoints: cache scan
    (cached-models, cache-stats, cache delete), HF downloads (model-download,
    download-stream SSE, downloads, cancel), HF lookups (hf-search/files/model-info/
    model-files, ds4 defaults), and disk/bookkeeping (model-upload, model-free-disk,
    model-add-known, model-mark/unmark-download). The front routes those /admin/api
    paths to the worker instead of an engine, so they stay responsive during
    generation and survive engine restarts — fixing the models/archive-style hangs
    for the cache/download sections.
    
    Mechanics:
    - cli.py: --system-only flag. main.py: proc title coderai-system + run branch.
    - codai/system_app.py: slim FastAPI = admin_router only + session/config init +
      internal-auth gate + /healthz + /internal/engine-state + reload-config. Stays
      torch-free (the cache/download/HF handlers import no torch).
    - engine_supervisor: spawns one role="system" worker on the next internal port
      (_engine_cmd gains a mode arg), health-polls it, and excludes it from model
      assignment (it owns no models) while still queueing it model-json reloads.
    - registry: Engine.role; can_serve()==False for role="system" so it's never an
      inference target. Front excludes it from the engine tiles.
    - app.py: _SYSTEM_PATHS + _system_engine() + _proxy_passthrough() route the
      cache/download paths to the worker (long client handles JSON + SSE).
    - routes.get_current_user now validates ANY session/session_<port> cookie by
      signature, so front/engine/system (different ports) all accept one browser cookie.
    
    Engine still physically defines these handlers (now dead — front routes to the
    worker); stripping them is a later cleanup. Verified: slim app serves cache-stats
    with cross-port cookie + internal token (403 without), engine-state; front routing
    picks the worker and excludes it from inference/tiles.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    b9bf7b18
cli.py 11.8 KB