• Stefy Lanza (nextime / spora )'s avatar
    front: serve settings (GET) locally; engine-freeze root-cause = GIL contention · f2cde28c
    Stefy Lanza (nextime / spora ) authored
    Root-cause finding (Path B): the engine event loop is genuinely freed during
    GGUF generation (work runs via asyncio.to_thread; llama_decode releases the GIL),
    so it's not a hard-blocked loop. But under CONTINUOUS generation the per-token
    Python work in llama-cpp-python keeps re-grabbing the GIL and starves the
    engine's own API handlers (sync ones especially) — an intrinsic ceiling that
    can't be cleanly removed while generating. So per "B first, A fallback", falling
    back to A: serve page data from the front (a separate process the GIL can't
    touch).
    
    This commit: /admin/api/settings GET is now served by the front. api_get_settings
    was refactored into a pure build_settings_dict(config, gpu_cards) shared by the
    engine handler and the front (front holds the same Config), so both return an
    identical payload. Front uses its torch-free gpu_detect.gpu_cards(). POST (save)
    still falls through to the engine, which persists + applies it.
    
    Verified: build_settings_dict standalone + front /admin/api/settings via TestClient.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    f2cde28c
app.py 64.2 KB