• Stefy Lanza (nextime / spora )'s avatar
    front: serve settings (GET) locally; engine-freeze root-cause = GIL contention · f2cde28c
    Stefy Lanza (nextime / spora ) authored
    Root-cause finding (Path B): the engine event loop is genuinely freed during
    GGUF generation (work runs via asyncio.to_thread; llama_decode releases the GIL),
    so it's not a hard-blocked loop. But under CONTINUOUS generation the per-token
    Python work in llama-cpp-python keeps re-grabbing the GIL and starves the
    engine's own API handlers (sync ones especially) — an intrinsic ceiling that
    can't be cleanly removed while generating. So per "B first, A fallback", falling
    back to A: serve page data from the front (a separate process the GIL can't
    touch).
    
    This commit: /admin/api/settings GET is now served by the front. api_get_settings
    was refactored into a pure build_settings_dict(config, gpu_cards) shared by the
    engine handler and the front (front holds the same Config), so both return an
    identical payload. Front uses its torch-free gpu_detect.gpu_cards(). POST (save)
    still falls through to the engine, which persists + applies it.
    
    Verified: build_settings_dict standalone + front /admin/api/settings via TestClient.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_011DDv7BchtZQWsnPG6Jm49m
    f2cde28c
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...