• Stefy Lanza (nextime / spora )'s avatar
    frontproxy: a wedged GPU can no longer hang the whole front · 09558635
    Stefy Lanza (nextime / spora ) authored
    At front startup _build_engines runs vulkaninfo to enumerate GPUs; on a
    wedged Polaris card that process blocks UNINTERRUPTIBLY (D-state), and
    subprocess.run(timeout=) can't kill a D-state child — so the front's
    uvicorn startup hung forever ("Waiting for application startup"),
    taking down the healthy NVIDIA engine too (502 everywhere).
    
    Add _call_with_timeout: runs the probe in a daemon thread and STOPS
    WAITING after a wall-clock timeout, leaving the un-killable call
    orphaned instead of blocking. Wrap vulkan_devices() (12s) so startup
    always completes, and _amd_stats() (6s) so a hung Radeon can't freeze
    the health/thermal poll threads that also monitor the healthy card.
    A wedged engine now degrades to "that engine down", not "everything down".
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
    09558635
Name
Last commit
Last update
..
admin Loading commit data...
api Loading commit data...
backends Loading commit data...
broker Loading commit data...
frontproxy Loading commit data...
models Loading commit data...
openai Loading commit data...
pydantic Loading commit data...
queue Loading commit data...
tasks Loading commit data...
__init__.py Loading commit data...
cli.py Loading commit data...
config.py Loading commit data...
main.py Loading commit data...
platform_paths.py Loading commit data...
system_app.py Loading commit data...