config: per-engine env overrides (RADV driver tuning for Polaris hangs)

New server.engine_env_overrides (engine name → {VAR: val}), merged into
the engine's process env at spawn. Lets low-level driver knobs be set
without hardcoding — set radeon → RADV_DEBUG=syncshaders to serialize
RADV shader dispatch and test whether the Polaris compute-ring async
race (ring comp_x.y.z timeout) is what wedges the RX 580 under sustained
Vulkan compute.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
parent 09558635
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API # Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here. # metadata and the admin web UI read from here.
__version__ = "0.1.55" __version__ = "0.1.56"
# Configure the CUDA caching allocator BEFORE torch is imported anywhere. # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
# expandable_segments lets the allocator return freed pages to the driver even # expandable_segments lets the allocator return freed pages to the driver even
......
...@@ -47,6 +47,11 @@ class ServerConfig: ...@@ -47,6 +47,11 @@ class ServerConfig:
# Levels: auto | low | high | manual | profile_standard | profile_peak. # Levels: auto | low | high | manual | profile_standard | profile_peak.
# Best-effort — an unprivileged container logs the manual host command. # Best-effort — an unprivileged container logs the manual host command.
dpm_force_performance_level_overrides: dict = field(default_factory=dict) dpm_force_performance_level_overrides: dict = field(default_factory=dict)
# Extra environment variables injected per engine (engine name → {VAR: val}),
# merged into that engine's process env at spawn. For low-level driver tuning
# without hardcoding — e.g. {"radeon": {"RADV_DEBUG": "syncshaders"}} to
# serialize RADV shader dispatch and avoid Polaris async compute-ring hangs.
engine_env_overrides: dict = field(default_factory=dict)
# ─── Frontend/engine split ─────────────────────────────────────────────── # ─── Frontend/engine split ───────────────────────────────────────────────
# coderai boots a thin, always-responsive *front* reverse proxy on the public # coderai boots a thin, always-responsive *front* reverse proxy on the public
# host/port and supervises one or more *engine* subprocesses (which do all # host/port and supervises one or more *engine* subprocesses (which do all
......
...@@ -347,6 +347,19 @@ class EngineSupervisor: ...@@ -347,6 +347,19 @@ class EngineSupervisor:
# values are honoured (e.g. CUDA_VISIBLE_DEVICES="" hides all CUDA cards). # values are honoured (e.g. CUDA_VISIBLE_DEVICES="" hides all CUDA cards).
for k, v in (engine.env or {}).items(): for k, v in (engine.env or {}).items():
env[str(k)] = str(v) env[str(k)] = str(v)
# Config-driven per-engine env overrides (low-level driver tuning, e.g.
# RADV_DEBUG=syncshaders on the radeon). Applied AFTER the auto-detected
# block so the user's value wins.
try:
_eov = (getattr(self.config.server, "engine_env_overrides", None)
or {}).get(engine.name) or {}
for k, v in _eov.items():
env[str(k)] = str(v)
if _eov:
print(f"[front] {engine.name}: env overrides "
f"{ {k: v for k, v in _eov.items()} }", flush=True)
except Exception:
pass
# The global host-RAM cap (offload.max_ram_gb) is SHARED across all engines, # The global host-RAM cap (offload.max_ram_gb) is SHARED across all engines,
# not split: tell each engine the front's PID so it measures the whole # not split: tell each engine the front's PID so it measures the whole
# fleet's RAM (front + every engine + workers) against the one cap. # fleet's RAM (front + every engine + workers) against the one cap.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment