feat(node-agent): add computer_control desktop automation and capability-based tool discovery
Major integration of desktop automation and structured capability reporting for Hermes Node Gateway. This enables a unified node execution framework with optional tool support that is automatically detected and advertised. Features: - ComputerController class (Linux/X11) using xdotool and import (ImageMagick) - Screenshot (full screen, optional path or base64-return) - Mouse: move(x,y), click(button), position() - Keyboard: type_text(text), key_press(key like 'Return', 'Ctrl+c') - Window: active_window() returns focused window info - Capability-based registration to gateway - Agent sends tool list ["exec", "browser_control", "computer_control"] on connect - gateway filters tools based on declared capabilities - missing PC deps handled: checks for xdotool/import; browser extension presence - Universal installer updated: - Prompts: enable browser control? enable computer control? - Optional per-node sexec permissions quick-edit (allow/deny/ask comma patterns) - Writes config.json with enable_browser, enable_computer_control and permissions JSON - Fixes: - agent formerly advertised capability key 'browser'; gateway expects 'browser_control' → aligned - installers baked agent compressed with base64 in shell script, no external files Files: - node-agent/: new ComputerController class; enhanced hermes_node_agent.py - ~/.hermes/plugins/hermes-node-gateway/: added COMPUTER_CONTROL_SCHEMA + handlers + routing - install_hermes_node_universal.sh (merged in repo dist/ location) Node Agent endpoint type: "computer_control" commands (gateway forwards to node agent) Gateway: registers tool 'computer_control' with schema and executes cc_result responses handled Deployed nodes can now: - Execute shell commands via sexec (preserves existing allow/ask/deny) - Control browsers (if extension installed) - Control desktop (if X11 + xdotool + ImageMagick installed) Hermes Gateway plugin now exposes 4 tools: node_list, node_status, node_exec, browser_control, computer_control ✓ Co-authored-by:Lisa <lisa@nexlab.net> Signed-off-by:
Lisa <lisa@nexlab.net>
parents
Showing
BROWSER_PROTOCOL.md
0 → 100644
browser_control_example.py
0 → 100755
browser_controller.py
0 → 100644
browser_controller.py.base
0 → 100644
hermes-node-agent.init.d
0 → 100644
hermes-node-agent.service
0 → 100644
hermes_node_agent.py
0 → 100644
hermes_node_agent.py.bak
0 → 100644
hermes_node_agent.py.bak2
0 → 100644
install.sh
0 → 100755
requirements.txt
0 → 100644
| websockets>=16.0 | ||
| playwright>=1.59.0 |
test_browser.py
0 → 100644
Please
register
or
sign in
to comment