-
Lisa authored
Major integration of desktop automation and structured capability reporting for Hermes Node Gateway. This enables a unified node execution framework with optional tool support that is automatically detected and advertised. Features: - ComputerController class (Linux/X11) using xdotool and import (ImageMagick) - Screenshot (full screen, optional path or base64-return) - Mouse: move(x,y), click(button), position() - Keyboard: type_text(text), key_press(key like 'Return', 'Ctrl+c') - Window: active_window() returns focused window info - Capability-based registration to gateway - Agent sends tool list ["exec", "browser_control", "computer_control"] on connect - gateway filters tools based on declared capabilities - missing PC deps handled: checks for xdotool/import; browser extension presence - Universal installer updated: - Prompts: enable browser control? enable computer control? - Optional per-node sexec permissions quick-edit (allow/deny/ask comma patterns) - Writes config.json with enable_browser, enable_computer_control and permissions JSON - Fixes: - agent formerly advertised capability key 'browser'; gateway expects 'browser_control' → aligned - installers baked agent compressed with base64 in shell script, no external files Files: - node-agent/: new ComputerController class; enhanced hermes_node_agent.py - ~/.hermes/plugins/hermes-node-gateway/: added COMPUTER_CONTROL_SCHEMA + handlers + routing - install_hermes_node_universal.sh (merged in repo dist/ location) Node Agent endpoint type: "computer_control" commands (gateway forwards to node agent) Gateway: registers tool 'computer_control' with schema and executes cc_result responses handled Deployed nodes can now: - Execute shell commands via sexec (preserves existing allow/ask/deny) - Control browsers (if extension installed) - Control desktop (if X11 + xdotool + ImageMagick installed) Hermes Gateway plugin now exposes 4 tools: node_list, node_status, node_exec, browser_control, computer_control ✓ Co-authored-by:
Lisa <lisa@nexlab.net> Signed-off-by:
Lisa <lisa@nexlab.net>
29a308f6
This project manages its dependencies using
pip.
Learn more