1. 30 Apr, 2026 1 commit
    • Lisa's avatar
      feat(node-agent): add computer_control desktop automation and capability-based tool discovery · 29a308f6
      Lisa authored
      Major integration of desktop automation and structured capability reporting for Hermes Node Gateway.
      This enables a unified node execution framework with optional tool support that is
      automatically detected and advertised.
      
      Features:
      - ComputerController class (Linux/X11) using xdotool and import (ImageMagick)
        - Screenshot (full screen, optional path or base64-return)
        - Mouse: move(x,y), click(button), position()
        - Keyboard: type_text(text), key_press(key like 'Return', 'Ctrl+c')
        - Window: active_window() returns focused window info
      - Capability-based registration to gateway
        - Agent sends tool list ["exec", "browser_control", "computer_control"] on connect
        - gateway filters tools based on declared capabilities
        - missing PC deps handled: checks for xdotool/import; browser extension presence
      - Universal installer updated:
        - Prompts: enable browser control? enable computer control?
        - Optional per-node sexec permissions quick-edit (allow/deny/ask comma patterns)
        - Writes config.json with enable_browser, enable_computer_control and permissions JSON
      - Fixes:
        - agent formerly advertised capability key 'browser'; gateway expects 'browser_control' → aligned
        - installers baked agent compressed with base64 in shell script, no external files
      
      Files:
      - node-agent/: new ComputerController class; enhanced hermes_node_agent.py
      - ~/.hermes/plugins/hermes-node-gateway/: added COMPUTER_CONTROL_SCHEMA + handlers + routing
      - install_hermes_node_universal.sh (merged in repo dist/ location)
      
      Node Agent endpoint type: "computer_control" commands (gateway forwards to node agent)
      Gateway: registers tool 'computer_control' with schema and executes cc_result responses handled
      
      Deployed nodes can now:
        - Execute shell commands via sexec (preserves existing allow/ask/deny)
        - Control browsers (if extension installed)
        - Control desktop (if X11 + xdotool + ImageMagick installed)
      
      Hermes Gateway plugin now exposes 4 tools:
        node_list, node_status, node_exec, browser_control, computer_control ✓
      Co-authored-by: 's avatarLisa <lisa@nexlab.net>
      Signed-off-by: 's avatarLisa <lisa@nexlab.net>
      29a308f6