• Lisa's avatar
    feat(node-agent): add computer_control desktop automation and capability-based tool discovery · 8479b9e3
    Lisa authored
    Major integration of desktop automation and structured capability reporting for Hermes Node Gateway.
    This enables a unified node execution framework with optional tool support that is
    automatically detected and advertised.
    
    Features:
    - ComputerController class (Linux/X11) using xdotool and import (ImageMagick)
      - Screenshot (full screen, optional path or base64-return)
      - Mouse: move(x,y), click(button), position()
      - Keyboard: type_text(text), key_press(key like 'Return', 'Ctrl+c')
      - Window: active_window() returns focused window info
    - Capability-based registration to gateway
      - Agent sends tool list ["exec", "browser_control", "computer_control"] on connect
      - gateway filters tools based on declared capabilities
      - missing PC deps handled: checks for xdotool/import; browser extension presence
    - Universal installer updated:
      - Prompts: enable browser control? enable computer control?
      - Optional per-node sexec permissions quick-edit (allow/deny/ask comma patterns)
      - Writes config.json with enable_browser, enable_computer_control and permissions JSON
    - Fixes:
      - agent formerly advertised capability key 'browser'; gateway expects 'browser_control' → aligned
      - installers baked agent compressed with base64 in shell script, no external files
    
    Files:
    - node-agent/: new ComputerController class; enhanced hermes_node_agent.py
    - ~/.hermes/plugins/hermes-node-gateway/: added COMPUTER_CONTROL_SCHEMA + handlers + routing
    - install_hermes_node_universal.sh (merged in repo dist/ location)
    
    Node Agent endpoint type: "computer_control" commands (gateway forwards to node agent)
    Gateway: registers tool 'computer_control' with schema and executes cc_result responses handled
    
    Deployed nodes can now:
      - Execute shell commands via sexec (preserves existing allow/ask/deny)
      - Control browsers (if extension installed)
      - Control desktop (if X11 + xdotool + ImageMagick installed)
    
    Hermes Gateway plugin now exposes 4 tools:
      node_list, node_status, node_exec, browser_control, computer_control ✓
    Co-authored-by: 's avatarLisa <lisa@nexlab.net>
    Signed-off-by: 's avatarLisa <lisa@nexlab.net>
    8479b9e3
manifest.json 764 Bytes