# Hermes Node Android

<p align="center">
  <img src="assets/branding/logo-readme.png" alt="Hermes Node Android logo" width="320" />
</p>

Android node agent for the Hermes Node Protocol.

This project is the Android sibling of the Linux `hermes-node-agent`. It reuses existing Hermes Node Gateway capability names where Android can provide equivalent intent, instead of inventing a separate `phone_control` gateway tool.

Remote:

```text
git@git.nexlab.net:lisa/hermes-node-android.git
```

## Current status

Working Android node APK that connects to the Hermes Node Gateway and implements the honest stock-Android subset of the Hermes node capability model.

Implemented:

- Android/Kotlin app project with Gradle wrapper.
- Config UI for gateway URL, token, node name, TLS mode, and advertised capabilities.
- Foreground service for persistent outbound WebSocket connection.
- Registration frame compatible with Hermes Node Gateway-style capability discovery.
- Reconnect loop and heartbeat frames.
- App-local config via `SharedPreferences`; no hardcoded token.
- Status display in the app.
- Permission/setup helpers in the UI for camera, microphone, Accessibility, and screenshot consent.
- Debug APK build verified locally.

Implemented capability handlers:

- `computer_control`
  - `launch_app`: implemented via package launch intent.
  - `open_url`: implemented via `ACTION_VIEW`.
  - `open_intent`: implemented for implicit Android intents.
  - `tap`, `swipe`, `long_press`: implemented via AccessibilityService gestures.
  - `type_text`: implemented via Accessibility focused-node text injection.
  - `key_press`: implemented for Android global actions like `back`, `home`, `recents`, `notifications`, `quick_settings`.
- `desktop_observe`
  - `screen_info`: implemented.
  - `active_window`: implemented when AccessibilityService is enabled.
  - `clipboard_get`: implemented.
  - cursor/mouse-position compatibility returns Android-specific note.
  - `list_windows` and `window_geometry`: compatibility implementations.
  - `screenshot` and `region_screenshot`: implemented via MediaProjection after explicit in-app user consent.
- `audio_control`
  - `get_audio_status`: implemented.
  - `list_audio_devices`: implemented via `AudioDeviceInfo`.
  - `capture_input`: implemented via `AudioRecord`, returning WAV as base64.
  - `play_audio`: implemented via `MediaPlayer` from either a local path or a base64 payload.
  - `capture_output`: explicitly unsupported on stock Android.
- `camera_control`
  - `list_cameras`: implemented via Camera2.
  - `get_camera_status`: implemented.
  - `capture_frame`: implemented via Camera2 + `ImageReader`, returning base64 JPEG.
  - `capture_video`: implemented as a visible foreground recording flow using `VideoCaptureActivity` plus service handoff; no hidden recording.
  - `inject_video`: explicitly unsupported on stock Android.
- Android-specific extensions
  - Notification reading: implemented via `NotificationListenerService`, with explicit user-granted access and honest limits.
  - Directory listing and file transfer: still planned via app-private storage plus user-granted Storage Access Framework roots; no fake arbitrary filesystem access.
- `notification_access`
  - `get_notification_status`: implemented.
  - `list_notifications`: implemented.
  - `open_notification`: implemented with conservative package/app fallback behavior.
  - `dismiss_notification`: implemented as best-effort.
- `browser_control`
  - `launch`, `navigate`, `open_url`: implemented via Android browser intents.
  - `screenshot`: implemented via the same MediaProjection capture backend used by `desktop_observe`.
  - DOM/CDP/playwright-style automation remains unsupported on stock Android.
- `exec`
  - advertised only if enabled in UI, but backend intentionally returns unsupported for now.

## Important architecture truth

This project now implements the *whole honest stock-Android implementation*.

That does **not** mean Android can do everything Linux can do. Some actions are fundamentally outside what an ordinary non-rooted Android app should pretend to support.

Still intentionally unsupported because of real platform boundaries:

- `audio_control.capture_output`: unavailable to normal stock Android apps.
- `camera_control.capture_video`: not implemented in the service backend because a correct/defensible stock-Android version needs a visible recording UX/activity pipeline, not a stealth recorder hidden behind the node protocol.
- `camera_control.inject_video`: not a normal stock-Android capability without rooted/vendor-specific backends.
- `browser_control` DOM/CDP/playwright-style automation: unsupported on stock Android; this backend is intent-based plus screenshot fallback.
- `exec`: intentionally disabled as non-parity Android shell execution.

So the implementation is complete in the sense that:
- every advertised capability now has a real backend or an explicit platform-boundary refusal,
- screenshot capture is no longer fake,
- browser_control is no longer just a placeholder,
- and the remaining gaps are architectural truths, not forgotten TODOs.

## Capability naming decision

Use existing gateway capability names where practical:

- `computer_control` = interactive device control.
  - Linux: mouse, keyboard, X11 desktop actions.
  - Android: app launch, URL/intent dispatch, touch/input abstractions, future AccessibilityService gestures.
- `desktop_observe` = structured screen/device observation.
  - Linux: active window, cursor, screen geometry, screenshots.
  - Android: display metrics, orientation, interactive state, active-window accessibility info, and MediaProjection screenshots.
- `audio_control` = device/media audio operations.
- `camera_control` = camera read paths where supported.
- `browser_control` = Android intent-based browser launch/navigation plus screenshot fallback.
- `exec` = Android shell execution if ever explicitly designed; disabled by default.

The gateway can inspect `capability_info.platform == "android"` and `platform_semantics == "android_phone_control"` for platform-specific hints while keeping the public tool names stable.

## Build

Prerequisites:

- JDK 17
- Android SDK with platform 35 available

Build debug APK:

```bash
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 \
ANDROID_HOME=/usr/lib/android-sdk \
ANDROID_SDK_ROOT=/usr/lib/android-sdk \
./gradlew --no-daemon assembleDebug
```

Output:

```text
app/build/outputs/apk/debug/app-debug.apk
```

## Install/test on Android phone

1. Copy/install the debug APK:

   ```bash
   adb install -r app/build/outputs/apk/debug/app-debug.apk
   ```

2. Open **Hermes Node** on the phone.
3. Fill in:
   - Gateway URL: `wss://<gateway-host>:8765`
   - Node name: e.g. `phone`
   - Gateway token: token from `~/.config/hermes-node-gateway/config.json`
   - Leave “Allow self-signed/untrusted gateway TLS” enabled if using the current self-signed gateway cert setup.
4. Tap **Save & start node**.
5. Android should show a persistent foreground-service notification.
6. On Hermes, check `node_list` / gateway node status for the Android node name.
7. In the Android app, also tap:
   - **Grant camera/microphone permissions**
   - **Open Accessibility settings** and enable Hermes Node if you want gestures/active window support
   - **Grant screenshot consent** if you want screenshot/browser screenshot support
8. Smoke-test actions from Hermes:
   - `desktop_observe` → `screen_info`
   - `desktop_observe` → `active_window`
   - `desktop_observe` → `screenshot`
   - `audio_control` → `get_audio_status`
   - `audio_control` → `list_audio_devices`
   - `audio_control` → `play_audio`
   - `camera_control` → `list_cameras`
   - `computer_control` → `open_url`
   - `browser_control` → `navigate`
   - `browser_control` → `screenshot`
   - `computer_control` → `tap` / `swipe` after Accessibility is enabled

## Repository layout

```text
.
├── app/
│   └── src/main/
│       ├── AndroidManifest.xml
│       ├── java/net/nexlab/hermesnodeandroid/
│       │   ├── ConfigStore.kt
│       │   ├── GatewayClient.kt
│       │   ├── HermesNodeService.kt
│       │   ├── MainActivity.kt
│       │   ├── NodeAgent.kt
│       │   ├── ProtocolModels.kt
│       │   ├── StatusStore.kt
│       │   └── capabilities/
│       └── res/
├── docs/
├── gradle/wrapper/
├── build.gradle.kts
├── settings.gradle.kts
└── README.md
```

## Design constraints

- Outbound-only connection from Android to gateway.
- No root requirement for baseline operation.
- Foreground service for persistent connectivity.
- No hidden capture/spyware behavior: camera, mic, screen capture, and accessibility features must use explicit Android permission/UX flows.
- No hardcoded tokens.

## License

GPL-3.0-or-later. See `LICENSE`.
