# Synheart Sensor Agent
[](https://github.com/synheart-ai/synheart-sensor-agent/actions/workflows/ci.yml)
[](https://github.com/synheart-ai/synheart-sensor-agent/issues)
[](https://github.com/synheart-ai/synheart-sensor-agent/actions/workflows/release.yml)
[](https://crates.io/crates/synheart-sensor-agent)
[](https://docs.rs/synheart-sensor-agent)
[](LICENSE)
[](SECURITY.md)
A privacy-first PC background sensor that captures keyboard and mouse interaction timing (not content) for behavioral research.
## Overview
Synheart Sensor Agent is a **pure collector** for behavioral signals. It ingests
*how* you interact — never *what* you type or read — from two sources:
1. **Native OS input** — keystroke *timing* and category, mouse movement
*magnitude*, and click/scroll timing, captured via OS-level event APIs
(`start`).
2. **Browser behavior** — anonymized interaction metrics streamed from the
[Synheart Behavior Chrome extension](https://chromewebstore.google.com/detail/synheart-behavior/ekfebekofikmclohefbeoneioakgpgkn)
to a local HTTP endpoint (`serve`, behind the `server` feature).
The agent does **not** extract behavioral features, build sessions, or compute
derived metrics. Raw events are emitted via an in-process channel (or accepted and
acknowledged over the local HTTP endpoint) and discarded immediately. Feature
extraction, windowing, session management, and any Human State Interface (HSI)
output are the responsibility of downstream consumers (e.g. `synheart-core-rust` /
`synheart-session-runtime`).
### Key Features
- **Privacy by Design**: Never captures key content, passwords, or screen coordinates
- **Pure Collector**: Emits raw timing/magnitude events; no feature extraction or storage of raw data
- **Transparency**: Cumulative collection statistics viewable via the `status` command
- **App Context** (library API): `get_frontmost_app_id()` returns the foreground application's bundle ID on demand — never window titles, URLs, or content. The `start` command does not record it.
- **macOS Support**: Native integration using Core Graphics event taps
- **Windows Support**: Native integration using the Windows Hooks API
- **Optional Local Server**: Feature-gated HTTP endpoint for receiving behavioral data from a companion (e.g. a browser extension)
## Privacy Guarantees
```
╔══════════════════════════════════════════════════════════════════╗
║ ✓ WHAT WE CAPTURE: ║
║ • When keys are pressed (timing only) ║
║ • Key categories (typing, navigation, backspace, enter, …) ║
║ • Common shortcut patterns (copy, paste, … — timing only) ║
║ • How fast the mouse moves (magnitude only) ║
║ • When clicks and scrolls occur (timing only) ║
║ • Foreground app bundle ID (for task-switch detection) ║
║ ║
║ ✗ WHAT WE NEVER CAPTURE: ║
║ • Which keys you press (no passwords, messages, etc.) ║
║ • Where your cursor is (no screen position tracking) ║
║ • Window titles, file names, or application content ║
║ • Any screen content ║
╚══════════════════════════════════════════════════════════════════╝
```
Run `synheart-sensor privacy` at any time to print the full privacy declaration.
## Installation
### From crates.io
```bash
cargo install synheart-sensor-agent
```
### From Source
```bash
# Clone the repository
git clone https://github.com/synheart-ai/synheart-sensor-agent.git
cd synheart-sensor-agent
# Build in release mode
cargo build --release
# The binary will be at target/release/synheart-sensor
```
### Requirements
- **Rust**: 1.82 or later (MSRV)
- **macOS**: 10.15 (Catalina) or later
- **Windows**: 10 or later
- **Permissions**: Input Monitoring (macOS) or Run as Administrator (Windows)
## Usage
### Grant Input Capture Permission
#### macOS
Before the agent can capture events, you need to grant Input Monitoring permission:
1. Open **System Preferences** > **Security & Privacy** > **Privacy**
2. Select **Input Monitoring** in the left sidebar
3. Click the lock icon and authenticate
4. Add the `synheart-sensor` application
5. Restart the application
#### Windows
Run the sensor agent as Administrator to capture input events:
1. Right-click the `synheart-sensor.exe` binary
2. Select **Run as administrator**
3. Or launch your terminal (Command Prompt / PowerShell) as Administrator before running the binary
### Commands
```bash
# Start capturing (all sources by default)
synheart-sensor start
# Start with specific sources
synheart-sensor start --sources keyboard
synheart-sensor start --sources mouse
synheart-sensor start --sources keyboard,mouse
# Pause collection (signals a running instance via the config file)
synheart-sensor pause
# Resume collection
synheart-sensor resume
# Show permission state, configuration, and cumulative statistics
synheart-sensor status
# Print the privacy declaration
synheart-sensor privacy
# Show the active configuration and its file path
synheart-sensor config
```
### Example Output
While running, the agent prints a periodic heartbeat with the cumulative event
count, then a session summary on exit:
```
Synheart Sensor Agent v0.4.0
Starting collection...
Keyboard: enabled
Mouse: enabled
Press Ctrl+C to stop
[sensor] 312 events captured
[sensor] 689 events captured
^C
Stopping collection...
Session Statistics:
- Keyboard events processed: 421
- Mouse events processed: 268
- Shortcut events processed: 12
- Session duration: 34 seconds
Privacy Guarantee:
- No key content captured
- No cursor coordinates captured
- Only timing, categories, and magnitude data retained
```
## Library Usage
The crate can also be used as a library to consume the raw event stream directly:
```rust,no_run
use synheart_sensor_agent::{Collector, CollectorConfig, SensorEvent};
let mut collector = Collector::new(CollectorConfig::default());
collector.start().expect("Failed to start collector (check Input Monitoring permission)");
// Receive privacy-preserving events as they are captured.
while let Ok(event) = collector.receiver().recv() {
match event {
SensorEvent::Keyboard(e) => { /* timing + category, never key content */ }
SensorEvent::Mouse(e) => { /* movement magnitude / click / scroll timing */ }
SensorEvent::Shortcut(e) => { /* shortcut category (copy/paste/…), timing only */ }
}
}
```
Each `SensorEvent` carries only timing, a category, and (for mouse movement/scroll)
a magnitude — never key codes, characters, or coordinates.
## Browser Behavior Ingestion (HTTP Server)
The [Synheart Behavior Chrome extension](https://chromewebstore.google.com/detail/synheart-behavior/ekfebekofikmclohefbeoneioakgpgkn)
captures anonymized browser interaction metrics (keystroke timing categories,
scroll/cursor patterns, tab-focus changes — never URLs, titles, or content) and
streams them to the sensor agent over a local HTTP endpoint behind the `server`
feature flag. The agent validates the payload and acknowledges it; it performs no
feature extraction or storage — orchestration is the caller's responsibility.
```bash
# Build with the server feature
cargo build --release --features server
# Start the server (binds to 127.0.0.1 only)
./target/release/synheart-sensor serve --port 8081
# Require a bearer token on /collect (recommended)
./target/release/synheart-sensor serve --port 8081 --token "$MY_TOKEN"
# or via the environment:
SYNHEART_SENSOR_TOKEN="$MY_TOKEN" ./target/release/synheart-sensor serve
```
Endpoints:
| `GET` | `/health` | none | Liveness check; returns status and version |
| `POST` | `/collect` | `Authorization: Bearer …`* | Accepts a `{ "session": { … } }` JSON payload |
\* When a token is configured (via `--token` or `SYNHEART_SENSOR_TOKEN`), `/collect`
requires a matching `Authorization: Bearer <token>` header and returns `401`
otherwise. Configure the same token in the extension popup. If no token is set, the
endpoint is open and intended for trusted loopback-only use.
The extension defaults to `http://127.0.0.1:8081/collect` (port configurable in the
extension popup). A browser extension with host permissions is not subject to
browser CORS for loopback requests, so no `chrome-extension://` origin needs to be
allow-listed; CORS here only governs page-origin callers (`http://localhost`,
`http://127.0.0.1`).
## Configuration
Configuration is stored at:
- **macOS**: `~/Library/Application Support/synheart-sensor-agent/config.json`
- **Windows**: `%APPDATA%\synheart-sensor-agent\config.json`
Default configuration:
```json
{
"sources": {
"keyboard": true,
"mouse": true
},
"data_path": "<platform data dir>/synheart-sensor-agent",
"paused": false
}
```
Cumulative collection statistics are persisted alongside the config as
`transparency.json` in `data_path`.
## Architecture
```
Native OS input ─┐
(keyboard/mouse) │
▼
┌─────────────────────────────────────────────────────────────┐
│ Synheart Sensor Agent │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Collector │────────────────────▶│ Raw Events │ │
│ │ (platform) │ │ (channel) │ │
│ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────┐ ┌──────────────────────────────┐ │
│ │Transparency │ │ HTTP server (--features │ │
│ │ Log │ │ server): POST /collect │◀──┼── Chrome
│ └─────────────┘ └──────────────────────────────┘ │ extension
└─────────────────────────────────────────────────────────────┘ (127.0.0.1:8081)
Pure sensor — windowing, features, sessions, and HSI are handled
downstream (e.g. synheart-core-rust / synheart-session-runtime).
```
## Development
### Building
```bash
# Debug build
cargo build
# Release build
cargo build --release
# Build with the optional HTTP server
cargo build --features server
# Run tests
cargo test
cargo test --all-features
# Run with logging
RUST_LOG=debug cargo run -- start
```
A `Makefile` provides convenience targets (`make fmt`, `make lint`, `make test`,
`make build`, `make ci`, …).
### Running the Demo
```bash
cargo run --example capture_demo
```
### Project Structure
```
synheart-sensor-agent/
├── Cargo.toml # Project manifest
├── src/
│ ├── main.rs # CLI entry point
│ ├── lib.rs # Library exports
│ ├── config.rs # Configuration management
│ ├── server.rs # Optional HTTP server (--features server)
│ ├── collector/
│ │ ├── mod.rs # Platform selection + re-exports
│ │ ├── types.rs # Privacy-preserving event types
│ │ ├── macos.rs # macOS implementation (Core Graphics)
│ │ ├── windows.rs # Windows implementation (Hooks API)
│ │ └── noop.rs # No-op collector for other platforms
│ └── transparency/
│ ├── mod.rs # Transparency module
│ └── log.rs # Privacy-preserving collection statistics
└── examples/
└── capture_demo.rs # Demo application
```
## Troubleshooting
### macOS: "Insufficient permissions to capture input events"
1. Open System Preferences > Security & Privacy > Privacy
2. Select Input Monitoring
3. Ensure the application is in the list and checked
4. If already checked, remove and re-add the application
5. Restart the application
### Windows: input events not captured
1. Right-click the terminal or binary and select "Run as administrator"
2. If using PowerShell, launch it with elevated privileges
3. Check that antivirus / Group Policy is not blocking low-level input hooks
4. Restart the application
### No events being captured
- Ensure you're actively typing or moving the mouse
- Check that the correct sources are enabled (`--sources all`)
- Verify permission is granted with `synheart-sensor status`
- **macOS**: Confirm Input Monitoring permission in System Preferences
- **Windows**: Confirm the process is running as Administrator
### High CPU usage
- Some CPU use is expected during heavy, continuous input
- Usage should be minimal when idle
## Contributing
This is a source-available repository. Issues and feature requests are welcome;
pull requests are not accepted at this time. See [CONTRIBUTING.md](CONTRIBUTING.md)
for the rationale and the supported contribution path.
## Security
To report a vulnerability, see [SECURITY.md](SECURITY.md).
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- Built with [Rust](https://www.rust-lang.org/)
- macOS event capture via [core-graphics](https://crates.io/crates/core-graphics)
- Windows event capture via [windows](https://crates.io/crates/windows)
- CLI powered by [clap](https://crates.io/crates/clap)