# bssh - Broadcast SSH
[](https://crates.io/crates/bssh)
[)](https://crates.io/crates/bssh)
)
[](https://deps.rs/repo/github/lablup/bssh)
A high-performance SSH client with **SSH-compatible syntax** for both single-host and parallel cluster operations, built with Rust and `russh`.
*Developed and maintained as part of the Backend.AI project.*
## Recent Updates
- **v3.0.1 (2026/09/01):** Replaced developer-only tracking and historical-version references in active help, runtime warnings, manpages, current release notes, and public documentation with behavior-focused guidance. Added rendered-help regression checks for all three binaries. Runtime behavior was unchanged.
- **v3.0.0 (2026/09/01):** Made single-destination sessions byte-transparent, applied SSH configuration at runtime, added OpenSSH configuration, session, forwarding, and multiplexing options, and restored seven colliding short flags to their OpenSSH meanings. The pinned OpenSSH suite passes 60 tests on macOS and 61 on Ubuntu.
- **v2.4.3 (2026/08/24):** Fix bssh-to-bssh SFTP downloads above 255 KiB, upgrade to russh 0.63.1 with two security fixes and host-certificate refusal, sync bssh-russh-sftp 2.4.0, and repair release automation for API-key notarization and Homebrew updates.
- **v2.4.2 (2026/08/14):** Republish the macOS binaries with a Developer ID signature and Apple notarization. The v2.4.1 and earlier macOS builds are killed on launch and deleted by macOS because their signing certificate was revoked. No source changes.
- **v2.4.1 (2026/08/03):** Close the four host key and forwarding known issues from v2.4.0: `accept-new` no longer disables verification without a home directory or an unusable known_hosts file, first-use recording is serialized across processes, `@cert-authority` lines can be rejected, forwarding targets keep their hostnames for the server to resolve, SOCKS4 honors `-4`/`-6`, and SOCKS5 accepts IPv6 destination literals.
- **v2.4.0 (2026/08/03):** Implement real TOFU verification for the default `accept-new` host key mode, make `bssh ping` report a 0/1/255 exit code, wire up the `-4`/`-6` flags and ssh_config `AddressFamily` across connect, forwarding, and jump chains, resolve ssh_config settings per target host, and accept bracketed IPv6 host literals.
- **v2.3.1 (2026/07/29):** Restore keyboard protocols leaked by disconnected PTY applications, preserve an outer TUI's keyboard state, raise the MSRV and Debian/Launchpad toolchain to Rust 1.96, and make Homebrew formula updates safer.
- **v2.3.0 (2026/07/18):** Roughly double bssh-server SFTP write throughput, fix the paramiko prefetch deadlock (with TCP_NODELAY), make `sftp.root`/`scp.root` chroot usable, make server SSH compression configurable, and drop the vendored `bssh-russh` fork for upstream russh 0.62.1.
- **v2.2.3 (2026/05/25):** Sync both internal russh forks to upstream stable and patch RUSTSEC-2026-0009 (a `time` stack-exhaustion DoS), raising the minimum supported Rust to 1.88.
- **v2.2.2 (2026/05/25):** Keep idle SSH sessions alive by lowering the default `--server-alive-interval` to 30s and leaving the client inactivity timeout disabled.
- **v2.2.1 (2026/05/19):** Dependency upgrade pass and russh fork sync picking up both halves of CVE-2026-46673, plus a bssh-russh dev-deps fix so its inline tests compile.
- **v2.2.0 (2026/05/18):** Collect `--password` once and share it across parallel tasks, add `BSSH_PASSWORD`, resolve all cargo-audit findings, and drop five redundant dependencies.
- **v2.1.4 (2026/05/10):** Stream SFTP transfers in 255 KiB chunks (~160x lower memory, ~11x faster 1 GiB upload) and pipeline up to 64 concurrent requests.
- **v2.1.3 (2026/04/30):** Fix SCP/SFTP path doubling and chroot config, vendor `russh-sftp` with a serde_bytes perf fix (+29% upload), and forward-port unreleased upstream russh fixes.
_See [CHANGELOG.md](./CHANGELOG.md) for the complete version history._
## Features
- **Measured SSH Compatibility**: Single-host behavior is continuously checked against a pinned OpenSSH regress suite; see the [measured score and documented skips](docs/openssh-regress.md)
- **Port Forwarding**: Full support for local (-L), remote (-R), and dynamic (-D) SSH port forwarding
- **Jump Host Support**: Connect through bastion hosts using OpenSSH ProxyJump syntax (`-J`)
- **Parallel Execution**: Execute commands across multiple nodes simultaneously
- **Hostlist Expressions**: pdsh-style range expansion (`node[1-5]`, `rack[1-2]-node[1-3]`) for compact host specification
- **Fail-Fast Mode**: Stop immediately on first failure with `--fail-fast` (`-k` remains available only in pdsh compatibility mode)
- **Interactive Terminal UI (TUI)**: Real-time monitoring with 4 view modes (Summary/Detail/Split/Diff) for multi-node operations
- **Cluster Management**: Define and manage node clusters via configuration files
- **Progress Tracking**: Real-time progress indicators with smart detection (percentages, fractions, apt/dpkg)
- **Flexible Authentication**: Support for SSH keys, SSH agent, password authentication, and encrypted key passphrases
- **Host Key Verification**: Secure host key checking with known_hosts support
- **Cross-Platform**: Works on Linux and macOS (see [Platform Support](#platform-support) for Windows status)
- **Output Management**: Multiple output modes (TUI, stream, file, normal) with auto-detection
- **Interactive Mode**: Interactive shell sessions with single-node or multiplexed multi-node support
- **SSH Config Caching**: High-performance caching of SSH configurations with TTL and file modification detection
- **Configurable Timeouts**: Set both connection timeout (`--connect-timeout`) and command execution timeout (`--timeout`) with support for unlimited execution
- **SSH Keepalive**: Configurable keepalive settings (`--server-alive-interval`, `--server-alive-count-max`) to prevent idle connection timeouts
## Platform Support
- **Linux and macOS**: Fully supported, including SSH agent authentication (`--use-agent`), PTY-based interactive sessions, and every CLI feature documented here.
- **Windows (native)**: Not currently supported as a client. The `bssh` crate does not build for a Windows target today because of unconditional Unix-only dependencies (`nix`, `signal-hook`, `libc`) and un-gated PTY/agent code; there is no Windows CI job or release artifact. Use **WSL2** to run `bssh` on Windows.
## Installation
### Install via Homebrew (macOS/Linux)
The easiest way to install `bssh` on macOS and Linux is through Homebrew:
```bash
brew tap lablup/tap
brew install bssh
```
### Install via Ubuntu PPA
For Ubuntu users, `bssh` is available through the official PPA:
```bash
# Add the PPA repository
sudo add-apt-repository ppa:lablup/backend-ai
sudo apt update
# Install bssh
sudo apt install bssh
```
### Install via Debian Package
For Debian and other Debian-based distributions, download the `.deb` package from the [releases page](https://github.com/lablup/bssh/releases):
```bash
# Download the latest .deb package (replace VERSION with the actual version)
wget https://github.com/lablup/bssh/releases/download/vVERSION/bssh_VERSION_OS_ARCH.deb
# Example: bssh_0.4.0_ubuntu24.04.noble_amd64.deb
# Install the package
sudo dpkg -i bssh_VERSION_OS_ARCH.deb
# If there are dependency issues, fix them with:
sudo apt-get install -f
```
### Download Pre-built Binary
Download the latest release from the [GitHub releases page](https://github.com/lablup/bssh/releases):
1. Go to https://github.com/lablup/bssh/releases
2. Download the appropriate binary for your platform
3. Extract the archive and place the binary in your `$PATH`
### Install from Cargo
```bash
cargo build --release
sudo cp target/release/bssh /usr/local/bin/
```
## Quick Start
### SSH-Compatible Mode (Single Host)
The 3.0 command-line contract gives these seven letters their OpenSSH meanings. Existing bssh scripts must use the corresponding long option for the displaced bssh feature; see the [3.0 short-flag migration guide](docs/openssh-short-flags-migration.md).
| `-N` | Do not execute a remote command | `--no-prefix` |
| `-f` | Go to the background after authentication | `--filter` |
| `-C` | Enable SSH compression | `--cluster` |
| `-A` | Enable authentication-agent forwarding | `--use-agent` |
| `-S path` | Select the multiplexing `ControlPath` | `--sudo-password` |
| `-k` | Disable GSSAPI credential delegation | `--fail-fast` |
| `-b address` | Bind the client to a source address | `--batch` |
```bash
# Connect to a host (just like SSH!)
bssh user@hostname
# Execute a command
bssh user@hostname "uptime"
# With specific port and key
bssh -p 2222 -i ~/.ssh/key.pem admin@server.com
# Using SSH options
bssh -o StrictHostKeyChecking=no user@host
# Query SSH capabilities
bssh -Q cipher
```
**PTY Session Escape Sequences:**
Like OpenSSH, bssh supports escape sequences in PTY sessions. These must be typed at the beginning of a line (after pressing Enter):
| `~.` | Disconnect from the remote host |
### Port Forwarding
```bash
# Local port forwarding (-L)
# Forward local port 8080 to example.com:80 via SSH
bssh -N -L 8080:example.com:80 user@host
# Remote port forwarding (-R)
# Forward remote port 8080 to localhost:80
bssh -N -R 8080:localhost:80 user@host
# Dynamic port forwarding / SOCKS proxy (-D)
# Create SOCKS5 proxy on local port 1080
bssh -N -D 1080 user@host
# Multiple port forwards
bssh -N -L 3306:db:3306 -R 80:web:80 -D 1080 user@host
# Bind to specific address
bssh -N -L 127.0.0.1:8080:web:80 user@host # Local only
bssh -N -L *:8080:web:80 user@host # All interfaces
# SOCKS4 proxy (specify version)
bssh -N -D 1080/4 user@host # SOCKS4
bssh -N -D *:1080/5 user@host # SOCKS5 on all interfaces
# Port forwarding with command execution
bssh -L 5432:postgres:5432 user@host "psql -h localhost"
# Port forwarding without opening a remote shell
bssh -N -L 8080:internal:80 user@gateway
```
### Jump Host Support (ProxyJump)
```bash
# Connect through a single jump host (bastion)
bssh -J jump@bastion.example.com user@internal-server
# Multiple jump hosts (connection chain)
bssh -J "jump1@proxy1,jump2@proxy2" user@final-destination
# Jump host with custom port
bssh -J admin@bastion:2222 user@internal-host
# IPv6 jump host
bssh -J "[2001:db8::1]:22" user@destination
# Combine with cluster operations
bssh -J bastion.example.com --cluster production "uptime"
# File transfer through jump host
bssh -J bastion.example.com -H internal-server upload app.tar.gz /opt/
bssh -J admin@bastion:2222 --cluster production download /etc/config ./backups/
# Interactive mode through jump hosts
bssh -J bastion.example.com user@internal-server
bssh -J "jump1,jump2" --cluster production interactive
# Multi-hop with file transfer
bssh -J "bastion1,bastion2,bastion3" -H target upload -r ./app/ /opt/app/
```
### Multi-Server Mode (Cluster Operations)
```bash
# Execute commands on multiple hosts (automatic command execution)
bssh -H "user1@host1.com,user2@host2.com:2222" "uptime"
# Using cluster from config
bssh --cluster production "df -h"
# Hostlist expressions (pdsh-style range expansion)
bssh -H "node[1-5]" "uptime" # node1, node2, node3, node4, node5
bssh -H "node[01-05]" "df -h" # Zero-padded: node01, node02, ...
bssh -H "node[1,3,5]" "ps aux" # Specific values: node1, node3, node5
bssh -H "rack[1-2]-node[1-3]" "uptime" # Cartesian product: 6 hosts
bssh -H "web[1-3].example.com" "nginx -v" # With domain suffix
bssh -H "admin@db[01-03]:5432" "psql --version" # With user and port
bssh -H "^/etc/hosts.cluster" "uptime" # Read hosts from file
# Filter specific hosts with pattern matching
bssh -H "web1,web2,db1,db2" --filter "web*" "systemctl status nginx"
bssh --cluster production --filter "db*" "pg_dump --version"
bssh -H "node[1-10]" --filter "node[1-5]" "uptime" # Filter with hostlist expression
# Exclude specific hosts from execution
bssh -H "node1,node2,node3" --exclude "node2" "uptime"
bssh --cluster production --exclude "db*" "systemctl restart nginx"
bssh -H "node[1-10]" --exclude "node[3-5]" "uptime" # Exclude with hostlist expression
# With custom SSH key
bssh --cluster staging -i ~/.ssh/custom_key "systemctl status nginx"
# Use SSH agent for authentication
bssh --use-agent --cluster production "systemctl status nginx"
# Use password authentication (will prompt for password)
bssh --password -H "user@host.com" "uptime"
# Use sudo password for privileged commands (prompts securely)
bssh --sudo-password --cluster production "sudo apt update && sudo apt upgrade -y"
# Combine sudo password with SSH agent authentication
bssh --use-agent --sudo-password --cluster production "sudo systemctl restart nginx"
# Use encrypted SSH key (will prompt for passphrase)
bssh -i ~/.ssh/encrypted_key --cluster production "df -h"
# Limit parallel connections
bssh --cluster production --parallel 5 "apt update"
# Set command timeout (10 seconds)
bssh --cluster production --timeout 10 "quick-check"
# No timeout (unlimited execution time)
bssh --cluster staging --timeout 0 "long-running-backup"
# Set connection timeout (default: 30 seconds)
bssh --cluster production --connect-timeout 10 "uptime"
# Different timeouts for connection and command
bssh --cluster production --connect-timeout 5 --timeout 600 "long-running-job"
# Configure SSH keepalive (prevent idle connection timeouts)
bssh --cluster production --server-alive-interval 30 "long-running-job"
# Disable keepalive (set interval to 0)
bssh --cluster production --server-alive-interval 0 "quick-job"
# Keepalive with custom max retries (default: 3)
bssh --cluster production --server-alive-interval 30 --server-alive-count-max 5 "long-running-job"
# Fail-fast mode: stop immediately on any failure
bssh --fail-fast -H "web1,web2,web3" "deploy.sh"
bssh --fail-fast --cluster production "critical-script.sh"
# Combine fail-fast with require-all-success for critical operations
bssh --fail-fast --require-all-success --cluster production "service-restart.sh"
```
### Output Modes
bssh automatically selects the best output mode based on your environment:
#### TUI Mode (Default in Terminals)
Interactive Terminal UI with real-time monitoring - automatically enabled when running in an interactive terminal.
```bash
# TUI mode automatically activates for multi-node commands
bssh --cluster production "apt-get update"
# Features:
# - Summary view: All nodes at a glance with progress bars
# - Detail view: Full output from specific node with scrolling
# - Split view: Monitor 2-4 nodes simultaneously in panes
# - Diff view: Compare output from two nodes side-by-side
# - Auto-scroll: Automatic following of new output
# - Progress detection: Percentage, fraction, apt/dpkg indicators
# - Real-time updates: Output streams in real-time (50ms polling)
# - Memory protection: 10MB buffer per node with graceful overflow
```
**TUI View Modes:**
| **Summary** | All nodes overview with status, progress bars, and recent output | Default view |
| **Detail** | Full output from a single node with scrollback | `1-9` (node number) |
| **Split** | Side-by-side display of 2-4 nodes (last 20 lines each) | `s` |
| **Diff** | Compare two nodes line-by-line | `d` |
**Keyboard Shortcuts:**
| **Global Keys** (work in any view) |||
| `q` | Any view | Quit TUI |
| `Ctrl+C` | Any view | Quit TUI |
| `?` | Any view | Toggle help overlay |
| `Esc` | Any view | Return to summary view (or close help) |
| `l` | Any view | Toggle log panel visibility |
| **Summary View** |||
| `1-9` | Summary | Jump to detail view for node N |
| `s` | Summary | Enter split view (first 2-4 nodes) |
| `d` | Summary | Enter diff view (first 2 nodes) |
| **Detail View** |||
| `↑` | Detail | Scroll up 1 line |
| `↓` | Detail | Scroll down 1 line |
| `←` | Detail | Switch to previous node |
| `→` | Detail | Switch to next node |
| `PgUp` | Detail | Scroll up 10 lines |
| `PgDn` | Detail | Scroll down 10 lines |
| `Home` | Detail | Jump to top of output |
| `End` | Detail | Jump to bottom (re-enables follow mode) |
| `f` | Detail | Toggle auto-scroll (follow mode) |
| `1-9` | Detail | Jump to specific node N |
| **Split View** |||
| `1-4` | Split | Focus on specific node (switch to detail view) |
| **Diff View** |||
| `↑/↓` | Diff | Scroll* |
| **Log Panel** (when visible) |||
| `j` | Log panel | Scroll log up |
| `k` | Log panel | Scroll log down |
| `+` | Log panel | Increase log panel height |
| `-` | Log panel | Decrease log panel height |
| `t` | Log panel | Toggle timestamps |
*\*Note: Diff view scroll is planned but not yet implemented.*
**Log Panel:**
The TUI includes an in-app log panel that captures error and warning messages without breaking the alternate screen. This prevents log messages from corrupting the TUI display during execution.
- Toggle visibility with `l` key
- Color-coded by level: ERROR (red), WARN (yellow), INFO (white), DEBUG (gray)
- Configurable buffer size via `BSSH_TUI_LOG_MAX_ENTRIES` environment variable (default: 1000, max: 10000)
- Panel height adjustable from 3-10 lines
**TUI Activation:**
- **Automatic**: Multi-node execution in interactive terminal
- **Disabled when**:
- Single node execution
- Output piped or redirected (`| tee`, `>`)
- CI environment (`CI=true`)
- Explicit flags (`--stream`, `--output-dir`)
- **Minimum terminal size**: 40x10 characters
#### Stream Mode (Real-time with Node Prefixes)
```bash
# Enable stream mode explicitly
bssh --cluster production --stream "tail -f /var/log/syslog"
# Output:
# [node1] Oct 30 10:15:23 systemd[1]: Started nginx.service
# [node2] Oct 30 10:15:24 kernel: [UFW BLOCK] IN=eth0 OUT=
# [node1] Oct 30 10:15:25 nginx: Configuration test successful
# Stream mode without hostname prefix (pdsh -N compatibility)
bssh --cluster production --stream --no-prefix "uname -a"
# Output (no [node] prefixes):
# Linux node1 5.15.0-generic
# Linux node2 5.15.0-generic
```
#### File Mode (Save to Per-Node Files)
```bash
# Save each node's output to timestamped files
bssh --cluster production --output-dir ./logs "ps aux"
# Creates:
# ./logs/node1_20251030_101523.stdout
# ./logs/node2_20251030_101523.stdout
# ./logs/node1_20251030_101523.stderr (if there are errors)
```
#### Normal Mode (Traditional Output)
```bash
# Automatically used when output is piped or redirected
# Manually disable TUI in terminals
CI=true bssh --cluster production "command"
```
### Batch Mode (Ctrl+C Handling)
bssh provides two modes for handling Ctrl+C during parallel execution:
**Default (Two-Stage)**:
- First Ctrl+C: Shows status (running/completed counts)
- Second Ctrl+C (within 1 second): Terminates all jobs
**Batch Mode (`--batch`)**:
- Single Ctrl+C: Immediately terminates all jobs
- Useful for non-interactive scripts and CI/CD pipelines
```bash
# Default behavior (two-stage Ctrl+C)
bssh --cluster production "long-running-command"
# Ctrl+C once: shows status
# Ctrl+C again (within 1s): terminates
# Batch mode (immediate termination)
bssh --cluster production --batch "long-running-command"
# Ctrl+C once: immediately terminates all jobs
# Useful for automation
bssh -H nodes --batch --stream "deployment-script.sh"
```
### pdsh Compatibility Mode
bssh supports pdsh compatibility mode, enabling it to act as a drop-in replacement for pdsh. This allows seamless migration from pdsh without modifying existing scripts.
**📖 Complete Documentation**:
- **[Migration Guide](docs/pdsh-migration.md)** - Step-by-step migration from pdsh to bssh
- **[Options Reference](docs/pdsh-options.md)** - Complete option mapping table
- **[Examples](docs/pdsh-examples.md)** - Real-world usage patterns
#### Activation Methods
**1. Binary symlink** (recommended for full compatibility):
```bash
# Create symlink (done automatically by Homebrew)
sudo ln -sf "$(which bssh)" /usr/local/bin/pdsh
# Now pdsh commands use bssh
pdsh -w host1,host2 "uptime"
```
**2. Environment variable**:
```bash
BSSH_PDSH_COMPAT=1 bssh -w host1,host2 "uptime"
```
**3. CLI flag**:
```bash
bssh --pdsh-compat -w host1,host2 "uptime"
```
**4. Shell alias**:
```bash
# Add to ~/.bashrc or ~/.zshrc
alias pdsh='bssh --pdsh-compat'
```
#### pdsh Option Mapping
| `-w hosts` | `-H hosts` | Target hosts (comma-separated) |
| `-x hosts` | `--exclude hosts` | Exclude hosts from target list |
| `-f N` | `--parallel N` | Fanout (parallel connections, default: 32) |
| `-l user` | `-l user` | Remote username |
| `-t N` | `--connect-timeout N` | Connection timeout in seconds |
| `-u N` | `--timeout N` | Command timeout in seconds |
| `-N` | `--no-prefix` | Disable hostname prefix in output |
| `-b` | `--batch` | Batch mode (single Ctrl+C terminates) |
| `-k` | `--fail-fast` | Stop on first failure |
| `-q` | (query mode) | Show target hosts and exit |
| `-S` | `--any-failure` | Return largest exit code from any node |
#### pdsh Mode Examples
```bash
# Basic command execution
pdsh -w node1,node2,node3 "uptime"
# With fanout limit
pdsh -w nodes -f 10 "df -h"
# Exclude specific hosts
pdsh -w node[1-5] -x node3 "hostname"
# Query mode: show target hosts without executing
pdsh -w host1,host2,host3 -x host2 -q
# Output:
# host1
# host3
# Combine multiple options
pdsh -w servers -f 20 -l admin -t 30 -N "systemctl status nginx"
# Fail fast mode
pdsh -w nodes -k "critical-operation.sh"
```
#### Query Mode with Glob Patterns
Query mode (`-q`) supports glob pattern matching for exclusions:
```bash
# Exclude hosts matching a pattern
pdsh -w web1,web2,db1,db2 -x "db*" -q
# Output:
# web1
# web2
# Use wildcards in exclusion
pdsh -w node1,node2,backup1,backup2 -x "*backup*" -q
# Output:
# node1
# node2
```
### Built-in Commands
```bash
# Test connectivity to hosts
bssh --cluster production ping
bssh -H "host1,host2" ping
# List configured clusters
bssh list
# Interactive mode (single or multiplexed)
bssh --cluster production interactive
bssh -H "host1,host2" interactive
# File transfer operations
bssh --cluster production upload local.txt /tmp/
bssh -H "host1,host2" download /etc/hosts ./backups/
```
## Exit Code Strategies
By default, bssh returns the main rank's exit code, matching MPI-oriented
tools such as `mpirun`, `srun`, and `mpiexec`. Choose an explicit strategy
when a workflow must account for every node.
| Default | Return the main rank's actual exit code | MPI workloads and CI/CD |
| `--require-all-success` | Return 0 only when every node succeeds; otherwise return 1 | Health checks |
| `--check-all-nodes` | Return the main rank's code, or 1 if the main rank succeeds but another node fails | Cluster validation |
### Examples
```bash
# Preserve the main rank's actual exit status.
bssh -H "node[1-8]" "mpirun ./simulation"
# Fail the health check if any node fails.
bssh --require-all-success -H "node[1-8]" "health-check"
```
See [examples/mpi_exit_code.sh](examples/mpi_exit_code.sh) and [examples/health_check.sh](examples/health_check.sh) for detailed examples.
## Authentication
bssh supports multiple authentication methods:
### SSH Key Authentication
- **Default keys**: Automatically tries `~/.ssh/id_ed25519`, `~/.ssh/id_rsa`, `~/.ssh/id_ecdsa`, `~/.ssh/id_dsa`
- **Custom key**: Use `-i` flag to specify a key file
- **Encrypted keys**: Automatically detects and prompts for passphrase
### SSH Agent
- **Auto-detection**: Automatically uses SSH agent if `SSH_AUTH_SOCK` is set
- **Explicit**: Use `--use-agent` to force SSH agent authentication
### Password Authentication
- Use `--password` to enable password authentication
- The password is prompted **once up-front**, before any parallel connection tasks start, and is shared securely across all nodes — the prompt appears exactly once regardless of how many hosts are targeted
- Password is prompted securely without echo
- For automation, set `BSSH_PASSWORD` in the environment (not recommended; see security notes in the Sudo Password section)
### Examples
```bash
# Use default SSH key (auto-detect)
bssh -H "user@host" "uptime"
# Use specific SSH key (prompts for passphrase if encrypted)
bssh -i ~/.ssh/custom_key --cluster production "df -h"
# Use SSH agent
bssh --use-agent --cluster production "systemctl status"
# Use password authentication
bssh --password -H "user@host" "ls -la"
# Authentication through jump hosts
bssh --use-agent -J bastion.example.com user@internal-server "uptime"
bssh -i ~/.ssh/prod_key -J "jump1,jump2" --cluster production "df -h"
```
### Sudo Password Support
bssh supports automatic sudo password injection for commands that require elevated privileges. When enabled, bssh will:
1. Securely prompt for the sudo password before command execution
2. Detect sudo password prompts in command output
3. Automatically inject the password when prompted
4. Clear the password from memory after use
```bash
# Basic sudo command (will prompt for sudo password)
bssh --sudo-password --cluster production "sudo apt update"
# Combine with SSH agent authentication
bssh --use-agent --sudo-password --cluster production "sudo systemctl restart nginx"
# Multiple sudo commands in a single session
bssh --sudo-password --cluster production "sudo apt update && sudo apt upgrade -y"
# Sudo with specific SSH key
bssh -i ~/.ssh/admin_key --sudo-password --cluster production "sudo reboot"
```
**Environment Variable Alternative:**
For automation scenarios, you can use the `BSSH_SUDO_PASSWORD` environment variable:
```bash
# NOT RECOMMENDED for security reasons
export BSSH_SUDO_PASSWORD="your-password"
bssh --sudo-password --cluster production "sudo apt update"
```
**Security Warnings:**
- Environment variables may be visible in process listings
- Avoid storing passwords in shell history
- The `--sudo-password` option with secure prompt is the recommended approach
- Password is automatically cleared from memory after use using `zeroize`
## Environment Variables
bssh supports configuration via environment variables:
### Jump Host Configuration
- **`BSSH_MAX_JUMP_HOSTS`**: Maximum number of jump hosts allowed in a chain
- Default: 10
- Absolute maximum: 30 (security cap)
- Invalid or zero values fall back to default
- Example: `BSSH_MAX_JUMP_HOSTS=20 bssh -J host1,host2,...,host20 target`
### Backend.AI Integration Variables
- **`BACKENDAI_CLUSTER_HOSTS`**: Comma-separated list of all cluster nodes
- **`BACKENDAI_CLUSTER_HOST`**: Current node hostname
- **`BACKENDAI_CLUSTER_ROLE`**: Node role (main/sub)
### SSH Authentication Variables
- **`SSH_AUTH_SOCK`**: SSH agent socket path (Unix-like systems)
### SSH Password Variable
- **`BSSH_PASSWORD`**: SSH password for automated password authentication
- Used when `--password` is set and `BSSH_PASSWORD` is non-empty; skips the interactive prompt
- **WARNING**: Not recommended for security reasons
- Environment variables may be visible in process listings and shell history
- Use the interactive `--password` prompt instead for security-sensitive operations
- Example: `BSSH_PASSWORD=secret bssh --password -H "user@host" "uptime"`
### Sudo Password Variable
- **`BSSH_SUDO_PASSWORD`**: Sudo password for automated sudo authentication
- **WARNING**: Not recommended for security reasons
- Environment variables may be visible in process listings
- Use the `--sudo-password` option with secure prompt instead
- Example: `BSSH_SUDO_PASSWORD=password bssh --sudo-password --cluster prod "sudo apt update"`
## Configuration
### Configuration Priority Order
bssh loads configuration from the following sources in priority order:
1. **Backend.AI Environment Variables** (automatic detection)
2. **Current directory** (`./config.yaml`)
3. **XDG config directory** (`$XDG_CONFIG_HOME/bssh/config.yaml` or `~/.config/bssh/config.yaml`)
4. **CLI specified path** (via `--config` flag, default: `~/.config/bssh/config.yaml`)
### Backend.AI Multi-node Session Support
When running inside a Backend.AI multi-node session, bssh automatically detects cluster configuration from environment variables. No manual configuration or cluster specification needed!
Backend.AI environment variables used:
- `BACKENDAI_CLUSTER_HOSTS`: Comma-separated list of all node hostnames
- `BACKENDAI_CLUSTER_HOST`: Current node's hostname
- `BACKENDAI_CLUSTER_ROLE`: Current node's role (main or sub)
Note: Backend.AI multi-node clusters use SSH port 2200 by default, which is automatically configured.
**Automatic Detection:**
When these environment variables are set, bssh automatically creates a "backendai" cluster and uses it by default when no `-c` or `-H` options are specified.
Example:
```bash
# Inside Backend.AI multi-node session, just run:
bssh "uptime" # Automatically executes on all cluster nodes
# Or specify a command explicitly:
bssh "nvidia-smi" # Check GPU status on all nodes
# Interactive mode also works automatically:
bssh interactive # Opens interactive session with all Backend.AI nodes
# You can still override with explicit options if needed:
bssh --cluster other-cluster "command" # Use a different cluster
bssh -H specific-host "command" # Use specific host
```
### Manual Configuration File
Create a configuration file at any of these locations:
- `./config.yaml` (current directory)
- `~/.config/bssh/config.yaml` (user config directory)
- `~/.bssh/config.yaml` (default location)
```yaml
defaults:
user: admin
port: 22
ssh_key: ~/.ssh/id_rsa
parallel: 10
timeout: 300 # Command timeout in seconds (0 for unlimited)
jump_host: bastion.example.com # Global default jump host (optional)
server_alive_interval: 30 # SSH keepalive interval in seconds (0 to disable)
server_alive_count_max: 3 # Max keepalive messages without response
# Global interactive mode settings (optional)
interactive:
default_mode: multiplex # single_node or multiplex
prompt_format: "[{node}] $ " # Variables: {node}, {user}, {host}, {pwd}
history_file: ~/.bssh_history
show_timestamps: false # Show timestamps in output
work_dir: /home/admin # Initial working directory
broadcast_prefix: "!all " # Prefix for broadcasting to all nodes
node_switch_prefix: "!" # Prefix for special commands
colors: # Node-specific colors in output
node1: red
node2: blue
node3: green
keybindings:
switch_node: "Ctrl+N"
broadcast_toggle: "Ctrl+B"
quit: "Ctrl+Q"
clusters:
production:
nodes:
- web1.example.com
- web2.example.com
- user@web3.example.com:2222
ssh_key: ~/.ssh/prod_key
timeout: 600 # Override default timeout for this cluster
jump_host: prod-bastion.example.com # Cluster-specific jump host
# Cluster-specific interactive settings (overrides global)
interactive:
default_mode: single_node
prompt_format: "prod> "
work_dir: /var/www
staging:
nodes:
- host: staging1.example.com
port: 2200
user: deploy
jump_host: staging-bastion:2222 # Node-specific jump host
- staging2.example.com
user: staging_user
jump_host: "" # Explicitly disable jump host for this cluster
```
### Jump Host Configuration in config.yaml
bssh supports configuring jump hosts at three levels in your configuration file, with environment variable support:
**Priority Order** (highest to lowest):
1. **CLI `-J` option** - Always takes precedence
2. **Node-level** - Per-node `jump_host` in detailed node config
3. **Cluster-level** - `jump_host` in cluster defaults
4. **Global default** - `jump_host` in top-level defaults
**Example:**
```yaml
defaults:
jump_host: ${BASTION_HOST} # Environment variables supported
clusters:
production:
nodes:
- host: prod1.internal
- host: prod2.internal
jump_host: "" # Explicitly disable for this node
jump_host: prod-bastion.example.com
direct-access:
nodes:
- host: direct.example.com
jump_host: "" # Empty string disables inheritance
```
**Notes:**
- Empty string (`""`) explicitly disables jump host inheritance at any level
- Environment variables (`${VAR}` or `$VAR`) are expanded in jump_host values
- Node-level jump_host requires detailed node syntax (not simple hostname strings)
**Important: Username Handling**
The `user` field in config applies only to **destination nodes**, not to jump hosts:
```yaml
clusters:
internal:
nodes:
- 192.168.0.100
user: admin # Used for destination (192.168.0.100)
jump_host: bai@bastion # User 'bai' for bastion (jump host)
```
- **With username in jump_host:** `bai@bastion:4300` → uses "bai" for bastion
- **Without username:** `bastion:4300` → uses your current local username (like OpenSSH)
If jump host authentication fails due to username mismatch, specify the username explicitly in the `jump_host` field.
## SSH Configuration Support
bssh fully supports OpenSSH-compatible configuration files via the `-F` flag or default SSH config locations (`~/.ssh/config`, `/etc/ssh/ssh_config`). In addition to standard SSH directives, bssh supports advanced options for certificate-based authentication and port forwarding control.
### Certificate Authentication Options
These options enable enterprise-grade PKI authentication using SSH certificates:
| **CertificateFile** | SSH certificate file for PKI authentication (max 100 files) | `CertificateFile ~/.ssh/id_rsa-cert.pub` |
| **CASignatureAlgorithms** | CA signature algorithms for certificate validation (max 50) | `CASignatureAlgorithms ssh-ed25519,rsa-sha2-512` |
| **HostbasedAuthentication** | Enable host-based authentication (yes/no) | `HostbasedAuthentication yes` |
| **HostbasedAcceptedAlgorithms** | Algorithms for host-based auth (max 50) | `HostbasedAcceptedAlgorithms ssh-ed25519,rsa-sha2-512` |
### Port Forwarding Control Options
These options provide fine-grained control over SSH port forwarding:
| **GatewayPorts** | Control remote port forwarding (yes/no/clientspecified) | `GatewayPorts clientspecified` |
| **ExitOnForwardFailure** | Terminate connection if port forwarding fails (yes/no) | `ExitOnForwardFailure yes` |
| **PermitRemoteOpen** | Allowed destinations for remote forwarding (max 1000) | `PermitRemoteOpen localhost:8080` |
### Proxy Options
These options control SSH proxy connection behavior:
| **ProxyCommand** | Run a shell command whose stdin/stdout carry the SSH transport; supports `%%`, `%h`, `%k`, `%n`, `%p`, and `%r` | `ProxyCommand nc %h %p` |
| **ProxyUseFdpass** | Request descriptor passing from ProxyCommand (recognized but not supported by bssh) | `ProxyUseFdpass no` |
**ProxyCommand** uses the user's shell so OpenSSH-style quoting, redirection, and pipelines work. `ProxyCommand none` explicitly selects a direct connection. `ProxyCommand` and `ProxyJump` follow OpenSSH's first-obtained ssh_config rule, while command-line `-J` takes precedence over both.
*Note: `ProxyUseFdpass yes` is rejected with an actionable error before the command starts. Use a streaming ProxyCommand without `-F`, or set `ProxyUseFdpass no`.*
### Command Execution and Automation Options
These options enable powerful automation workflows and command execution features:
| **PermitLocalCommand** | Allow local command execution after connection (yes/no, default: no) | `PermitLocalCommand yes` |
| **LocalCommand** | Execute local command after successful connection (supports tokens: %h, %H, %n, %p, %r, %u) | `LocalCommand rsync -av ~/project/ %h:~/project/` |
| **RemoteCommand** | Execute command on remote host instead of shell | `RemoteCommand tmux attach -t dev \|\| tmux new -s dev` |
| **KnownHostsCommand** | Command to fetch host keys dynamically (supports tokens) | `KnownHostsCommand /usr/local/bin/fetch-host-key %H` |
| **ForkAfterAuthentication** | Fork to background after authentication (yes/no) | `ForkAfterAuthentication yes` |
| **SessionType** | Session type: none/subsystem/default | `SessionType none` |
| **StdinNull** | Redirect stdin from /dev/null (yes/no) | `StdinNull yes` |
**Token Substitution:**
LocalCommand and KnownHostsCommand support the following tokens:
- `%h` - Remote hostname (from config)
- `%H` - Remote hostname (as specified on command line)
- `%n` - Original hostname
- `%p` - Remote port
- `%r` - Remote username
- `%u` - Local username
- `%%` - Literal percent sign
### Host Key Verification & Security Options
These options provide enhanced security and host key management features:
| **NoHostAuthenticationForLocalhost** | Skip host key verification for localhost (yes/no, default: no) | `NoHostAuthenticationForLocalhost yes` |
| **HashKnownHosts** | Hash hostnames in known_hosts file for security (yes/no, default: no) | `HashKnownHosts yes` |
| **CheckHostIP** | Check host IP address in known_hosts (yes/no, **deprecated** in OpenSSH 8.5+) | `CheckHostIP no` |
| **VisualHostKey** | Display ASCII art of host key fingerprint (yes/no, default: no) | `VisualHostKey yes` |
| **HostKeyAlias** | Alias for host key lookup in known_hosts | `HostKeyAlias lb.example.com` |
| **VerifyHostKeyDNS** | Verify host keys using DNS SSHFP records (yes/no/ask, default: no) | `VerifyHostKeyDNS ask` |
| **UpdateHostKeys** | Accept updated host keys from server (yes/no/ask, default: no) | `UpdateHostKeys ask` |
### Additional Authentication Options
These options provide fine-grained control over authentication behavior:
| **NumberOfPasswordPrompts** | Password retry attempts (1-10, default: 3) | `NumberOfPasswordPrompts 1` |
| **EnableSSHKeysign** | Enable ssh-keysign for host-based auth (yes/no, default: no) | `EnableSSHKeysign yes` |
### Network & Connection Options
These options control network-level connection behavior:
| **BindInterface** | Bind connection to specific network interface | `BindInterface tun0` |
| **IPQoS** | Set IP QoS/DSCP values (interactive bulk) | `IPQoS lowdelay throughput` |
| **RekeyLimit** | Control SSH session key renegotiation (data time) | `RekeyLimit 1G 1h` |
### X11 Forwarding Options
These options control X11 display forwarding behavior:
| **ForwardX11Timeout** | Timeout for untrusted X11 forwarding (0 = no timeout) | `ForwardX11Timeout 1h` |
| **ForwardX11Trusted** | Enable trusted X11 forwarding (yes/no, default: no) | `ForwardX11Trusted yes` |
### Authentication and Security Management Options
These options provide essential authentication management, security enforcement, and user convenience features:
| **IdentitiesOnly** | Only use identity files specified in config, ignore SSH agent (yes/no) | `IdentitiesOnly yes` |
| **AddKeysToAgent** | Auto-add keys to SSH agent (yes/no/ask/confirm) | `AddKeysToAgent yes` |
| **UseKeychain** | **[macOS only]** Use macOS Keychain for SSH key passphrases (yes/no) | `UseKeychain yes` |
| **IdentityAgent** | Custom SSH agent socket path or "none" | `IdentityAgent ~/.1password/agent.sock` |
| **PubkeyAcceptedAlgorithms** | Restrict allowed public key algorithms (max 50) | `PubkeyAcceptedAlgorithms ssh-ed25519,rsa-sha2-512` |
| **RequiredRSASize** | Minimum RSA key size in bits (1024-16384, warns <2048) | `RequiredRSASize 2048` |
| **FingerprintHash** | Fingerprint hash algorithm (md5/sha256) | `FingerprintHash sha256` |
**Key Benefits:**
- **IdentitiesOnly**: Solves multi-account authentication conflicts
- **AddKeysToAgent**: Eliminates manual ssh-add commands
- **UseKeychain**: Seamlessly integrates with macOS Keychain for passphrase management (Apple-specific OpenSSH extension)
- **IdentityAgent**: Enables modern agent tools (1Password, gpg-agent, etc.)
- **PubkeyAcceptedAlgorithms**: Enforces security policies
- **RequiredRSASize**: Prevents weak RSA keys
- **FingerprintHash**: Flexibility for legacy systems
**Platform Notes:**
- **UseKeychain** is an Apple-specific patch to OpenSSH and only available on macOS
- Fully integrated with macOS Keychain via Security Framework for secure passphrase storage and retrieval
- Passphrases are automatically stored after successful authentication and retrieved from Keychain on subsequent connections
- For cross-platform configurations, use `IgnoreUnknown UseKeychain` to prevent errors on non-macOS systems
### SSH Config Examples
#### Certificate-based Authentication
```ssh-config
# ~/.ssh/config
# Production servers with certificate authentication
Host *.prod.example.com
User admin
CertificateFile ~/.ssh/prod-user-cert.pub
CertificateFile ~/.ssh/prod-host-cert.pub
CASignatureAlgorithms ssh-ed25519,rsa-sha2-512,rsa-sha2-256
HostbasedAuthentication yes
HostbasedAcceptedAlgorithms ssh-ed25519,rsa-sha2-512
```
#### Strict Port Forwarding Control
```ssh-config
# Secure hosts with restricted port forwarding
Host *.secure.prod.example.com
GatewayPorts clientspecified
ExitOnForwardFailure yes
PermitRemoteOpen localhost:8080
PermitRemoteOpen db.internal:5432
PermitRemoteOpen cache.internal:6379
```
#### Proxy Connection Optimization
```ssh-config
# HTTP CONNECT proxy
Host internal-server
ProxyCommand nc -X connect -x proxy.example.com:1080 %h %p
# SOCKS proxy with netcat
Host *.internal.example.com
ProxyCommand nc -x socks.example.com:1080 %h %p
```
#### Command Execution and Automation
```ssh-config
# Development server with automatic file synchronization
Host dev-server
User developer
PermitLocalCommand yes
LocalCommand rsync -av ~/project/ %h:~/project/
# Auto-attach to tmux session on connection
Host project-server
RemoteCommand tmux attach -t project || tmux new -s project
RequestTTY yes
# Cloud instances with dynamic host key fetching
Host *.cloud.example.com
KnownHostsCommand /usr/local/bin/fetch-cloud-key %H
StrictHostKeyChecking accept-new
# Background SSH tunnel for port forwarding
Host tunnel
ForkAfterAuthentication yes
SessionType none
LocalForward 8080 internal-server:80
StdinNull yes
```
#### Host Key Verification, Authentication, and Network Options
```ssh-config
# Local development environment - skip localhost verification
Host localhost 127.0.0.1 ::1
NoHostAuthenticationForLocalhost yes
NumberOfPasswordPrompts 1
# Security-hardened configuration with host key protection
Host *.secure.example.com
HashKnownHosts yes
VisualHostKey yes
VerifyHostKeyDNS ask
UpdateHostKeys ask
CheckHostIP no
# Load-balanced service with shared host key
Host lb-node-*
HostKeyAlias lb.example.com
# Multi-homed host with specific interface binding
Host vpn-only
BindInterface tun0
IPQoS lowdelay throughput
# High-security session with frequent rekeying
Host sensitive-data
RekeyLimit 500M 30m
# X11 forwarding with timeout and trust for graphics workstation
Host graphics-workstation
ForwardX11 yes
ForwardX11Trusted yes
ForwardX11Timeout 2h
```
#### Authentication and Security Best Practices
```ssh-config
# Multi-account setup - prevent agent key conflicts
Host work
HostName work.example.com
IdentityFile ~/.ssh/work_rsa
IdentitiesOnly yes
Host personal
HostName github.com
User git
IdentityFile ~/.ssh/personal_ed25519
IdentitiesOnly yes
# Auto-add keys to SSH agent for convenience
Host *
AddKeysToAgent yes
# macOS-specific: Use Keychain for SSH key passphrases
# For cross-platform configs, add IgnoreUnknown to prevent errors on non-macOS systems
Host *
IgnoreUnknown UseKeychain
UseKeychain yes
# Custom SSH agent integration (1Password, gpg-agent)
Host secure-*
IdentityAgent ~/.1password/agent.sock
# Disable SSH agent for specific hosts
Host no-agent-host
IdentityAgent none
# Security-hardened production servers
Host *.prod.example.com
# Only allow modern, secure algorithms
PubkeyAcceptedAlgorithms ssh-ed25519,rsa-sha2-512,rsa-sha2-256
# Enforce strong RSA keys
RequiredRSASize 2048
# Use modern fingerprint hashing
FingerprintHash sha256
# Legacy system compatibility
Host legacy.example.com
# Allow older RSA keys
RequiredRSASize 1024
# Use MD5 for legacy fingerprint verification
FingerprintHash md5
```
#### Complete Example with Include and Match
```ssh-config
# Base security settings
Host *
HostbasedAuthentication no
ExitOnForwardFailure no
PermitLocalCommand no
# Production certificate configuration
Host *.prod.example.com
CertificateFile ~/.ssh/prod-cert.pub
CASignatureAlgorithms ssh-ed25519,rsa-sha2-512
HostbasedAuthentication yes
# Match directive for secure hosts
Match host *.secure.prod.example.com
GatewayPorts clientspecified
ExitOnForwardFailure yes
PermitRemoteOpen localhost:8080
# Development hosts with automation
Match host *.dev.example.com
PermitLocalCommand yes
LocalCommand notify-send "Connected to %h"
# Specific host overrides
Host web.secure.prod.example.com
User webadmin
Port 443
CertificateFile ~/.ssh/web-specific-cert.pub
```
### Using SSH Config with bssh
```bash
# Use default SSH config (~/.ssh/config)
bssh user@host.prod.example.com
# Use custom SSH config file
bssh -F ~/custom-ssh-config user@host.prod.example.com
# SSH config works with cluster operations
bssh --cluster production "uptime"
# Config options apply to all cluster nodes
bssh -F ~/.ssh/prod-config --cluster production upload app.tar.gz /opt/
```
## Command-Line Options
```
Options:
-H, --hosts <HOSTS> Comma-separated list of hosts (user@host:port format)
-C Enable SSH compression
-f Go to background after authentication
--cluster <CLUSTER> Cluster name from configuration file
--filter <PATTERN> Filter hosts by pattern (supports wildcards like 'web*')
--exclude <HOSTS> Exclude hosts from target list (comma-separated, supports wildcards)
--config <CONFIG> Configuration file path [default: ~/.config/bssh/config.yaml]
-u, --user <USER> Default username for SSH connections
-i, --identity <IDENTITY> SSH private key file path (prompts for passphrase if encrypted)
-A Enable authentication-agent forwarding
--use-agent Use SSH agent for authentication (Unix/Linux/macOS only)
--password Use password authentication (will prompt for password)
-S <CONTROL_PATH> Multiplexing control socket path
--sudo-password Prompt for sudo password to auto-respond to sudo prompts
-N Do not execute a remote command
-k Disable GSSAPI credential delegation
-b <BIND_ADDRESS> Bind to a local source address
-J, --jump-host <JUMP_HOSTS> Jump hosts: [user@]host[:port],... (uses local user if not specified)
-L, --local-forward <SPEC> Local port forwarding [bind_address:]port:host:hostport
-R, --remote-forward <SPEC> Remote port forwarding [bind_address:]port:host:hostport
-D, --dynamic-forward <SPEC> Dynamic port forwarding (SOCKS) [bind_address:]port[/version]
--strict-host-key-checking <MODE> Host key checking mode (yes/no/accept-new) [default: accept-new]
--parallel <PARALLEL> Maximum parallel connections [default: 10]
--timeout <TIMEOUT> Command timeout in seconds (0 for unlimited) [default: 300]
--connect-timeout <SECONDS> SSH connection timeout in seconds (minimum: 1) [default: 30]
--server-alive-interval <SECONDS> SSH keepalive interval in seconds (0 to disable) [default: 30]
--server-alive-count-max <COUNT> Max keepalive messages without response [default: 3]
--output-dir <OUTPUT_DIR> Output directory for command results
--no-prefix Disable hostname prefix in output
--fail-fast Stop on the first failed node
--batch Terminate all jobs on the first Ctrl+C
-v, --verbose Increase verbosity (-v, -vv, -vvv)
-h, --help Print help
-V, --version Print version
```
## Examples
### Backend.AI Multi-node Session
```bash
# Inside Backend.AI session - automatic cluster detection
bssh "hostname" # Shows hostnames of all nodes
bssh "nvidia-smi --query-gpu=name,memory.total --format=csv" # GPU info
bssh "python train.py --distributed" # Run distributed training
```
### Run system updates
```bash
bssh --cluster production "sudo apt update && sudo apt upgrade -y"
```
### Check disk usage
```bash
### Restart services
```bash
bssh --cluster webservers "sudo systemctl restart nginx"
```
### Collect logs
```bash
bssh --cluster production --output-dir ./logs "tail -n 100 /var/log/syslog"
```
### Long-running commands with timeout
```bash
# Set 30 minute timeout for backup operations
bssh --cluster production --timeout 1800 "backup-database.sh"
# No timeout for data migration (may take hours)
bssh --cluster production --timeout 0 "migrate-data.sh"
# Quick health check with 5 second timeout
bssh --cluster monitoring --timeout 5 "health-check.sh"
```
### Interactive Mode
Start an interactive shell session on cluster nodes:
```bash
# Interactive session on all nodes (multiplex mode - default)
bssh --cluster production interactive
# Interactive session on a single node
bssh --cluster production interactive --single-node
# Custom prompt format
bssh -H server1,server2 interactive --prompt-format "{user}@{host}> "
# Set initial working directory
bssh --cluster staging interactive --work-dir /var/www
# Interactive mode with keepalive for long-running sessions (e.g., tmux)
bssh --cluster production --server-alive-interval 30 --server-alive-count-max 5 interactive
```
#### Interactive Mode Configuration
Interactive mode can be configured in your `config.yaml` file with both global and per-cluster settings. CLI arguments always override configuration file settings.
**Global Configuration** (applies to all clusters unless overridden):
```yaml
interactive:
default_mode: multiplex # or single_node
prompt_format: "[{node}] $ "
history_file: ~/.bssh_history
show_timestamps: true # Add timestamps to output
work_dir: /home/user
broadcast_prefix: "!all " # Custom prefix for broadcast commands
node_switch_prefix: "!" # Custom prefix for special commands
```
**Per-Cluster Configuration** (overrides global settings):
```yaml
clusters:
production:
interactive:
default_mode: single_node # Different mode for this cluster
prompt_format: "PROD> "
work_dir: /var/app
```
**Configuration Priority**:
1. CLI arguments (highest priority)
2. Cluster-specific configuration
3. Global configuration
4. Built-in defaults
In multiplex mode, commands are sent to active nodes with visual indicators:
```
[● ● ●] bssh> uptime
[node1] 10:23:45 up 5 days, 2:14, 1 user, load average: 0.15, 0.12, 0.09
[node2] 10:23:45 up 3 days, 4:22, 2 users, load average: 0.23, 0.19, 0.17
[node3] 10:23:45 up 7 days, 1:45, 1 user, load average: 0.08, 0.11, 0.10
[● ● ●] bssh> exit
```
#### Interactive Mode Special Commands
Interactive mode supports special commands for node management. By default, these commands start with `!` but the prefix can be customized in the configuration file.
| `!all` | Activate all connected nodes |
| `!broadcast <cmd>` | Execute command on all nodes temporarily (without changing active nodes) |
| `!node<N>` or `!n<N>` | Switch to node N (e.g., `!node1`, `!n2`) |
| `!list` or `!nodes` | List all nodes with their connection status |
| `!status` | Show currently active nodes |
| `!help` or `!?` | Show help for special commands |
| `exit` | Exit interactive mode |
**Note**: The `!` prefix and `!broadcast` command can be customized via configuration:
```yaml
interactive:
node_switch_prefix: "@" # Use @ instead of !
broadcast_prefix: "@all " # Use @all instead of !broadcast
```
#### Node Indicators in Prompt
The prompt shows node status with visual indicators:
- `●` Active node (commands will be executed)
- `○` Inactive node (connected but not receiving commands)
- `·` Disconnected node
Examples:
- `[● ● ●] bssh>` - All 3 nodes active
- `[● ○ ○] bssh>` - Only first node active
- `[1 · ·] (1/3) bssh>` - Node 1 active, nodes 2 and 3 inactive
For large clusters (>10 nodes), the prompt uses a compact format:
- `[All 50/50] bssh>` - All 50 nodes active
- `[None 0/50] bssh>` - No nodes active
- `[Nodes 1,2,3... +47] (50/50) bssh>` - Specific nodes active
#### Example Interactive Session
```bash
$ bssh --cluster production interactive
Connected to 3 nodes
[● ● ●] bssh> !status
Active nodes: node1.example.com, node2.example.com, node3.example.com
[● ● ●] bssh> !node1
Switched to node 1
[● ○ ○] (1/3) bssh> hostname
[node1] node1.example.com
[● ○ ○] (1/3) bssh> !broadcast date
Broadcasting command to all connected nodes...
[node1] Thu Aug 22 10:30:00 UTC 2025
[node2] Thu Aug 22 10:30:00 UTC 2025
[node3] Thu Aug 22 10:30:00 UTC 2025
[● ○ ○] (1/3) bssh> !all
All nodes activated
[● ● ●] bssh> df -h /
[node1] Filesystem Size Used Avail Use% Mounted on
[node1] /dev/sda1 20G 5.5G 14G 30% /
[node2] Filesystem Size Used Avail Use% Mounted on
[node2] /dev/sda1 20G 7.2G 12G 38% /
[node3] Filesystem Size Used Avail Use% Mounted on
[node3] /dev/sda1 20G 4.1G 15G 22% /
[● ● ●] bssh> exit
Goodbye!
```
## Output File Management
When using the `--output-dir` option, bssh saves command outputs to structured files:
### File Structure
```
output-dir/
├── hostname1_20250821_143022.stdout # Standard output
├── hostname1_20250821_143022.stderr # Standard error (if any)
├── hostname2_20250821_143022.stdout # Per-node outputs
├── hostname2_20250821_143022.error # Connection/execution errors
├── hostname3_20250821_143022.empty # Marker for no output
└── summary_20250821_143022.txt # Overall execution summary
```
### File Types
- **`.stdout`**: Contains standard output from successful commands
- **`.stderr`**: Contains standard error output (created only if stderr is not empty)
- **`.error`**: Contains error messages for failed connections or executions
- **`.empty`**: Marker file when command produces no output
- **`summary_*.txt`**: Overall execution summary with success/failure counts
### File Headers
Each output file includes metadata headers:
```
# Command: df -h
# Host: server1.example.com
# User: admin
# Exit Status: 0
# Timestamp: 20250821_143022
[actual command output follows]
```
### Example Usage
```bash
# Save outputs to timestamped directory
# Collect system information
bssh --cluster all-servers --output-dir ./system-info "uname -a; df -h; free -m"
# Debug failed services
bssh --cluster webservers --output-dir ./debug "systemctl status nginx"
```
## Development
Read [ARCHITECTURE](ARCHITECTURE.md) documentation for more information.
### Building
```bash
cargo build
```
Release builds use a tuned profile (`lto = "fat"`, `codegen-units = 1`) so the SSH framing, cipher, and SFTP hot paths inline across crates. For maximum throughput on known hardware, additionally pin the target CPU:
```bash
# Portable AVX2-class server build (Haswell/Zen and newer)
RUSTFLAGS="-C target-cpu=x86-64-v3" cargo build --release
# Optimized only for the build machine itself
RUSTFLAGS="-C target-cpu=native" cargo build --release
```
### Testing
```bash
cargo test
```
### Running locally
```bash
cargo run -- -H localhost "echo hello"
```
## SSH Implementation
This project's SSH functionality is built using:
- **[russh](https://github.com/Eugeny/russh)**: A pure Rust implementation of the SSH protocol, providing a modern and safe foundation for SSH communications without relying on C libraries. This is the core SSH library used directly as a dependency.
- **Implementation patterns from [async-ssh2-tokio](https://github.com/Miyoshi-Ryota/async-ssh2-tokio)**: While not used as a direct dependency, portions of the implementation code and architectural patterns from async-ssh2-tokio have been adapted and integrated into this project to provide high-level async/await APIs that work seamlessly with the Tokio runtime.
This combination enables bssh to achieve high performance parallel SSH operations while maintaining memory safety and avoiding common security vulnerabilities associated with traditional C-based SSH implementations.
## License
This project is licensed under the Apache License 2.0.
See the [LICENSE](./LICENSE) file for details.
## Changelog
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history. Recent highlights are in the [Recent Updates](#recent-updates) section near the top of this file.