breakmancer

Drop a breakpoint into any shell.

Need to debug a build script, but don't have a way to SSH into the server to poke around in the build environment—or you do, but you need a way to pause the script at exactly the right point? breakmancer allows you to do just that!

Functionally, this is a reverse shell for people who are authorized to jump into a remote server. Unlike the malicious kind of reverse shell, this one aims to preserve the security of the target.

Requirements

A development computer that is capable of listening on a public IP address (or an intranet IP if it's on the same network as the computer you're debugging.)
A target computer where you're trying to debug a script and that you have permission to shell into. This computer must be able to reach out to the development computer over TCP.

Usage

In this example, the development computer is reachable on port 12345 at the domain name home.timmc.org. The target is a GitHub Actions workflow.

#. Start controller session locally: On my laptop I run breakmancer start home.timmc.org:12345. It prints out instructions and a command line and waits for a connection. #. Set up the breakpoint: I add the breakpoint command that was printed out above to my GitHub workflow file. It might look like this:

name: Breakmancer Demo
on: [push]
jobs:
  demo:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Do some stuff
        run: |
          date > something.txt

      - name: Invoke breakpoint
        run: |
          chmod +x ./breakmancer
          ./breakmancer break --auth dual-pub -c home.timmc.org:12345 -i 4TXapNw9DLbnjhFx7

I also commit the breakmancer binary to my branch. (I could fetch it with curl instead, of course.)
I've adjusted the command to call ./breakmancer (it's not on the PATH) and I've ensured the binary is actually executable. #. Run: I push the branch, the GitHub Action runs, and breakmancer reaches back to my laptop. #. Verify: The breakpoint prints out a verification string like xWeBv1Qw2BnxEBuQ and instructs me to paste it into the controller when prompted. #. Use the shell: A shell opens, and I can run commands on the breakpoint from the controller session:

Connection from [::ffff:52.234.38.67]:61440
Session ready. You can enter single-line commands. Use `exit` to exit.
>> hostname
[out] fv-az1756-422
[exit: 0]
>> pwd
[out] /home/runner/work/manual-testing/manual-testing
[exit: 0]
>>

End: When I'm done, I use exit (or ^C or ^D) to exit and allow the workflow to continue; at the new prompt I can choose to wait for a new connection or end the program entirely.

Audit log: Looking at the GitHub workflow output, I see a log of what commands were run:

[2025-04-28T00:23:52Z] Breakpoint will attempt to connect to controller at 'localhost:12345', expecting controller identity of '4TXapNw9DLbnjhFx7'.
Connecting to controller at 12.34.56.78:12345
[2025-04-28T00:23:52Z] Waiting for first command...
[2025-04-28T00:23:57Z] Running command: hostname
[2025-04-28T00:23:59Z] Running command: pwd
Controller asked for normal execution to resume; exiting breakpoint.

Tip: Once the controller has exited, the secret and command line given to the breakpoint side will no longer work and will need to be replaced. That can be annoying. So until you're sure that you're done, you may want to leave the controller running.

Build, test, and release

Review changelog to ensure nothing important has been omitted
Update version in Cargo.toml
Update the changelog version (add a new release header below the Unreleased header)
cargo clippy --all-targets && cargo test && cargo doc --no-deps
cargo build --release
Do any manual testing
Commit version change
cargo publish
git tag -a vX.Y.Z -m X.Y.Z
git push && git push --tags

Manual testing

Some interesting things to enter:

(yes "out" | head -n20)& (yes " err" | head -n20 >&2) to show interleaving of stdout and stderr
echo start; sleep 10; echo stop to experiment with blocking commands
while true; do date; sleep 0.001; done for a rapid output stream (not as fast as yes)

Limitations

As breakmancer does not create a PTY, buffering can occur in some pipelines. This creates two problems:
1. All of the output happens at once. In the command for x in {1..5}; do date; sleep 1; done | head we see nothing for 5 seconds, and then 5 lines of output all at once. (The output reveals that each date invocation did indeed occur 1 second apart.)
2. If such a streaming command never terminates, the output is never sent. The command while true; do date; sleep 1; done | head will hang forever with no output.
In those examples, dropping head from the pipeline (or piping to a different binary such as cat) allows the output to stream normally.
See TODO.md for more bugs and wanted features.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.