# framewatch — Integration Guide for Agents & Other Projects
This document tells another program (or AI agent) **exactly where to call
framewatch, how to call it, and how to read what it produces.**
There are two ways to use framewatch:
1. **As a CLI tool** — run the prebuilt binary, point it at a window, read the
files it writes. *(Easiest; language-agnostic.)*
2. **As a Rust library** — embed the engine/sinks directly in a Rust app.
A machine-readable version of everything below lives in
[`dist/framewatch.json`](../dist/framewatch.json). A real example of the output
lives in [`dist/sample-session/`](../dist/sample-session).
---
## Codex-native integration
The repository ships a Codex skill at
[`../.agents/skills/framewatch/SKILL.md`](../.agents/skills/framewatch/SKILL.md).
Codex discovers repository skills under `.agents/skills`, so the workflow is
available automatically when Codex opens this checkout. Invoke it explicitly as
`$framewatch` or ask Codex to capture, inspect, or record a Windows application
window.
For use from a different repository:
1. Put a current `framewatch.exe` on `PATH` (or give Codex its absolute path),
then verify `framewatch --version` and the required subcommand's `--help`.
2. Copy `.agents/skills/framewatch/` into the target repository at the same
relative path. For a skill available in every repository, copy it to
`~/.agents/skills/framewatch/` instead.
3. Start a new Codex task if the skill does not appear immediately.
The root [`../AGENTS.md`](../AGENTS.md) is contributor guidance for working on
Framewatch itself; the skill is the reusable integration for operating the tool.
---
## 0. Where the binary is
After a release build (`cargo build --release --features "cli wgc gui record"`), the
binary is copied to:
```
dist/framewatch.exe
```
(i.e. `<repo-root>\dist\framewatch.exe` — resolve `dist/framewatch.exe` against
wherever you cloned/built this repository.)
Verify it:
```sh
dist\framewatch.exe --version # prints the crate version, e.g. framewatch 0.6.0
```
> **Platform.** Live capture is Windows-only (Graphics Capture API), compiled in
> via the `wgc` feature. `windows` and `watch` need a real window to capture.
> On non-Windows builds these commands return a clear "requires the wgc feature"
> error; the library engine still works everywhere.
---
## 1. CLI usage
### 1.1 List capturable windows
```sh
dist\framewatch.exe windows
```
Prints a table to **stdout**: `HWND`, `EXE`, `TITLE`. Use it to discover a target
(by title text, exe name, or the numeric HWND).
```
HWND EXE TITLE
1647775574 WindowsTerminal.exe Windows PowerShell
134338 Claude.exe Claude
...
```
### 1.2 Watch a window (the main command)
```sh
dist\framewatch.exe watch --title "Visual Studio Code" --out ./.framewatch
```
Selectors (choose one):
| `--title <text>` | match window title by a case-insensitive substring (literal, not a regex) |
| `--exe <name>` | match executable basename, e.g. `Code.exe` (case-insensitive) |
| `--hwnd <id>` | match the exact numeric window handle |
| `--pid <id>` | match the window owned by this **process id** — exact, so back-to-back captures never latch onto a *stale* window from an earlier run of the same exe |
> **Doing repeated captures of the same exe?** Match by `--pid` (or use `shot
> --launch`, §1.3), not `--exe` — otherwise a previous still-alive window can win.
Common knobs (all optional; sensible defaults):
| `--out <dir>` | `./.framewatch` | parent output directory |
| `--settle-ms <n>` | `350` | quiescence before declaring "settled" |
| `--value-sample-ms <n>` | `1000` | throttle for volatile-region samples |
| `--roi <X,Y,W,H>` | — | **crop** capture + detection + output to a pixel region (clip host chrome / titlebar / menu bar). Coords are relative to the captured frame's top-left. |
| `--labels-file <path>` | — | tail a newline-delimited file; each line becomes a `mark` event and captions the next frame (§1.5) |
| `--config <file>` | — | load a `framewatch.toml` base config (flags override it) |
| `-v`, `-vv` | — | log verbosity (or set `RUST_LOG`) |
**Sensitivity** — the defaults are tuned to ignore noise, which means a small
change can be coalesced away. A 24 px status banner across the top of a browser
window is about 3 % of the height and, once tiled, only trips one row:
| `--min-area-ratio <r>` | `0.002` | fraction of the frame that must change to count as meaningful activity. Lower it for a thin strip. |
| `--tile-change-threshold <n>` | `12` | per-tile luma delta (0-255) needed to call a tile changed. Lower it for low-contrast changes. |
| `--tile-grid <COLSxROWS>` | `32x18` | detection grid. A finer grid makes a small change a larger fraction of one tile. |
In practice, try `--roi` first: cropping to the strip you care about makes it
100 % of the frame and needs no threshold tuning at all. Reach for the knobs
above when you need the whole window *and* the small change.
`--roi` crops everything downstream: the saved PNGs are just that region, and
change-detection ignores motion outside it (so e.g. host window chrome won't
trigger captures). `window.rect` in the timeline still reports the full source
window. Example — clip a 1920×1040 guest area below a host titlebar/menu:
```sh
dist\framewatch.exe watch --title "QEMU" --roi 0,52,1920,1040 --until-settled --out ./.framewatch
```
**Lifecycle flags — use these to avoid coordinating two processes:**
| `--wait <secs>` | poll/retry for the target window to **appear** instead of failing instantly — so it doesn't matter whether you launch the app or `watch` first |
| `--until-settled` | exit after the **first settled frame** (deterministic one-shot: wait for the UI to settle, grab it, done) |
| `--duration <secs>` | exit after N seconds (time-bounded one-shot; also stops an idle window) |
| `--frames <n>` | exit after N images have been saved |
**Behavior:** by default `watch` **blocks** until **Ctrl+C** or the **target
window closes**, writing files incrementally (each event flushed immediately) so a
session is readable while running. With any lifecycle flag it becomes a bounded /
one-shot run. Recommended agent one-liner (no launch-order races, exits on its own):
```sh
dist\framewatch.exe watch --title "My App" --wait 15 --until-settled --out ./.framewatch
```
> **Already-static screens:** a target with no motion (e.g. a quiescent login
> screen) produces only the `initial` frame and never a `settled` event — there's
> nothing to settle from, and that initial frame *is* the stable capture. So pair
> `--until-settled` with `--duration <secs>` as a fallback bound:
> `--wait 15 --until-settled --duration 8` exits on settle if the UI animates, or
> after 8s with the stable `initial` frame if it's already static.
On start it prints (to stdout):
```
framewatch: writing session to ./.framewatch/2026-06-14T06-22-17_Code
framewatch: press Ctrl+C to stop.
```
> **Parse that first line** to learn the session directory — or compute it
> yourself (see §2.1), or just watch `<out>/` for a new subdirectory.
### 1.3 One-shot to a single file (`shot`) — best for scripted/batch capture
`shot` collapses launch → wait-for-window → one settled frame → teardown into a
single command, writes the PNG to a **path you choose**, prints that path on
stdout, and uses the **exit code** to signal success. No session directory, no
timestamped glob, no two-process orchestration.
```sh
# Launch a held program, capture its window (matched by the launched PID), kill it:
dist\framewatch.exe shot --launch "game.exe --freecam --pos 1,2,3" --out-file shot.png --timeout 25
# Or against an already-running window (no launch):
dist\framewatch.exe shot --pid 41234 --out-file shot.png
dist\framewatch.exe shot --title "QEMU" --roi 0,52,1920,1040 --out-file guest.png
```
| `--launch "<cmd>"` | spawn this program, capture **its** window (by PID), then kill it on exit. Whitespace-split; use `"..."` to group an argument. |
| `--out-file <path>` | exact PNG path to write (required, deterministic) |
| `--title/--exe/--hwnd/--pid` | selector when not using `--launch` |
| `--timeout <secs>` | overall budget to wait for the window + a settled frame (default 20) |
| `--settle-ms <n>` | quiescence to declare "settled" |
| `--roi <X,Y,W,H>` | crop (clip host chrome) |
| `--settle-best-effort` | if nothing settles before the timeout, write the latest frame anyway instead of failing |
**Contract:** on success `shot` prints the written path to **stdout** and exits
`0`. If no frame settles before `--timeout` (and `--settle-best-effort` is not
set), it writes nothing and exits **non-zero** (3) — so a script can branch on the
exit code instead of globbing. Capturing by the launched PID means repeated
captures never pick up a stale window.
> **`--launch` caveat:** the captured window must be **owned by the launched
> process**. A normal `game.exe` qualifies; "relauncher" apps that hand off to a
> separate process (e.g. Win11 Notepad/Calculator as Store apps) do not — for
> those, launch the app yourself and pass `--pid`. Statically-quiescent targets
> need `--settle-best-effort` (only `initial` is produced — see §2.2).
```sh
# scripted use:
if path=$(framewatch shot --launch "game.exe --scene city" --out-file city.png --timeout 30); then
echo "captured $path"
else
echo "never settled" >&2
fi
```
### 1.4 Label what the application is doing (`mark`, `--labels-file`)
A capture says *when* the window changed. It cannot say what the application
thought it was doing. Marks close that gap, and they land in the same
`timeline.jsonl`, so nothing downstream has to align two clocks.
**From a script or an agent**, at any time, including while `watch` is running:
```sh
dist ramewatch.exe mark --label "before-checkout"
dist ramewatch.exe mark --label "signed-in" --json '{"user":"agent.buyer.01"}'
```
With no `--session` it targets the newest session under `--out` (default
`./.framewatch`). The line is appended by `mark` itself, so it survives whether or
not a watcher is running; a live watcher additionally attaches the label to the
next frame.
**From the application**, with no extra process per label: append lines to a file
and let the watcher tail it.
```sh
dist ramewatch.exe watch --title "My App" --labels-file ./console.log
```
Each line becomes one mark. A plain line is the label. A JSON object is kept whole
in `data`, and its `note`, `label`, or `kind` field becomes the label — so an app
can append the event objects it already emits without reshaping them. Partial
lines are held until the newline arrives, and the tail starts at the end of an
existing file so a re-run does not replay yesterday's labels.
A Playwright bridge is three lines:
```js
page.on('console', (message) => appendFileSync('console.log', `${message.text()}
`))
```
**Timing.** Followed files are read at the moment a frame is captured, so a label
written before a frame lands on *that* frame, not the one after it. Labels
arriving after the last frame are still written when the session closes.
**What you get.** The next captured frame carries
`marks_since_last_frame: ["before-checkout"]`, and when exactly one mark preceded
it the file is named `frames/000004_settled_before-checkout.png`. Several marks
before one frame all appear in the array, but the frame keeps its plain name —
one label is a name, several are ambiguous.
### 1.5 GUI picker / ROI editor (for humans)
```sh
dist\framewatch.exe gui
```
Pick a window, drag-draw `Spinner` / `Volatile` / `Watch` / `Ignore` regions,
tune sliders, and **Save config & ROIs** (writes `framewatch.toml`). Hand-marking
the spinner means zero guesswork in detection. Then feed that `framewatch.toml`
to `watch --config`.
---
## 2. The output contract (what an agent reads)
A single run produces one **session directory**:
```
<out_dir>/<session_id>/
├─ session.json # manifest: target, time range, config, ROI hints, counts
├─ timeline.jsonl # one JSON event per line, chronological ← the main feed
├─ README_FOR_AGENT.md # this contract, in brief
└─ frames/
├─ 000000_initial.png
├─ 000003_settled.png
└─ ...
```
### 2.1 `session_id`
Format: `"%Y-%m-%dT%H-%M-%S_<exe-stem>"`, e.g. `2026-06-14T06-22-17_Code`.
### 2.2 How to consume (recommended algorithm)
1. Read `session.json` for the target app, time range, config, and ROI hints.
2. Stream `timeline.jsonl` line by line (each line is a complete JSON object).
3. For most purposes, **only open images** for events with
`kind == "settled"` or `kind == "busy_end"` — those are stable, meaningful
states. Use the `note` and `coalesced_frames` of `busy_start` / `value_sample`
events for timing context without opening images.
4. `coalesced_frames` tells you how much activity each saved image represents.
A whole "run the tests" workflow collapses to ~4 lines / 2 images instead of ~75
screenshots.
### 2.3 `timeline.jsonl` — event schema
One object per line. Example (`settled` event):
```jsonc
{
"session_id": "2026-06-14T06-22-17_Code",
"seq": 1, // monotonic event index
"id": "f000001", // stable id = "f" + zero-padded seq
"kind": "settled", // see kinds below
"wall_time": "2026-06-14T06:22:17.826Z", // UTC timestamp of the frame
"elapsed_ms": 198, // ms since session start
"image": "frames/000001_settled.png", // path relative to session dir, or null
"window": {
"title": "...", "exe": "Code.exe", "class": "Chrome_WidgetWin_1",
// rect is [x, y, WIDTH, HEIGHT] (NOT [left, top, right, bottom]); virtual-desktop
// pixels, so x/y can be negative or large on multi-monitor setups.
"hwnd": 67890, "rect": [x, y, w, h], "dpi": 96, "foreground": true
},
"change": {
"changed_tiles": 0, "tile_grid": [32, 18], "area_ratio": 0.0,
"bboxes": [[x, y, w, h], ...], // pixel boxes of changed clusters
"dhash": "c02b2b2b2b23c001", // present only when an image was saved
"hamming_to_prev_emit": 24 // dHash distance to previous saved image
},
"busy": { "active": false, "regions": [{ "label": "test-runner", "active": false }] },
"timing": { "since_prev_emit_ms": 198, "active_for_ms": 165, "quiescent_for_ms": 165 },
"coalesced_frames": 5, // frames observed & collapsed since prev image
"note": "Settled after 0.17s of activity in 0 region(s).", // human-readable
"marks_since_last_frame": ["before-checkout"] // app labels, omitted when empty
}
```
Fields that are absent when not applicable are simply omitted (`image` is `null`
for image-less events; `dhash` / `hamming_to_prev_emit` / `timing.*` appear only
when known).
**Mark lines.** A line whose `kind` is `"mark"` is an application label, not a
capture, and carries none of the window/change/busy/image fields:
```jsonc
{
"session_id": "2026-06-14T06-22-17_chrome",
"kind": "mark",
"wall_time": "2026-06-14T06:22:19.104Z",
"elapsed_ms": 1485,
"note": "before-checkout",
"data": { "kind": "annotation", "seq": 4 } // present when the label came from JSON
}
```
Read them for what the application thought it was doing. When a mark arrived
while the watcher was running, the *next* frame repeats the label in
`marks_since_last_frame` and — when exactly one mark preceded it — is named after
it (`frames/000004_settled_before-checkout.png`). Prefer that to matching
timestamps yourself.
### 2.4 Event kinds
| `initial` | ✅ | First frame of the session (the starting state). |
| `busy_start` | — | A spinner/animation region started. Activity is underway. |
| `busy_end` | ✅ | The spinner stopped. `coalesced_frames` = frames collapsed. **Open this image.** |
| `settled` | ✅ | The window went quiet after activity — the "money frame". **Open this image.** |
| `value_sample` | — | A throttled sample of a volatile region (counter/progress). Usually note-only. |
| `transition_start` | — | Activity began (off by default). |
| `manual` | ✅ | A user/host-requested capture. |
| `mark` | — | Not a capture: an application label (see above). |
### 2.5 `session.json` — manifest
```jsonc
{
"session_id": "...",
"tool": "framewatch 0.6.0",
"target": { "title": "...", "exe": "...", "selected_via": "cli" },
"started_at": "...Z",
"ended_at": "...Z", // set on clean shutdown
"config": { "settle_ms": 350, "tile_grid": [32,18], "dedup_hamming": 8, "value_sample_ms": 1000 },
"roi_hints": [ { "kind": "spinner", "label": "...", "rect_norm": [x,y,w,h] } ],
"counts": { "frames_observed": 5123, "images_saved": 64, "events": 92 },
"timeline": "timeline.jsonl"
}
```
---
## 3. Optional: pre-annotate regions with `framewatch.toml`
Hints make detection faster and more accurate. ROIs are in **client-normalized**
coordinates (`0.0..1.0`), so they survive resizes/DPI changes.
```toml
target = { title = "Visual Studio Code" }
out_dir = "./.framewatch"
settle_ms = 350
value_sample_ms = 1000
[[rois]]
rect_norm = [0.02, 0.94, 0.04, 0.05]
[[rois]]
kind = "ignore"
label = "clock"
rect_norm = [0.92, 0.0, 0.08, 0.03]
```
- `spinner` — busy indicator; its changes never trigger a save (only busy edges).
- `volatile` — fast-changing value; sampled on a throttle, not saved per change.
- `watch` — lowered threshold; always counts as meaningful.
- `ignore` — excluded from diffing entirely (clocks, cursors, the capture border).
Run it: `framewatch.exe watch --config framewatch.toml`.
---
## 4. Embedding as a Rust library
Add framewatch as a path/git dependency. The **core engine + sinks pull no
Windows or GUI deps**; opt into live capture with the `wgc` feature.
```toml
# Cargo.toml of the consuming project
[dependencies]
# Path is relative to the consuming crate (adjust to where this repo lives),
# or use a git dependency once published.
framewatch = { path = "../framewatch", default-features = false }
# For live Windows capture, also enable wgc:
# framewatch = { path = "../framewatch", default-features = false, features = ["wgc"] }
# Once on crates.io:
# framewatch = { version = "0.1", default-features = false, features = ["wgc"] }
```
### 4.1 One-call convenience
```rust
use framewatch::{Config, Target, DirectorySink};
let config = Config::builder()
.target(Target::ByTitleRegex("Visual Studio Code".into()))
.out_dir("./.framewatch")
.settle_ms(350)
.build()?;
let sink = DirectorySink::new(&config)?; // writes the session directory
framewatch::watch(config, sink)?; // blocks until window closes (needs `wgc`)
```
### 4.2 Drive the loop yourself / use a custom sink
```rust
use framewatch::{Config, Target, Engine, CaptureBackend, ControlFlow, Sink, ChannelSink, SystemClock};
let config = Config::builder().target(Target::ByExe("Code.exe".into())).build()?;
let mut engine = Engine::new(config.clone(), SystemClock);
let mut backend = framewatch::default_backend(&config)?; // WGC on Windows + wgc
let (mut sink, rx) = ChannelSink::unbounded(); // receive events in your app
sink.on_event(&event).ok(); // or inspect `event.meta` / `event.image` directly
}
ControlFlow::Continue
})?;
```
### 4.3 Key public API
| `framewatch::enumerate_windows()` | list capturable windows (`Vec<WindowInfo>`) |
| `framewatch::watch(config, sink)` | capture target into a sink until interrupted |
| `framewatch::default_backend(&config)` | platform capture backend (`Box<dyn CaptureBackend>`) |
| `framewatch::Engine` | pure `(state, RawFrame, now) -> events` state machine |
| `framewatch::Config` / `ConfigBuilder` | configuration (also `Config::from_toml_path`) |
| `framewatch::DirectorySink` / `ChannelSink` / `CompositeSink` | outputs |
| `framewatch::CaptureEvent` / `CaptureMeta` | the event + its serializable metadata |
| `framewatch::MockBackend` | replay frames/PNGs (tests, CI, demos) |
Full API docs: `cargo doc --open` (or docs.rs once published).
---
## 5. Quick reference card
```sh
# discover
dist\framewatch.exe windows
# capture (blocks; Ctrl+C to stop) — pick ONE selector
dist\framewatch.exe watch --title "<substring>" --out ./.framewatch
dist\framewatch.exe watch --exe "Code.exe"
dist\framewatch.exe watch --hwnd 67890
dist\framewatch.exe watch --config framewatch.toml
# one-shot, no launch-order coordination needed:
dist\framewatch.exe watch --title "My App" --wait 15 --until-settled --out ./.framewatch
dist\framewatch.exe watch --title "My App" --wait 15 --duration 8 --out ./.framewatch
# crop to a region (clip host chrome) — e.g. a guest area below a titlebar/menu:
dist\framewatch.exe watch --title "QEMU" --roi 0,52,1920,1040 --wait 15 --until-settled --duration 8 --out ./.framewatch
# one settled frame to a chosen file (launch + capture + kill; prints path, exit code):
dist\framewatch.exe shot --launch "game.exe --freecam" --out-file shot.png --timeout 25
dist\framewatch.exe shot --pid 41234 --out-file shot.png # exact window, no stale match
# label what the app is doing, from anywhere (safe while `watch` runs):
dist ramewatch.exe mark --label "before-checkout"
dist ramewatch.exe mark --label "signed-in" --session ./.framewatch/2026-06-14T06-22-17_chrome
# let the app write its own labels; the watcher tails the file:
dist ramewatch.exe watch --title "My App" --labels-file ./console.log --out ./.framewatch
# notice a small change (a 24px status banner, say) that the defaults coalesce away:
dist ramewatch.exe watch --title "My App" --min-area-ratio 0.0005 --tile-change-threshold 8
# record a window to video while narrating, then bundle an LLM package (see §6):
dist\framewatch.exe record --title "My Game" --duration 60
# then read: <out>/<session_id>/timeline.jsonl (+ session.json, frames/*.png)
# open images only for kind == "settled" | "busy_end"
# (for `record`: read <out>/<session_id>/PROMPT.md — see §6)
```
---
## 6. Recording packages (`record`)
`record` is the opposite of `watch`: instead of a deduped story of "money frames",
it **continuously records** one window to video while the user narrates into the
microphone, then transcribes the narration locally and writes a package an LLM
can act on. Build with `--features "wgc record"`; **`ffmpeg` must be on PATH**.
For a crates.io installation that includes this subcommand, use:
```sh
cargo install framewatch --features "wgc record"
framewatch transcriber setup # optional preflight; record also provisions automatically
```
No separate speech-to-text setup is required. The first recording that needs a
transcript automatically downloads the pinned whisper.cpp runtime and `base.en`
model (~150 MiB) into the user cache, verifies SHA-256 checksums, and reuses them
for later recordings. Set `FRAMEWATCH_WHISPER_DIR` to override the cache parent,
use `--no-transcribe` to skip setup/transcription, or use `--transcribe-cmd` to
override the managed engine.
```sh
# Record until Ctrl+C (or --duration), transcribing with managed Whisper:
dist\framewatch.exe record --title "My Game" --duration 60 --out ./.framewatch
# Or override it with another local command ({audio}/{output} are substituted):
dist\framewatch.exe record --title "My Game" --duration 60 \
--transcribe-cmd "other-transcriber -f {audio} -o {output}"
# Or skip transcription (video + audio only):
dist\framewatch.exe record --pid 41234 --no-transcribe
```
Selectors (`--title/--exe/--hwnd/--pid`), `--launch`, `--out`, `--roi`, `--wait`,
and `--duration` behave exactly as in `watch`/`shot`. Extra options: `--fps`
(default 30), `--mic <name>` (default input device), `--no-audio` (record
video-only), and the transcription choices above. Stop with **Ctrl+C** (the mp4
is finalized cleanly) or `--duration`. If no microphone is available, recording
falls back to video-only automatically.
For `record`, `--duration` is measured from encoder startup, after the target
window resolves and its first frame arrives; `--wait` time does not consume the
requested recording duration. Odd window/crop dimensions are padded by one pixel
when needed so H.264/yuv420p output always has even dimensions.
`record --launch` waits up to 15 seconds for the launched window by default;
an explicit `--wait` or nonzero config `wait_ms` overrides that default.
### 6.1 Package layout
```text
<out_dir>/<session_id>/
├─ PROMPT.md # START HERE — the task prompt, transcript inline
├─ recording.mp4 # the window video (narration muxed in)
├─ audio.wav # the raw microphone narration
├─ transcript.json # { language?, duration_ms, segments: [{start_ms,end_ms,text}] }
├─ transcript.srt # the same transcript as SubRip subtitles
├─ recording.json # manifest (see below)
└─ README_FOR_AGENT.md
```
`session_id` has the same `%Y-%m-%dT%H-%M-%S_<exe-stem>` format as a capture session.
### 6.2 How to consume
1. **Read `PROMPT.md`.** It is self-contained: it embeds the full timestamped
transcript inline, so a text-only model can follow the instructions with no
tools.
2. **If you can see video,** also use `recording.mp4`. Every transcript segment's
`start_ms`/`end_ms` is measured **from the start of the video**, so a spoken
instruction maps to a specific on-screen moment. Ingest the mp4 directly, or
pull a still frame at a timestamp (`-ss` is in **seconds** = `start_ms` / 1000):
```sh
ffmpeg -ss 12.500 -i recording.mp4 -frames:v 1 frame.png ```
### 6.3 `transcript.json` schema
```jsonc
{
"language": "en", // omitted if unknown
"duration_ms": 64200,
"segments": [
{ "start_ms": 1250, "end_ms": 4800, "text": "First, open the settings panel." },
{ "start_ms": 5000, "end_ms": 8200, "text": "See this dropdown — set it to manual." }
]
}
```
### 6.4 `recording.json` — manifest
```jsonc
{
"session_id": "2026-06-15T00-30-31_game",
"tool": "framewatch 0.6.0",
"kind": "recording", // distinguishes this from a capture session.json
"target": { "title": "My Game", "exe": "game.exe", "selected_via": "cli" },
"started_at": "...Z", "ended_at": "...Z",
"video": { "path": "recording.mp4", "container": "mp4", "codec": "h264",
"fps": 30.0, "width": 1920, "height": 1080, "duration_ms": 64200 },
// "audio" is omitted entirely for a video-only recording (no microphone):
"audio": { "path": "audio.wav", "sample_rate": 48000, "channels": 1, "duration_ms": 64300 },
"transcript": { "path": "transcript.json", "srt": "transcript.srt",
"engine": "whisper.cpp", // "whisper.cpp" | "command" | "none"
"model": "base.en", "segment_count": 2, "language": "en" },
"artifacts": ["recording.mp4","audio.wav","transcript.json","transcript.srt",
"recording.json","PROMPT.md","README_FOR_AGENT.md"]
}
```
### 6.5 Transcription engines
| Managed Whisper (default) | _(none)_ | first-use HTTPS download (~150 MiB), then local cache |
| External override | `--transcribe-cmd "<cmd>"` | any compatible local transcriber |
| None | `--no-transcribe` | — (empty transcript; video + audio only) |
The managed runtime/model are pinned and checksum-verified before use. They are
downloaded rather than embedded in the crate, keeping `cargo install` small while
making the default recording workflow self-configuring.
There is no microphone, or you don't want one? Recording is video-only then: it
warns and writes a package with no `audio.wav` and an empty transcript (and the
manifest omits the `audio` block). `--no-audio` opts out of mic capture explicitly.
The `--transcribe-cmd` template is whitespace-split (quotes group args). `{audio}`
is replaced with the WAV path and `{output}` with a framewatch-chosen output base
path; if neither placeholder appears, the WAV path is appended and framewatch reads
the command's **stdout**. The command must emit **framewatch transcript JSON**
(`{ "segments": [...] }`) or **SubRip (SRT)** — framewatch detects which.
A live recording (real window + mic + ffmpeg) is Windows-only; the transcript /
package / prompt code is pure and runs everywhere.