PaperBoy
A Rust-native alternative to Postman. Collections are Hurl
.hurl files and environments are .vars files (KEY=value), so everything is
plain text you can commit, diff and review. No hosted service, no telemetry,
nothing leaves your machine.
One binary, three front-ends over the same core:
| Front-end | How | Notes |
|---|---|---|
| Terminal UI | paperboy |
The default. Full client. |
| Graphical UI | paperboy -g |
eframe/egui. Behind the gui Cargo feature. |
| Headless runner | paperboy -c collection.hurl |
For scripts and CI. Also runs reports (-r). Exits non-zero on failure. |
Collections, environments, themes and the git workflows behave identically in all three.
- Install
- Concepts
- Terminal UI
- Graphical UI
- Environments and secrets
- Git remotes
- Importing from Postman
- Headless runner
- Changelog
Install
Prebuilt binaries
The quickest route, and the one that needs no Rust toolchain and none of the
build prerequisites below. Download from the latest
release, unpack, and put
paperboy somewhere on your PATH.
| File | Platform |
|---|---|
paperboy-<version>-linux-x86_64.tar.gz |
Linux x86-64 — glibc 2.35+ (Ubuntu 22.04+, Debian 12+, Fedora 36+) |
paperboy-<version>-macos-arm64.tar.gz |
macOS, Apple Silicon |
paperboy-<version>-windows-x86_64.zip |
Windows x86-64 |
paperboy-<version>-linux-x86_64-headless.tar.gz |
Linux x86-64, headless runner only |
Every file except -headless is the same binary in all three modes: it runs
-c headless, the terminal UI, and --gui. There is no separate GUI download,
and choosing the full build costs nothing if you never open it — the graphical
front-end loads its display libraries only when --gui is passed, so the binary
still runs on a machine that has none.
The Linux archives also need the system libxml2 (libxml2.so.2), which
every desktop distribution installs and a minimal container image does not —
if that is where you are heading, use the image below. The Windows build links
libxml2 in and needs nothing; the macOS build uses the copy macOS itself ships
at /usr/lib, so it needs nothing either. RHEL 9 and Rocky/Alma 9 ship glibc
2.34 and so are not covered by the Linux archive; build from source or use
the container there.
Each release publishes SHA256SUMS beside the archives:
Container
For CI images and anywhere a slim base has no system libxml2 — which the Linux binaries above need:
The image is the headless build, so it takes the same arguments and writes the
same reports as any other PaperBoy; /work is its working directory, which is
why the mount above is all the setup there is.
From source
--locked is recommended: it builds the dependency versions PaperBoy was
tested against rather than re-resolving to whatever is newest. (A yanked
arrayref release took plain cargo install down on 2026-08-20 while
--locked kept working — and it took the terminal-only build with it, since
Cargo resolves optional dependencies whether or not their feature is on.)
The gui feature is opt-in because eframe/winit/wgpu roughly double the
dependency tree. Both builds share the same state file, so you lose nothing by
switching. Running --gui without it prints the command to install one with it.
--no-default-features turns the terminal UI off and leaves the headless
runner, which is the shape wanted for CI images and Docker containers: it drops
40 dependencies and about a third of PaperBoy's own source, none of which a
scripted -c/-r run would ever execute. The resulting binary takes the same
arguments and writes the same reports; only the interactive front-end is
missing, and running it with no arguments says so rather than doing nothing.
Build prerequisites
Only when building from source — the prebuilt binaries and the container need none of this.
Five things Cargo can't fetch for you:
| Platform | Command |
|---|---|
| macOS | xcode-select --install then brew install pkg-config libxml2 |
| Debian/Ubuntu | sudo apt install build-essential pkg-config libxml2-dev libclang-dev perl |
| Fedora/RHEL | sudo dnf install pkgconf-pkg-config gcc make perl libxml2-devel clang-devel |
| Arch | sudo pacman -S pkgconf base-devel perl libxml2 clang |
| Alpine | sudo apk add build-base pkgconfig perl libxml2-dev clang-dev |
| Windows (MSVC) | see Windows below |
libxml2 + pkg-config because hurl/hurl_core depend unconditionally on
the libxml crate (Hurl's XPath asserts are libxml2's XPath engine), and that
crate is a binding to a system libxml2 rather than a vendored copy.
libclang because bindgen generates those bindings at build time. A C
compiler, perl and make because PaperBoy pulls curl in directly with
static-curl/static-ssl, so libcurl and OpenSSL are compiled from vendored
sources — which is why there is no libcurl-dev row above. The gui feature
adds no build-time requirement; its X11/Wayland libraries are dlopened at
runtime.
On Windows (MSVC) the same needs are met by different tools, so it gets its own section below.
On macOS the Command Line Tools cover everything except pkg-config, which is
the failure most people hit. The second is subtler: macOS does ship libxml2,
but the SDK carries only its headers and a linker stub — there is no
libxml-2.0.pc anywhere, so pkg-config cannot discover the system copy and
the build fails even though the library is right there. The simple fix is
Homebrew's keg-only libxml2, pointed at explicitly:
That bakes an /opt/homebrew/... path into the binary, which is fine for a
local build and not for one you intend to hand to anyone else. The released
macOS archive instead writes a one-off libxml-2.0.pc pointing at the SDK, so
it links /usr/lib/libxml2.2.dylib and runs on any Mac; see the
Point pkg-config at the macOS SDK's libxml2 step in
.github/workflows/release.yml if you need to
do the same. Note that static linking is not an alternative here: Homebrew's
bottle ships no libxml2.a, and even with one, Apple's linker has no
-Bstatic, so -lxml2 against a directory holding both takes the .dylib.
Windows
Nothing here is unusual, but almost every piece arrives from a different place than it does on Unix, so in order:
-
Visual Studio Build Tools, workload Desktop development with C++ — this is
cl.exe,nmakeand the Windows SDK.cargofinds them through the registry, so they don't have to be onPATH; building from an x64 Native Tools Command Prompt for VS is still the least surprising way. -
libxml2, from vcpkg. On MSVC the
libxmlcrate asks vcpkg and nothing else — there is no pkg-config fallback — and it finds vcpkg throughVCPKG_ROOT(or a previousvcpkg integrate install), not by looking forvcpkg.exeonPATH:git clone https://github.com/microsoft/vcpkg C:\vcpkg C:\vcpkg\bootstrap-vcpkg.bat setx VCPKG_ROOT C:\vcpkg C:\vcpkg\vcpkg install libxml2:x64-windows-static-mdvcpkg.exeis called by its full path on purpose: bootstrapping builds it inside the tree and adds nothing toPATH, so a barevcpkg installin the shell that just ran the bootstrap fails with "not recognized". In PowerShell that is true even from insideC:\vcpkg— unlikecmd, PowerShell does not search the current directory, so there it is.\vcpkg install …. If your tree is somewhere with a space in the path, quote it ("C:\Program Files\vcpkg\vcpkg" install …) and, in PowerShell, put&in front.The triplet is the part worth reading twice.
x64-windows-static-mdis a static libxml2 built against the dynamic CRT, which is what Rust's MSVC targets link — it is what vcpkg-rs asks for by default, and installing the plainx64-windowsport instead is the usual first wrong turn. Usearm64-windows-static-mdon ARM64. (VCPKGRS_DYNAMIC=1switches to the DLL ports, andVCPKGRS_TRIPLEToverrides the choice outright —build.rshonours both when it tells you which triplet is missing.) -
LLVM, Strawberry Perl and NASM:
winget install --id LLVM.LLVM -e --accept-package-agreements --accept-source-agreements winget install --id StrawberryPerl.StrawberryPerl -e --accept-package-agreements --accept-source-agreements winget install --id NASM.NASM -e --accept-package-agreements --accept-source-agreementsOne package per line, with the agreement prompts pre-answered, so the block can be pasted in one go: a tool that stops to ask something reads the next pasted line as the answer and quietly swallows a command. (
winget install A B Calso works, on v1.5 and later.) Orchoco install llvm strawberryperl nasm -y, from an Administrator prompt. LLVM supplieslibclang.dllfor bindgen — if it isn't found, point at it withsetx LIBCLANG_PATH "C:\Program Files\LLVM\bin". Perl and NASM are for the vendored OpenSSL build (perl Configure, then NASM for the crypto assembly), and NASM's installer does not add itself toPATH, which OpenSSL needs it to be — so addC:\Program Files\NASMyourself. -
A placeholder icon for
hurl's build script. This one is an upstream bug rather than a missing tool, and it stops the build outright:$cargoHome = if ($env:CARGO_HOME) { $env:CARGO_HOME } else { "$env:USERPROFILE\.cargo" } $dir = Join-Path $cargoHome "registry\src\bin\windows" New-Item -ItemType Directory -Force -Path $dir | Out-Null Copy-Item paperboy.ico (Join-Path $dir "logo.ico") # any .ico will dohurl8.0.1 embeds a Windows icon from../../bin/windows/logo.ico, a path that only resolves inside hurl's own git checkout; the published crate ships nothing outside its own directory, sorc.exereportsRC2135: file not foundand the build script panics. It is not conditional, so no Windows build of anything depending onhurl8.0.1 can succeed without this.rc.exeresolves the path from the build script's working directory, which Cargo sets to the package root, so../../lands inregistry\src— hence the command above. The icon is linked into the finished executable, so use one you are happy to see onpaperboy.exe.Upstream has fixed it by skipping the icon when the file is absent, but the fix is unreleased and 8.0.1 is still the newest crate. PaperBoy's own
build.rscannot paper over it: Cargo runs a dependency's build script before its dependent's, so hurl's has already failed by the time ours would run. The prebuilt Windows binary is built this way and is unaffected — this applies only to building from source.
build.rs checks all of this before the build starts and names the triplet and
the tree it actually looked in, so a wrong VCPKG_ROOT reads as one line rather
than as a link error several minutes later.
PaperBoy's build.rs checks for all five before the build gets going and fails
with the package-manager command your machine actually wants (it detects
Homebrew/MacPorts, apt, dnf, yum, zypper, pacman, apk). It fails rather than
warns because Cargo runs build scripts concurrently and doesn't replay their
warnings — a warning lands dozens of Compiling … lines above the real error.
It never installs anything, and it can't prompt: a build script has no terminal.
Checks that could be wrong (cross-compilation, target-suffixed PKG_CONFIG_*,
the libclang heuristic) only warn, and PAPERBOY_SKIP_DEP_CHECK=1 disables it
entirely.
If you have a libxml2 lying around and want to skip pkg-config and bindgen,
libxml's build script takes an explicit path — --config reaches transitive
build scripts:
Vendoring libxml2 instead isn't possible from here: the crate has no vendored
build, no libxml2-src exists, and it declares no links key, so there's no
DEP_* channel to reach into it with. It would have to be added upstream.
From a checkout:
PaperBoy builds in four shapes — headless, terminal, terminal + GUI, and GUI
alone — and CI checks all four, because a configuration nothing builds is a
configuration that stops compiling. Each must also stay warning-free; the
dead-code analysis is carried by the two shapes that include the terminal UI
(see the note at the top of src/main.rs for why).
Concepts
- Collection — a
.hurlfile: an ordered list of requests with method, URL, headers, cookies, body/form fields and optional[Captures]/[Asserts]. Postman.jsonexports open directly and are converted on the way in. - Environment — a
.varsfile ofKEY=valuelines supplying{{ VAR }}values. See Environments and secrets. - Workspace — a folder of collections, reports and environments, browsed through a single tab as a filesystem tree.
- Report — a
.trailfile: a PaperTrail script that runs requests from a collection, loops over environments or data, and writes CSV/JSON/HTML/XLSX. Editable as text or as blocks; runnable from the UI or headlessly. - Generated value — a
# [Gen]row: an expression evaluated just before a request is sent, supplying the nonces, timestamps and signatures a pre-request script used to. See Generated values. - Scratch Space — tab 0. A collection with no file behind it until you save it.
- Request names encode folders.
Auth/Tokens/Refreshbrowses as a folder path; Postman's folder structure imports into this automatically.
Terminal UI
Press ? or F1 for the full, current key list. The essentials:
| Key | Action |
|---|---|
Tab / Shift+Tab |
Move focus between panes |
↑/↓, j/k |
Move selection |
←/→, h/l |
Switch tabs / scroll list text horizontally |
Enter |
Edit the selected request (or descend into a folder row) |
Shift+R |
Edit as raw Hurl text — for anything the form doesn't expose |
F5, Ctrl+Enter |
Run the current request |
Alt+F5 |
Run the whole collection in one Hurl execution |
n / b |
New request / set the base URL |
f / s |
File menu / Settings menu |
Ctrl+S |
Save the open report, else the active collection |
[ / ], PageUp/PageDown |
Previous / next tab |
Ctrl+Shift+←/→ |
Reorder the active tab |
x / u |
Delete / undo — requests, tabs and environments each keep their own undo stack |
o / x (report results) |
Sort by the cursor's column / re-run the cursor's row (or every marked row) — see Reading a report's results |
Space / Shift+X (report results) |
Mark the cursor's row to re-run with the others / re-run every failed row on show |
/ |
Find a request anywhere in the collection (whole tree on a Workspace tab) |
Alt+↑/↓ |
Reorder requests — the order Alt+F5 and the CLI follow |
m / c |
Move / copy a request to another collection in the workspace |
p (Requests) |
Link an environment to this collection |
a (Response) |
Build an assert or a capture from what came back |
a (Env pane) |
Make an environment active |
r (Env pane) |
Retry a failed secret lookup |
w (Workspace tab) |
Reopen the file-tree picker |
+/-, </> |
Resize the response pane / left column |
q, Ctrl+C |
Quit |
In the request wizard:
| Key | Action |
|---|---|
[/], PageUp/PageDown |
Switch section tab (All│Headers│Cookies│Queries│Options│Form│Body│Asserts│Captures│Reports│Generated). [/] only when focus isn't on a text field, so brackets stay typable |
Alt+1–9, Alt+0 |
Jump straight to a section (Alt because most terminals can't report Ctrl+digit) |
Ctrl+↑/↓ |
Previous / next section |
Ctrl+D / Ctrl+E |
Delete a row / toggle its enabled checkbox |
Ctrl+Z / Ctrl+Shift+Z |
Undo / redo within the focused text cell |
← from a Key cell |
Reach the enabled checkbox — it's the leftmost column |
Ctrl+F or Enter on a File value |
Open a file picker |
F2, Ctrl+Enter |
Save |
Esc |
Cancel (asks first if there are unsaved edits) |
Worth knowing:
[Form]/[Multipart],[Cookies],[Captures],[Asserts]and[BasicAuth]are all editable as tables in the wizard; the expected status is just an assert (status == 200). Saving picks the right Hurl section: all-text fields become[Form], any file field promotes it to[Multipart]. File paths are colour-coded by whether they resolve and are readable (relative to the collection's directory, matching where Hurl looks). ABase64 Filefield is encoded at send time behind a configurable prefix, sodata:image/png;base64,yields a ready-made data URI.- Asserts and captures can be built from a response. With a reply on
screen,
ain the Response pane opens a two-step palette: pick a value the server actually sent — status, duration, any header, any value in the JSON body, listed beside what it currently is — then pick what to say about it. The rows are the Hurl lines themselves (jsonpath "$.data.token" == "ey…"), so what you choose is what gets written. Typing narrows the list, and anything selected in the body pre-fills the filter. The last row on every value is keep it in a variable, which adds a[Captures]row under a name taken from the field itself — the fastest way to chain one request into the next. Choosing the status sets theHTTP <status>line rather than adding a competing assert. - The request preview substitutes
{{ VAR }}and colours each by status — green loaded, cyan literal, orange loading, red missing — while the editor keeps the original text. Secrets are masked as eight dots. - A JSON body can be laid out again —
pin the main view,Alt+Pin the request editor, Format in the GUI. The body is re-indented in place rather than round-tripped through a JSON parser, so comments, a bare{{ TEMPLATE }}standing where a value goes, number spelling (1.50,1e3, a 19-digit id) and duplicate keys all survive; a body that isn't JSON is refused rather than mangled. - Captures are visible in two places, because there are two questions. The
Response pane's Captures tab lists what that request captured when it
last ran, marking a value a later run has since replaced as superseded. The
live pool — what
{{ VAR }}is worth right now — is in the terminal UI'svVariables popup and the GUI's Environments panel ▸ Variables tab. Both list the bound environment's variables and the capture pool together, in the precedence substitution applies: the environment overridden by the pool, so an environment row a capture is shadowing is marked as such rather than quietly showing a value that isn't the one being sent. Values are masked by default (m/ Reveal) since a capture is usually a token; copying still yields the real value, and# [Gen]computed values are never listed. - Sections start empty and dropdowns only auto-open on an empty cell, so arrowing through a populated table doesn't keep reopening them.
- Settings ▸ Preferences persists: confirm on exit/clear, confirm before
deleting an environment or a request, always-save-when-prompted, whether
Alt+F5runs the collection in batch mode (chaining cookies and captures), whether Esc discards request edits without asking (off by default), and the default Request view (JSON or Hurl). - Settings ▸ Theme ships three presets — Britannia, Parisian Purple,
Dannebrog — one per UI language (English/French/Danish), and follows the
language until you pick one by hand.
Ctrl+Nclones a preset into an editable custom theme;Enteron a colour opens an RGB picker that previews live and auto-saves.Ctrl+Ddeletes a custom theme. - Saving. Save overwrites the file the tab came from without confirmation; Save As… always prompts, and confirms an overwrite. Every File-menu item has a bracketed mnemonic that both selects and activates it.
Graphical UI
paperboy -g, from a build with the gui feature. Feature-for-feature
equivalent to the terminal UI — same tabs, folder tree, request editor,
response viewer, environments panel, reports, theme editor, git remotes, and
the same three languages. What differs:
- Panels and result columns are resized by dragging; double-click a column border to hand it back to the automatic fit. Hand-set widths persist as long as the report keeps producing the same columns.
Tab/Shift+Tabcycle panels in the terminal UI's order. The focused request list is arrow-driven:Home/End,Enterto run,F2to rename,Deleteto delete,Ctrl+Zto undo. A Workspace tree addsLeft/Rightto collapse and expand, andPageUp/PageDownfor ten rows.- Single-letter shortcuts don't carry over — in a desktop window those keys
are text. Globally:
F5/Ctrl+Enterrun,Ctrl+S/Ctrl+Shift+Ssave,Ctrl+Wcloses,Ctrl+Zundoes a delete,Alt+Fopens the File menu,F1shows every shortcut. - Asserts and captures are built by right-clicking the response. Click a
value in the body and choose Assert this… — the field under the caret is
worked out from the raw JSON, so it works on a minified body as well as a
pretty-printed one. Right-clicking a header row does the same for that
header, and the Assert… button beside Copy opens the same builder on the
whole list of values the reply carried. The list is filterable by name or by
value, and "keep it in a variable" adds the
[Captures]row. - The File menu is grouped by verb (New / Import / Open / Save). Open ▸ Collection and Load ▸ Environment take Postman exports too — they work out what the file holds. Every dialog reopens where you left it.
- Workspaces are editable in place: New adds a collection, report or environment; drag files and folders onto another folder to move them, or onto the empty space to move them back to the root. Nothing escapes the workspace root and nothing is silently overwritten.
- Reports bind to their collection by a relative path (
../apis/billing.hurlincluded), so a workspace survives being zipped up or handed over. Thecollectiondropdown offers the report's own workspace first and hides outside collections behind a toggle. - The window remembers itself — size, every splitter you dragged, the open view, the selected report/request and the Workspace node.
The PaperTrail block editor
Reports get a Blocks view alongside Source and Results: a
drag-and-drop editor where blocks are dragged from a palette, reordered, nested
inside FOR loops (which move as one, body included) and dropped on the trash
bar to delete. The drag outline and the drop marker are both drawn as the
block's own silhouette, at its real width and indent.
Editable on the blocks: the request a step runs, its alias, response format and
SHOW(…)/HIDE(…)/STATISTICS(…) lists; a FOR loop's binder, source, roles
and PARALLEL(n) concurrency; and the report's own settings — collection,
output, environment, root, baseline, columns — in a boxed panel at the
top of the flow. Those apply to the report rather than running as a step, so
they're deliberately not blocks. output names a format (csv, json,
html, xlsx), not a filename; only the CLI's -o takes a path. Everything has
hover help, and Source is highlighted with the terminal UI's colours,
underlining whatever the parser rejected.
Desktop icon on Linux
Wayland has no per-window icon protocol, so shells match the window's app id
against an installed .desktop file. The first GUI launch writes
$XDG_DATA_HOME/paperboy/paperboy_logo.png and
$XDG_DATA_HOME/applications/paperboy.desktop (with StartupWMClass=paperboy
for X11) if they aren't already there, and never touches them again — so you
can customise them. The shell may need a rescan (log out, or restart it) to
notice. Delete both and relaunch to regenerate, which is also how you refresh
Exec= after moving the binary.
Reading a report's results
A finished — or a still-running — report shows its rows in a Results grid, with a filter bar, a summary and, for a ground-truthed report, a confusion matrix. Both front-ends behave the same way:
- Sorting is by column. Click a header in the GUI, or press
oon a column in the terminal UI: ascending, then descending, then back to run order. Numbers sort as numbers, text naturally (img2beforeimg10), and empty cells always sink to the bottom. It works mid-run too — rows still on their way have nothing to sort by, so they wait at the bottom. Sorting is a view: exports keep the run's order. - A single row can be re-run, which is what a transient failure asks for —
a dropped connection, a momentary 503 — where re-running the whole report is
the only other fix, and where in a comparison the bad row reads as a
regression that never happened. Right-click the row and choose Re-run this
row in the GUI, or press
xon it in the terminal UI. Only that row's loop iterations go again (both sides of a comparison row), with the parameters the result was run with, and the fresh answer replaces the old one in place —Result,Correct,Trend, the statistics footer and the error list included. A re-run uses the report as it was when the result was produced, so editing the script afterwards doesn't strand the result. During a run, a row that has already come back re-runs straight away beside the run; while a re-run is going, further rows are queued (↻) and go as the next batch. - Many rows can go at once. A network blip tends to spoil dozens of rows,
not one. Pick them out — ctrl-click (command on macOS) a row in the GUI,
shift-click to take everything between it and the last one picked, or
press
Spaceon a row in the terminal UI (Escdrops the lot) — then re-run them together: Re-run the N selected rows in the right-click menu, orxin the terminal UI, which takes the picked rows when there are any and the cursor's row otherwise. They go as one batch, so a report'sPARALLELsetting applies to them as it does to a run. - Or just re-run what failed. A report with a failed row offers a Failed
filter (first on the bar,
/in the terminal UI) and a Re-run the failed rows action in the right-click menu —Shift+Xin the terminal UI. It takes the failed rows the grid is showing, so a filter or a sort narrows what goes again. A row counts as failed when its request errored, or when the baseline side of a comparison did. - A comparison states both sides' statistics for the whole run. A finished
comparison carries each
SHOWn field twice — the candidate's value and, in abaseline.…column, the baseline's — soSTATISTICS(…)describes each side separately. That pairing only exists once the report collapses, so a run in flight used to show the candidate's mean alone and a long comparison had to finish before it could be judged. The live footer now carries the baseline's figures as well, as extraMean (baseline)rows beneath the candidate's and under the same column, alongside aMean (comparison)row naming the candidate's half, so neither reads as the run's overall figure (a report with one side only keeps the plainMean). They are there from the first row and they stay on a run that was stopped part-way, so a long comparison can be stopped and read. When a field was renamed, the two halves still meet: a comparison names its copied column after the query it read (baseline.dfa.TimeWait) while the statistics may be declared on the alias ("Response Time": TimeWait STATISTICS(MEAN)), and the footer puts the baseline's figures under the column that states the candidate's rather than several columns away. - A statistics panel, beside the matrices. The GUI restates the same figures next to the confusion matrix — one line per statistic, the two sides of a comparison side by side, plus the total request time each side has spent so far. The footer keeps its rows; this is a second reading of the same numbers for the reader who wants the comparison rather than the column, and it fills space a matrix would otherwise leave empty.
- Stopping a run keeps everything that came back — rows, summary, matrix,
filters and sorting — and the run can be resumed: the Resume button
beside Run in the GUI, or
ron the stopped run in the terminal UI (yresumes,nstarts again from scratch). Resuming runs only the rows that hadn't come back, into their places, and finishes the report as a whole run would. A stopped run's partial rows can't be re-run individually; resume it instead. - Filters and metrics are live. Differences, Correct, Incorrect,
Regressions, Improvements and the confusion-matrix cells (
/in the terminal UI) all select from the rows that have come back, judged as the finished report will judge them. Accuracy, Fixed/Regressed and every matrix cell are there from the first frame and fill in as rows arrive. A case with only one side back yet isn't counted as a difference — it has its own Waiting for pair filter while the run goes.
Environments and secrets
A .vars file is one KEY=value per line. Values can be:
| Form | Example | Resolved by |
|---|---|---|
| Literal | USERNAME=demo |
— |
| Process env var | BASE_URL={{ env:DEMO_BASE_URL }} |
The process environment |
| 1Password | API_TOKEN={{ op://Vault/Item/field }} |
The local op CLI |
| AWS SSM | DB_PASSWORD={{ ssm:/path/to/param }} |
Local AWS auth |
Provider references resolve in the background at load time, and the resolved
values are never persisted — state.json keeps only the reference. Every
1Password reference across every open collection resolves in a single op inject call, so you get one authorization prompt rather than one per
collection. r in the Environment panel retries a single failed entry. Editing
a value into something that looks like a reference triggers a load attempt, and
a "still secret?" checkbox decides whether the new value stays masked.
Loading a .vars file substitutes nothing on its own. It only joins the
Global Environments list. It then has to be activated on a tab: a in the
Global Environments panel (GUI: the Active button, a double-click, or
right-click → Activate).
The environment belongs to the tab, not to the app: one per tab, and activating
one leaves every other tab alone. Open the same collection in two tabs to run it
against staging and prod side by side. A collection still showing raw
{{ VAR }}, or a red "variables in this request are undefined" band, nearly
always means this step was missed on that tab.
A variable that is defined but empty is not undefined and warns about nothing
— it substitutes as an empty string. With Basic Auth that produces a
well-formed request that comes back 401.
Generated values
Some values can't be written down: a nonce, a timestamp, an HMAC over the two.
Postman uses a pre-request script; PaperBoy uses a # [Gen] block of named
expressions, evaluated immediately before the request is sent.
# [Gen] 3
# nonce = random_hex(16)
# ts = timestamp
# sig = hmac_sha256_b64(API_SECRET, concat(nonce, ts))
POST https://api.example.com/orders
X-Nonce: {{nonce}}
X-Timestamp: {{ts}}
Authorization: HMAC {{sig}}
The expressions stay in comments and the request refers to results as ordinary
{{name}} placeholders, so the file remains a plain .hurl file: stock hurl
parses it byte for byte and runs it given --variable nonce=… --variable sig=….
Nothing else could work — Hurl reads a placeholder only as far as the first
character outside A-Za-z0-9_- and discards the rest silently, so
{{ hmac_sha256(K, M) }} would be sent as the value of hmac_sha256. PaperBoy
now refuses to save such a placeholder rather than let it truncate.
The block may sit above the request line, as here, or immediately below it; both are read. PaperBoy writes it below when it saves, so a hand-written file in the other order moves its block down the first time it is saved and is otherwise unchanged.
A bare identifier is a variable reference — an environment variable, a request
parameter, or an earlier row in the same block. Calls nest. Rows are evaluated
in order and a row may only refer to one above it. Values are computed per run,
never previewed, and never written to state.json; a secret read through
{{ op://… }} is no more exposed by signing with it than by sending it.
What a block computes stays available to the rest of the session, exactly as a
[Captures] value does: sign a request, and the request after it can echo the
same {{nonce}} — including when you run it on its own. (Memory only, for the
reason above: a fresh PaperBoy computes fresh values.) A [Captures] row of
the same name is the later, more specific statement and wins.
One request per name, though. "Run All" and paperboy -c normally run one
request at a time, so each block is evaluated in its own window and two
requests may each have their own nonce. A batch run (the --batch flag,
or the Run All batch preference) is a single Hurl call over the whole file with
one variable set, so there the two share the first request's value — a
signature computed over another request's nonce. Both front-ends say so before
starting such a run, and --batch prints the warning too; the fix is usually
to not use batch.
The same applies to a name the environment already defines. Running one request at a time, a block's value overrides the environment's from that request onwards; a batch has one variable set for the whole file, so it cannot override from partway through without changing what the earlier requests send. Batch therefore leaves the environment's value in place and says which names it did that to.
counter counts within the process, not within a run: it starts at 1 the first
time it is evaluated and keeps going for as long as PaperBoy is open, so sending
the same request three times gives 1, 2, 3. It is a sequence, not a setting, and
is not saved — a restarted PaperBoy counts from 1 again.
Edit the block in the request wizard's Generated section (Alt+0), in the
GUI editor's Generated tab, or as text. Both editors offer the functions as
you type — with their arguments named — and the GUI's Function… menu lists
them all; either way the call is written at the caret, over any part-typed
name, with the caret left between the brackets. Both say what is wrong with a
row while
it is still a typo rather than leaving it to be a 401: an unknown function, the
wrong number of arguments, an expression that doesn't parse. Placeholders that a
generator will fill render in the theme's generated colour and keep their
braces, because the value doesn't exist yet.
| Time | timestamp, timestamp_ms, iso8601, date(fmt) (strftime, UTC) |
| Random | uuid, counter, random_int(lo, hi), random_hex(n), random_alnum(n), random_base64(n) |
| Encoding | base64, base64url, base64_decode, hex, urlencode, urldecode, json_string |
| Hashes | md5, sha1, sha256, sha512 |
| MACs | hmac_sha1(key, msg), hmac_sha256, hmac_sha512 |
| Text | concat(…), upper, lower, trim, split(text, sep, n), regex(text, pattern) |
| JSON | jsonpath(text, path) |
| Request | method, url, path, query, header(name), body, request_name |
jsonpath(text, path) reads a value out of a JSON document the block already
has in hand — this request's own body(), or a response an earlier request
captured whole. When the value comes straight from a response a [Captures]
row is the right tool; this is for the cases a capture can't reach, which is
anything that has to be computed from the value: signing part of a payload,
or building this request's body out of pieces of the last one's. It walks $,
.name, ["name"], [n] and [?(@.key == 'x')] — the last of which is how
you address an API that returns its fields as a list of key/value objects. A
string comes back as its text (not with the quotes still on), an object or
array as compact JSON, and null is an error rather than the four characters
null. Wildcards, recursive descent, slices and unions are refused by name
rather than half-implemented, pointing you at the [Captures] row that has
Hurl's full JSONPath: the same path meaning two different things in one
request is worse than not being able to write it.
Every hash and MAC returns lowercase hex — matching sha256sum and CryptoJS's
.toString(), so a ported Postman script lands right — and each has a _b64
variant returning standard padded Base64 and a _b64url variant returning the
URL-safe alphabet without padding — the encoding a JWT segment is made of,
where +, / and = are all wrong. The encoding is in the name rather
than a default because a signature in the wrong one is the right length,
entirely plausible to look at, and rejected with the same 401 as a wrong
secret. Note that base64(sha256(m)) is not sha256_b64(m): the first
encodes 64 hex characters, the second the 32 bytes they spell.
The Request functions read the request the block belongs to — the method as
sent, the body as it goes on the wire (no JSON comments, no switched-off
headers) — which is how a signature over "the thing I am about to send" is
written. They read the text as authored, substituted against the rows above
them: a row reading body() sees earlier rows filled in and later ones still as
{{name}}, so a value can never depend on a row that depends on it. Without a
request behind the block — the editor's live check on a row you are still
typing — they say so rather than answering with nothing, because an HMAC over a
silently empty body is a signature that authorises nothing.
split counts pieces from the end when given a negative index, so the last
segment of a path is split(path(), "/", -1) — JavaScript's .pop(), which is
the shape these scripts are written in. regex is the escape hatch for what
split can't reach: the first capture group if the pattern has one, otherwise
the whole match. Both treat "no such piece" and "matched nothing" as faults
rather than an empty answer, since that text goes on to be signed or sent.
A name is written bare, not in braces. Everywhere else in PaperBoy a
variable is {{name}}; inside a generator expression it is just name, because
an expression already names things — concat("Bearer ", TOKEN). Writing
"{{TOKEN}}" there is refused rather than accepted as a string, since a
signature over the eight characters {{TOKEN}} is the right length, entirely
plausible, and rejected with the same 401 as a wrong secret. (\{ is the
escape, for a string that really does want a brace.) A row can also be a plain
literal — expected = "APPROVED" — which is how an assert compares against a
per-request expectation: jsonpath "$.status" == "{{expected}}".
Canonicalisation is yours. PaperBoy signs exactly the bytes you assemble; it will not build a canonical request from the live headers, so AWS SigV4 and friends are out of scope. Chaining a MAC into the key of the next one isn't expressible either, since every value here is text.
The block works headlessly too. paperboy -c … evaluates each request's rows
in its own window, so a generator can read a value an earlier request captured
and two requests each get their own nonce. --batch is a single Hurl call over
the whole file and has no such window: there every block is evaluated once
before the run, a name computed by two requests takes the first one's value for
both, and the run says so before it starts.
A row that fails — unknown function, wrong arity, a name nothing defines —
reports rather than blocks the send. It binds nothing, so {{sig}} goes out
literally and comes back a loud 401, which is easier to diagnose than a
refusal.
Importing from Postman maps the dynamic variables that have an exact equivalent:
$guid/$randomUUID and $isoTimestamp become Hurl's own {{newUuid}} and
{{newDate}}, while $timestamp, $randomInt and $randomAlphaNumeric become
[Gen] rows. The rest
are renamed and listed in CONVERSION-NOTES.md as values you must supply —
guessing at $randomFirstName would send a plausible wrong value, which is
harder to notice than a request that won't run.
Worked examples — collections to import, and the .hurl file they should become
— are in examples/postman/.
Git remotes
Load and save collections, environments and whole workspaces straight from a remote, with no local clone: PaperBoy lists refs, fetches just enough history to read the file tree, and checks out only the files you actually asked for. Nothing else in the repo touches your disk, however large it is.
Loading (File ▸ Load ▸ kind ▸ From Git…): give the URL — https://… or
git@…, with an optional access token used only for that fetch (GitHub-style
https://x-access-token:<token>@host/… is handled for you) — then pick a ref
and a file, both filterable as you type. ↓ on the URL field offers your
recent URLs. Loading a collection then offers to pair an environment from the
same listing, with no second round-trip. Anything loaded from git shows a ⎇ in
its tab title and remembers its origin.
Workspaces ask which files to fetch first — .hurl and .json (default),
.hurl only, .json only, or everything — and then whether to keep the
download temporarily or copy it somewhere permanent immediately.
A temporary workspace is never cleaned up. Its files live in a temp folder for as long as the tab exists — including across a close and undo, and across restarts. They accumulate. Choose "save to a permanent location" when asked, or later via File ▸ Save ▸ Workspace ▸ Save As…, which copies the folder and stops tracking it as temporary.
Saving (File ▸ Save ▸ Collection ▸ To Git…) pushes a commit directly to the remote. The URL is prefilled from where the collection came from, so you can redirect it to a fork. You choose the in-repo path, whether the attached environment goes in the same commit, and a branch or tag:
- A branch defaults to the one you loaded from, so
Enterjust appends a commit.↓lists the remote's branches. No merge or rebase is attempted — a non-fast-forward is reported as an error. - A tag must be new. The remote is re-fetched immediately before the check, and an existing tag is always rejected with no way to force it.
The message defaults to Update <name> via PaperBoy and is editable. The author
is your git identity, or PaperBoy <paperboy@localhost> if you have none. A
branch push updates the remembered origin and clears the modified markers; a tag
push clears the markers but leaves the origin on your working branch.
To Git… only works for something loaded from git. For anything else, Save As… into your own clone and use git normally.
Importing from Postman
Already have an export? Just open it — Open ▸ Collection and Load ▸ Environment both work out what the file holds, and File ▸ Import from Postman ▸ From an exported file says so explicitly. No API key, no account.
To pull from an account, File ▸ Import ▸ Postman account… (terminal) or File ▸ Import from Postman ▸ From my Postman account… (GUI). Give it an API key, pick a workspace, choose what to bring and where, and the result opens as a workspace. Paste a workspace id — or its Postman address — on the first step to skip the listing entirely.
Migrating off Postman altogether: Ctrl+A on the workspace list, or Import
all in the GUI. Everything the list is showing is imported (so the filter is
honoured), each workspace into its own folder, so two "Billing API" collections
from different workspaces both survive.
Postman rate-limits its API, so the wizard shows what it found and roughly how long the download will take before fetching anything, then reports the remaining time from the rate it is actually achieving and says when it is pausing to stay inside the limit. A Postman API key carries its owner's full access and can't be scoped, so a missing workspace is one your account isn't a member of.
The same import runs headlessly:
| Flag | Effect |
|---|---|
--postman-key |
The key, instead of $POSTMAN_API_KEY. Takes the same {{ … }} provider references as a .vars file — '{{ op://Private/Postman/credential }}' keeps it out of your shell history. Never written to disk, stripped from error messages. |
--postman-all |
Every visible workspace, each into its own folder under -o. Empty workspaces are skipped and inaccessible ones reported rather than fatal, so forty workspaces aren't stopped by one. Excludes --postman-workspace. |
--postman-what |
collections, environments or all. |
--postman-format |
postman (default) keeps the JSON byte for byte; hurl converts. |
--overwrite |
Replace a non-empty destination, which is otherwise refused. |
--postman-base-url |
Another tenant, e.g. https://api.eu.postman.com for EU Enterprise. |
The result is a folder of Collections/ and Environments/; open it with Open
▸ Workspace.
Converting to Hurl
--postman-format hurl brings across requests, folders (as Folder/Name
titles), headers, query parameters, raw bodies and form/multipart fields, plus:
- Auth, including inheritance. Collection- and folder-level auth is applied
to requests that don't set their own, and
noauthopts back out.basic,bearerandapikey(header or query) are mapped. - Collection variables, which have nowhere to live in a
.hurlfile, as<name> (collection variables).varsbeside the environments. pm.<store>.set("NAME", body.a.b)calls in test scripts, as[Captures].- Dynamic variables.
{{$guid}}and{{$isoTimestamp}}become Hurl's own{{newUuid}}/{{newDate}};{{$timestamp}},{{$randomInt}}and{{$randomAlphaNumeric}}become generated values. The rest are listed as values to supply. - Pre-request scripts, as far as they reduce to values PaperBoy can compute:
pm.environment.set("id", uuid.v4()),Date.now(),Math.floor(Date.now() / 1000),new Date().toISOString(),pm.variables.replaceIn("{{$guid}}")and literals become generated values. - Test scripts, as the status and assertions they always make:
pm.response.to.have.status(400)becomes the request's expected status, andpm.expect(...)checks on the body, headers and response time become[Asserts]. A deep equality against a literal document (.to.eql({ id: 7, name: "Ada" })) is written out one leaf at a time, since Hurl has no predicate that takes a document — which also makes a failure name the field that differed. Only checks that run unconditionally are taken — anything inside anif, a loop or a helper function is left for you, since an assertion that was meant for one branch fails every run. setNextRequest, as a note saying which of the four things it was doing: polling (which Hurl writes as[Options] retry), an order you can write down in the file or asREQUESTlines in a PaperTrail flow, a run that stopped early, or a request name built as the script ran. PaperBoy runs a collection in file order, so none of them convert — but they are four different problems with four different fixes.- Scripts on a folder or on the collection, which Postman runs for every request inside; they are converted for each request they cover, and reported once against the folder that holds them.
Hurl doesn't cover everything Postman does. Anything dropped — the rest of a
script, OAuth 2, GraphQL bodies — is listed per request in
CONVERSION-NOTES.md at the root of the import; no file means nothing was lost.
A collection this build can't read is written out as its original JSON, so
converting can't cost you data.
Headless runner
-c takes a .hurl file or a Postman export. -e supplies the environment.
Exit status is 0 only if every request passed.
By default each request's method, URL, status, asserts, captures and truncated
body print as it finishes, coloured unless the output isn't a terminal or
NO_COLOR is set. Streaming runs one request at a time through the same hurl
runner, so captures still chain — but it can't carry Hurl's automatic cookie jar
between requests, and says so at startup. An explicit [Cookies] section is
unaffected. -b/--batch runs the collection as a single Hurl call, trading
incremental output for cookie continuity.
Reports
-r report.trail runs a PaperTrail report and exits.
Without -c/-e the report's own # collection: / # environment: headers
apply, resolved relative to the report. -e is repeatable: each file is named
by its stem and becomes selectable in an ENVS loop, so -e prod.vars -e staging.vars satisfies FOR … IN ENVS BASELINE("prod"), COMPARISON("staging");
the first is also the base variable layer. -o's extension picks the format
(.csv, .json, .html, .xlsx, .pdf, or .xml for JUnit), - writes CSV
to stdout, and
omitting it derives the filename from the report's own headers.
-o is repeatable, and every file comes from one run of the requests — the
report is rendered once per format from the same result, never run twice. That
is what an application embedding PaperBoy needs: -o out.html -o out.json gives
it a rendering to show a user and a structure to parse, with no risk of the two
disagreeing because they came from separate runs. - may be given at most once
(two formats down one pipe would interleave into neither), the same path twice
is refused as a typo, and every format is checked before any request is sent,
so a misspelled extension costs nothing. If one file fails to write, the others
are left in place and the run exits 1.
--param NAME=VALUE supplies a value for a PARAM the report declares, and is
repeatable. It is what lets one report serve many runs: a report that declares
PARAM FOLDER CASES_DIR = "./cases"
FOR CASE IN FOLDERS "{{CASES_DIR}}"
is pointed somewhere new with --param CASES_DIR=./batch-07 rather than by
editing the file — which is never rewritten, so a report under version control
keeps meaning the same thing to everyone. The value beats the declared default,
is checked against the declaration (a CHOICE must be one of its options, a
NUMBER must parse), and a name the report doesn't declare is an error rather
than a value that silently does nothing. A PARAM with no default requires
a --param, since there is nothing to fall back on.
Exit codes are a contract for callers: 0 ran clean, 1 a setup error or a
run in which something failed — a request that did not come back, an assertion
that did not hold — 3 some steps were skipped because something they depended
on failed, 4 the run was stopped before it finished (see Stopping a
run), 5 a --fail-under or
--require-net-gain gate was not met, 130 a stop that was repeated and so forced, and 2
(clap's) means the command line itself was wrong. Progress goes to stderr, so
-o - leaves stdout clean for a pipe.
In a pipeline (--fail-under, --require-net-gain, JUnit)
A report is a deploy check as much as it is a document, and a pipeline reads one thing: the exit code. Two failures are worth telling apart.
The run broke. A request that never came back, or that failed its
assertions, fails the run — exit 1, the same as paperboy -c. The row still
appears in the report with its error in the cell, because the point of a report
is to show every case rather than stop at the first, but the process does not
claim success.
The answers were wrong. Everything was sent and read, and the API's answers
did not match the ground truth. That run is faultless by every measure above,
and --fail-under is what turns it into a failed pipeline step:
The gate is measured on the whole run's Correct roll-up — the per-row verdict,
over the rows that had a TRUTH to compare against — and fails the run with
exit 5 when it comes out below the threshold. Rows with no ground truth are
not counted as wrong: they were never asked. The comparison is made on the
unrounded figure, so --fail-under 100 means every scored row was right
rather than "right to one decimal place"; where the rounded figure would
contradict the verdict, the summary prints both numbers to the precision that
separates them (accuracy 94.96% — UNDER the required 95.00%). A report that
scores no column at all has nothing to gate on and is refused before anything
is sent, rather than passing every deploy on no evidence; --dry-run and
--postman-import are refused for the same reason. If a run that declared a
TRUTH nevertheless scores nothing, that is a broken run and exits 1: there
is no accuracy to hold to a threshold. A run that was stopped (exit 4) is
not given a gate verdict at all — the figure would be drawn from whichever rows
happened to finish. 5 is its own code because the two failures call for
different responses: 1 is worth retrying, 5 never is.
The answers got worse. An accuracy floor is an absolute line, and a suite
that has climbed well above it can break rows for months without ever reaching
it — a run at 98% that breaks three rows in a hundred still clears
--fail-under 95 comfortably. --require-net-gain gates on the change
instead, against the baseline the report already compares with:
It counts rows the way the Trend column does — rows fixed minus rows
regressed, over the rows scored on both sides — and fails the run with exit
5 when that comes out below the number given. 0 is "do not go backwards",
which is the usual one and needs no maintenance as the suite improves; a
positive number demands progress, and a negative one is a tolerance
(--require-net-gain -2 lets two net regressions through). Trading a fix for a
regression passes at 0; a run that wants to hear about churn asks for 1.
The two gates can be used together and share exit 5: either one failing is
enough. --require-net-gain needs both halves of the comparison it is named
after — something to be right or wrong about (a TRUTH) and something to have
been right or wrong about last time (an ENVS BASELINE(…)/COMPARISON(…) clause
or a # baseline: snapshot) — and a report missing either is refused before
anything is sent, exactly as --fail-under is. A run that was stopped, or one
where no row reached a baseline with a truth on both sides, gets no verdict:
that is a broken run (exit 1), not a quality miss.
-o results.xml writes JUnit XML, which is what makes a run show up as
test results in a CI UI rather than as a file nobody opens: one <testcase> per
row, named by the row's key so a case keeps its identity between runs, with a
failed request as <error>, a wrong answer as <failure> carrying both sides
of the comparison, and a dry run's cases as <skipped>. Each case gets exactly
one of those, in that order of seniority — a row whose request failed is an
<error> even if its answer was also wrong, because the stricter JUnit schemas
allow a case one outcome and a case counted twice makes every consumer's
arithmetic disagree; the demoted verdict is written into the case's
<system-out> rather than lost. Run errors that belong to no row (an empty
glob, a producer that failed before any row existed) and the run's caveats
become one extra case, so a run that produced nothing because something was
wrong is never a green suite of zero tests — and so the caveats are visible in
the CI systems that only read output attached to a case. It is an ordinary -o, so one run can write
-o results.xml -o report.html and give the pipeline its verdict and a human
the detail.
Watching a run from another program (--progress-json)
--progress-json replaces the human done/total counter with newline-delimited
JSON on stderr: one event object per line, flushed as it happens, so a caller
reads it incrementally (for line in proc.stderr and an ordinary json.loads
per line — no streaming parser).
plan arrives once, after the projection pass and before a single request is
sent, so a UI can draw the whole empty grid up front. row_started and
row_completed fill it — several rows are in flight at once under PARALLEL,
which is why every row event carries path: the row's structural identity
("0.3", "0.1.2.0", "" for a report with no loop), assigned before the run
and stable however the workers interleave. Alongside it, row_index is the
slot that row occupies in the projected grid — and, for a report that doesn't
collapse an ENVS comparison, the index of its row in a -o out.json report.
That is what makes the live grid and the finished file explicitly linkable: a
consumer can keep progress cheap and read the full values from the report at
the end. run_finished is the last line and carries the exit code the process
is about to use. With --fail-under it also carries a gate object
({"metric":"accuracy","required":95.0,"actual":92.4,"compared":250, "passed":false,"note":"…"}), and null without one — so a consumer can tell a
gate that passed from a deploy nobody checked.
It also carries the run's own arithmetic, so a caller gating on quality itself
does not have to parse the report it just wrote: rows_ok/rows_failed beside
rows (counted over the rows that actually ran — a projected or never-reached
row is neither), a scored object ({"compared":250,"correct":231, "incorrect":19,"accuracy":92.4}, the same unrounded percentage gate.actual
reports), and a movement object ({"fixed":3,"regressed":1,"still_wrong":2, "unchanged":244}) against the report's baseline. Both are deliberately null
rather than zeroed when there is nothing to report: a run with no ground truth
has not scored 0%, and one with no baseline has not held steady. These read the
run rather than the table, so a # columns: directive that hides a column
cannot change them. With --require-net-gain, movement also carries that
gate's verdict — required_net_gain, net_gain, passed and note — beside
the counts it was drawn from, rather than in a second place that could drift
from them.
The stream stays cheap deliberately: DETAIL and IMAGE columns — the ones the
report itself marks as drill-down content, a raw response body among them — are
named in plan as withheld_columns and left out of every row, as is any cell
over 4 KB (named in that row's withheld). Those bytes are already being
written to the report file; streaming them per row as well would make progress
cost more than the run.
schema is on every event so a consumer can refuse a stream it doesn't
understand rather than mis-read it. The human progress counter and summary are
suppressed rather than interleaved (the summary only when it would share stderr,
i.e. with -o -), so the stream is nothing but events.
run_finished is unconditional. A fatal setup error — an unreadable
collection, a mistyped --param, a validation failure — emits no plan and no
rows, but it still ends the stream with run_finished, exit_code: 1 and the
diagnostic in errors, so a consumer always has exactly one terminal event to
wait on and never has to parse prose to find out what went wrong. (Only a
command-line usage error is prose: clap rejects it and exits 2 before PaperBoy
runs at all.)
--dry-run --progress-json streams the whole grid too — plan and a
row_started/row_completed pair per projected row, cells and all — without
sending a request. It is the cheap way for a consumer to draw the grid, or to
check its own parsing, before committing to a real run.
A dry run's report file is a plan, and says so. Its rows are the ones the
run would produce with the response columns blank, so every format carries the
caveat in the same idiom a stopped run's PARTIAL uses: a DRY RUN banner
above the HTML, "dry_run": true at the top level of the JSON (and on
run_finished, beside the existing key on plan), a final DRY RUN,… record
in the CSV, a bold row under the XLSX summary, the PDF title. Nothing in it is
scored either: a ground-truth verdict reads untested rather than incorrect,
a comparison Result reads not run (dry run) rather than claiming the
candidate matched its baseline, and there are no metrics at all — a preview has
no accuracy, not an accuracy of 0%.
Stopping a run (Ctrl-C, --grace, --stop-on-stdin)
An interrupted run still writes what it has. Press Ctrl-C (or send SIGTERM)
during a report run and PaperBoy winds down rather than dying:
- No new rows start. The switch is read where a row is claimed, so the
rows already in flight — up to five live HTTP calls under
PARALLEL(5)— finish whole rather than being abandoned mid-upload. CLEANUPstill runs. This is the one that would otherwise bite:CLEANUPexists to delete sessions and release locks, and an interrupt that skipped teardown would leak them against a real service. Every block returns on its ordinary path, so teardown happens exactly as it would have.- The requested outputs are written, marked partial. Not "nothing to
report" — a four-hour run stopped at three hours is still three hours of
answers. Every format says so in its own idiom: a
PARTIALbanner above the HTML,"partial": truewithrows_completed/rows_plannedat the top level of the JSON, a finalPARTIAL,37 of 120 rows ranrecord in the CSV, a bold row under the XLSX summary, and the PDF title. A report that silently looked complete while covering 60% of the corpus would be a trap. - Exit
4. "I was told to stop" is not "your API is broken" (1) and not "steps were skipped" (3). --progress-jsonstill ends withrun_finished, carryinginterrupted: true,partial: trueand the two counts, preceded by arun_stoppingevent naming thesourceof the stop and thegrace_secondsit allows. An interrupt never reduces to a silent EOF — and nothing follows the terminal event either: a row that lands after a run has been given up on is not announced, because a consumer that finalises its state onrun_finishedwould have no way to take it.
--grace SECONDS (default 30) bounds the wind-down. If rows are still in
flight when it expires, PaperBoy gives up on them and writes the report from
the rows that did finish — a degraded rendering, without the metrics and
ground-truth scoring that can only be computed over a complete run, and
carrying a warning that CLEANUP may not have run. Its rows are still in
report order: they are harvested as they finish, so under PARALLEL they
arrive scrambled, and they are laid back into the report's canonical order
before it is written — a report is the same at any degree of parallelism, and
two stopped runs of the same corpus are diffable. A second Ctrl-C skips
even that and exits 130 immediately: the escape hatch for a CLEANUP hanging
on the very service that stopped responding.
--stop-on-stdin adds an explicit, cross-platform stop channel: a line reading
stop on stdin begins the same wind-down, and so does EOF. Signals are awkward
on Windows, where there is no SIGTERM to send a child process, and a caller
that already reads --progress-json from stderr gets a bidirectional protocol
by adding one flag. It is opt-in precisely because EOF counts: most CI runners
start processes with stdin already closed.
Dependency graphs
Inside a GRAPH … END region the order statements are written in stops being
the order they run in. PaperBoy reads what each request needs and what each
produces, works out the ordering that satisfies those, and runs that. A region
that cannot be ordered — two requests each waiting on the other — is an error,
and nothing in the report is sent: the author has said written order is not the
specification, so falling back to it would be the one answer guaranteed to be
wrong.
GRAPH release
REQUEST auth/login AS login # captures token
REQUEST orders/create AS order # uses {{token}}
REQUEST orders/get USING(id = {{order.id}})
END
Wrapping an existing block in GRAPH … END changes nothing while everything
succeeds: with no edges to reorder by, a region runs in written order. Two
things do change once something goes wrong or is ambiguous, and both are the
point of the region rather than accidents of it:
- A failure stops what depended on it. Flat, every later request is sent
regardless; in a region, the steps downstream of a failure are skipped and
the run exits
3. A request that cannot work without a token nobody captured has nothing to tell you, and sending it anyway costs a real call against a real service. - An ambiguous capture is an error, not last-writer-wins. Flat, two
requests capturing
tokenare resolved by written order. In a region there is no written order to fall back on, so a step reading{{token}}that two steps in its region capture is refused. Name the one you mean —{{login.token}}— and it is unambiguous again.
A dotted name in a report's own text always means step, then capture. The
namespace is reserved: an environment variable whose key happens to contain a
dot cannot answer one, because it would answer precisely when the step had not
captured — quietly supplying a stale credential at the moment the reference was
supposed to fail. A dotted name whose prefix is not a step in scope is an error,
wherever it is written: a USING value, a computed column, or a producer path.
Inference reads every place a value can actually be used: a URL, a header, a
body, a [Captures] or [Asserts] expression, and the bare identifiers of a
# [Gen] row, which are variable references despite having no braces. It does
not read a disabled row or a [Reports] field — nothing substitutes into
either, so an edge drawn from one would reorder a region for text that never
runs.
--targets a,b runs only the named steps and whatever they transitively
depend on, so a release check can ask for one answer without paying for the
whole report. Naming a step that no region declares is an error rather than a
silent empty run, and so is naming a set of targets that leaves out a step the
rest of the report still refers to: the reference could only reach the run as a
literal {{create.sid}}. Only qualified references are checked: a plain
{{sid}} is answered by whatever is standing in the capture chain, and pruning
can't know what else might answer it — the environment isn't even loaded at that
point — so a selection that leaves a flat reference's producer out is allowed
through. A TRUTH template counts as a reference like any other, wherever it is
written — on a computed column, on a variable column, on a WITH field or in
the columns: directive. It is resolved against the row's cells, which are
keyed by step, so pruning the step it names would leave every row in that column
scoring Untested for no stated reason. A USING(…) override is read as part of the request too, since it
decides what is actually sent — it can strand a teardown that looked clean, or
clear one that didn't. --dry-run lists the steps grouped by how deep in the graph they sit,
which is how you check the shape of a region without sending anything.
DEPENDS states an ordering the data doesn't show. Inference only sees values
flowing from a capture to a reference, and some dependencies leave no such
trace — uploading a file that a later request fetches by an id it already had,
say. REQUEST dfa/result DEPENDS upload says so outright. Names are the step
names, separated by commas, and a DEPENDS is only meaningful inside a region:
outside one, written order already is the order, so PaperBoy rejects it
rather than let it look like it did something.
Clauses may be written in any order, and a long statement may gather them into a bracketed group opening on the statement's own line:
GRAPH
REQUEST dfa/result AS result (
DEPENDS upload, session
USING(query.id = "{{session.id}}")
)
END
Running a region in parallel
PARALLEL(n) GRAPH … END lets up to n steps overlap. Each is taken the
moment its dependencies are done — not a wave at a time, which would make the
region cost the slowest step at every depth. A cap is permission, not an
instruction: a chain still runs one at a time however high n is set, and the
report is identical at any degree, because rows, columns and errors are merged
in plan order rather than in the order workers happened to finish.
Shuffling, and why
A region is a claim that its edges — inferred and declared — are the complete set. PaperBoy cannot verify that claim. It can help you falsify it.
With the default tie-break, ready steps run in written order, so a dependency
nobody declared keeps working by accident and surfaces months later when
something unrelated moves. --shuffle picks at random among the steps that are
ready, which turns that into a failure now, and prints the seed:
Shuffle : seed 4711 (replay with --shuffle=4711)
--shuffle=4711 replays that run. Shuffling only reorders steps that may
legally run in any order; it never runs a step before what it depends on.
How exact the replay is depends on the degree. A sequential region replays
exactly: the seed alone decides every choice. A PARALLEL(n) region replays
its dispatch preferences exactly, but which step becomes ready next also
depends on which request came back first, and no seed controls the network. So
a shuffled failure in a parallel region is far more likely to reproduce under
its seed than without one, but it is not guaranteed to. If you find one and
want it nailed down, re-run the seed with PARALLEL(1) — a missing dependency
is a property of the ordering, not of the concurrency, so it will still be
there.
Cleanup
CLEANUP marks a request that undoes something — deleting a session, releasing
a lock. It is written where it belongs logically but runs at the end of its
block: at the end of the flow at the top level, at the end of each iteration
inside a FOR. Cleanups run in reverse dependency order, so a thing is torn
down before whatever it was built on.
REQUEST auth/login AS login
CLEANUP auth/logout USING(header.Authorization = "{{login.token}}")
REQUEST orders/create
A cleanup whose dependency never succeeded is skipped — there is nothing to undo — and a cleanup that fails is reported as a warning rather than an error, because a teardown failing is nearly always a consequence of the real failure and shouldn't be allowed to bury it.
"Its dependency" means whatever it names in DEPENDS, plus whichever step
actually produced each value it reads. A request that fails can still have
captured, so the step a cleanup is gated on is the one whose value it will be
handed, not the last one that happened to succeed — otherwise a teardown could
be authorised by one step and then sent with a different, failed step's
identifier. Cleanups can depend on each other, by DEPENDS or by reading one
another's captures, and are ordered accordingly; a cycle between them is
refused: every member of a ring waits on a member that has not run, so none of
them is sent. A cleanup can only depend on one in its own
block, and can only read one's captures there too: an enclosing block unwinds
after this one, so a cleanup out there could never have run in time. For the
same reason an ordinary step can never read a cleanup's capture — teardown runs
after every step in its block.
Which step a cleanup waits for and which one it is gated on are two different questions. The order has to be settled before anything has run, so it can only ask which steps declare a name; the gate is asked when the teardown is dispatched, by which point the step that actually wrote the value it is being handed is a fact. A sibling cleanup that was skipped, or that ran without capturing, therefore doesn't vouch for a value it never wrote — the step that did write it is the one that has to have succeeded.
Carrying the requests in the report
A report normally names a collection to draw its requests from. It can instead
carry them itself, in a REQUESTS section — plain Hurl, which must be the last
thing in the file:
# name: Health check
GRAPH
REPORT REQUEST ping
END
REQUESTS
# ping
GET https://example.com/ping
[Asserts]
status == 200
That runs with no # collection: line and no sibling .hurl file:
paperboy -r health.trail. A report that embeds its requests may still name a
collection as well, in which case both sets are available and a name used by
both is an error — a reference has to mean one thing.
Embed when the requests exist only to serve the flow, so that the whole check
travels as one file and nothing can be moved out from under it. Reference a
collection when the requests are the API surface under test and other things
use them too. Note what embedding does and doesn't buy: it removes the sibling
collection file, not a fixture directory that FOR … IN FILES reads.
Exit codes
| Code | Meaning |
|---|---|
0 |
Everything ran and every assertion passed. |
1 |
Something failed: a request, an assertion, or the report itself. |
3 |
Steps were skipped because something they depended on failed. |
4 |
The run was stopped before it finished. |
5 |
A --fail-under or --require-net-gain gate was not met: the run was clean, the answers were not good enough. |
3 implies 1 — a skip only ever follows a failure — and says the run is
additionally incomplete, so a pipeline that only cares about pass/fail can
treat any non-zero code the same way while one that reruns can tell the
difference. 2 is left alone: it is what clap uses for a bad command line.