tak-cli 0.0.5

Benchmark command-line programs and track their performance over time. Experimental; do not use.
Documentation
# Tak Settings Registry
#
# Source of truth for every setting tak supports. Modelled on the same file in
# mise, fnox and aube, for the same reason: a setting defined in one place
# cannot drift out of sync with the code that reads it or the docs that
# describe it.
#
# `build.rs` reads this file and generates, into OUT_DIR:
#   - `Settings`, a struct with one typed field per setting
#   - `Settings::default()`, from the declared defaults
#   - `SETTINGS`, a metadata slice used by `tak settings` and by docs tooling
#
# Adding a setting is a block here and nothing else. Reading one is
# `settings.<name>`.
#
# Schema for each entry:
#   type           TOML-ish type spec. Only `list<string>` is implemented;
#                  build.rs fails loudly on anything else rather than emitting
#                  something that compiles but does not work.
#   default        TOML literal, parsed as the declared type
#   sources.cli    CLI flags that set this setting
#   sources.env    Environment variables that set this setting
#   sources.config Keys in `tak.toml`, dotted
#   docs           Longer explanation, markdown
#   examples       Concrete invocations
#   since          Version the setting appeared in
#
# Precedence, highest first: CLI > environment > tak.toml > default.

[env_deny]
type = "list<string>"
default = ["GITHUB_TOKEN", "GH_TOKEN"]
sources.cli = ["--env-deny"]
sources.env = ["TAK_ENV_DENY"]
sources.config = ["env.deny"]
docs = """
Environment variables removed from every command tak measures.

Two reasons this defaults to a non-empty list rather than to nothing.

**Determinism.** A CLI that finds a forge token in its environment often does
more with it than without — authenticating, fetching, checking rate limits. A
measurement that moves depending on whether CI happened to export a token is
not a measurement of the code under test. It lands in the series as an
unexplained step change on the day someone edits an unrelated workflow.

**Credentials.** `tak backfill` downloads release binaries and executes them,
and any CI run that can push notes has a repository-write token in scope.

Setting this replaces the default list rather than adding to it. To keep the
defaults and remove more, list them alongside. To keep the defaults and remove
fewer, use `env_allow` — it is subtracted from this list, so the two compose
without either having to restate the other.

Names are matched exactly. There is no globbing: a benchmark whose behaviour
depends on which variables happen to match a pattern is the problem this
setting exists to avoid.

tak's own network calls are unaffected. `backfill` authenticates with `curl`
directly rather than through the measurement path.
"""
examples = [
  "tak run --env-deny AWS_PROFILE --env-deny AWS_REGION",
  "TAK_ENV_DENY=GITHUB_TOKEN,GH_TOKEN,NPM_TOKEN tak run",
]
since = "0.0.3"

[env_allow]
type = "list<string>"
default = []
sources.cli = ["--env-allow"]
sources.env = ["TAK_ENV_ALLOW"]
sources.config = ["env.allow"]
docs = """
Environment variables kept even though `env_deny` lists them.

Subtracted from `env_deny`, so a project can opt one variable back in without
restating the whole default list. A CLI whose measured path genuinely requires
a token — a client that cannot start unauthenticated, say — needs this.

Doing so makes the measurement depend on something outside the repository.
That is a real cost, not a formality: the numbers become conditional on the
environment the run happened to have, and a token expiring will read as a
performance change.

Listing a variable here that `env_deny` does not mention has no effect. This
setting removes entries from the deny list; it does not add anything to the
environment.
"""
examples = ["tak run --env-deny GITHUB_TOKEN --env-allow GITHUB_TOKEN"]
since = "0.0.3"

[gate_pct]
type = "float"
default = 1.0
sources.cli = ["--gate-pct"]
sources.env = ["TAK_GATE_PCT"]
sources.config = ["gate.pct"]
docs = """
How much an instruction count may rise before `tak compare` fails.

A percentage of the base measurement. Only instruction counts are gated. Wall
clock is reported and never gated: on the same hardware it moves 4-20% run to
run, so a threshold tight enough to catch a real regression would fire
constantly, and one loose enough to stay quiet would catch nothing.

The default of 1% is about fifty times the ~0.02% instruction counting
reproduces to, leaving room for the small differences a compiler or dependency
bump can produce without turning the gate into noise.

Raise it to report without effectively failing. Setting it to zero fails on any
increase at all, which sounds appealing and is not: one extra instruction on a
startup path is not worth blocking a pull request over.
"""
examples = [
  "tak compare origin/main --gate-pct 0.5",
  "TAK_GATE_PCT=5 tak compare origin/main",
]
since = "0.0.4"

[credit]
type = "bool"
default = true
sources.cli = ["--no-credit"]
sources.env = ["TAK_CREDIT"]
sources.config = ["report.credit"]
docs = """
Whether generated reports end with a line naming tak.

On by default. A report that appears in someone's pull request should say what
put it there — a reader who has never heard of tak needs a way to find out, and
a maintainer evaluating the comment needs to know what to turn off.

Turn it off with `--no-credit`, `TAK_CREDIT=0`, or `credit = false` under
`[report]`. Nothing else about the report changes.
"""
examples = [
  "tak compare origin/main --no-credit",
  "TAK_CREDIT=0 tak compare origin/main",
]
since = "0.0.4"