sarah 0.1.8

Count tokens, lines and function sizes per file to help AI agents decide what to refactor
sarah-0.1.8 is not a library.

sarah

Sarah is our beloved mother that tracks our behavior and make sure we don't screw up. Enforcing tokens , line count, folder structure and more.

Counts tokens, lines and function sizes per file so AI agents (and humans) can see when, what and how to refactor a repo.

Built on existing crates: ignore (gitignore-aware parallel walking), bpe-openai (tokens; exact tiktoken counts), tokei (code/comment/blank lines), tree-sitter-tags (function spans), rayon (parallelism; files over 256 KB are tokenized in chunks).

Install

Download a binary for Linux (x86_64/aarch64, static musl), macOS (Apple Silicon/Intel) or Windows (x86_64) from Releases, or:

cargo install sarah

Usage

sarah [PATH]                      # human table
sarah -f json                     # full report for agents
sarah -f md                       # markdown to paste into an agent's context
sarah --fail-on-warn              # exit 1 if anything exceeds limits (CI / agent loops)
sarah -f md --json-out r.json     # markdown on stdout, full JSON in a file
sarah --dirs                      # add per-folder totals to the table
sarah -f github                   # GitHub Actions ::warning annotations
sarah -f sarif > sarah.sarif      # SARIF 2.1.0 for code scanning
sarah -f json > baseline.json     # save today's warnings...
sarah --baseline baseline.json    # ...then report only new ones, or ones that got worse
sarah --changed-since origin/main # only warnings for files changed on this branch
sarah -f html > sarah.html        # self-contained treemap of files by size (works offline)

sarah init                        # write sarah.toml with limits at today's maximums
sarah ratchet                     # lower those limits as the code shrinks (never raises them)
sarah mcp                         # serve sarah to AI agents over MCP (stdio)

A directory named like a command (init, ratchet, mcp) needs a path prefix: sarah ./init.

Key flags: --sort tokens|lines|name, -n/--top N, --tokenizer o200k|cl100k, --max-file-tokens 8000, --max-file-lines 500, --max-fn-lines 80, --max-fn-tokens N, --max-fn-depth N, --max-fn-params N, --max-fn-cyclomatic N and --max-fn-cognitive N (off by default), -i/--include GLOB, -e/--exclude GLOB, --hidden, --no-functions, --no-default-excludes (lockfiles, *.min.js, *.map and generated files are skipped by default), --config FILE (default: <PATH>/sarah.toml when it exists), --no-allow (ignore sarah-allow comments).

Exit codes: 0 success, 1 warnings with --fail-on-warn, 2 error.

Function extraction: Rust, Python, JavaScript, TypeScript/TSX, Go, Java, C, C++, C#, Ruby, PHP, Kotlin, Swift, Scala, Lua, Elixir, Zig, Bash. Each function has:

  • depth: the deepest chain of if/loop/match/switch/try blocks (else if chains count once).
  • params: declared parameters, without receivers (self, cls) and C's (void). Not reported for Elixir and Bash.
  • cyclomatic: McCabe, 1 + each branch, loop, catch, ternary, case/match arm and &&/||.
  • cognitive: SonarSource cognitive complexity. Each branch, loop, switch/match and catch adds 1 plus its nesting level; else if, else and each run of like boolean operators (a && b && c) add 1. Both are close approximations from the syntax tree (Elixir case arms are not counted in cyclomatic).

The table and markdown output do not show these columns; JSON has them. Their limits warn as too_many_params and complex_function.

Generated files are skipped: a file whose first 5 lines contain @generated, DO NOT EDIT, Code generated, auto-generated, autogenerated or automatically generated. They are counted under skipped.generated; --no-default-excludes counts them as code.

The JSON report also has directories: per folder, its direct files and dirs, and total_files, lines and tokens for the whole subtree.

Allow comments: sarah-allow

Turn off function or file warnings in the code, like clippy's #[allow]:

// sarah-allow(large_function, deep_function): one flat state machine
/// Docs, attributes and decorators may sit between the comment and the function.
#[instrument]
fn run() { /* ... */ }
# sarah-allow-file(large_file): generated
  • sarah-allow(kinds): reason goes directly above a function: large_function, deep_function, too_many_params, complex_function.
  • sarah-allow-file(kinds): reason goes anywhere in a file: large_file and every function kind.
  • No list (sarah-allow: reason) allows every kind of that scope.
  • The : is required; the reason after it may be empty.
  • Any line-comment style works (//, #, --, /* */); the comment must be on its own line.
  • A malformed directive, or one above no function, is a bad_allow warning.
  • Allowed warnings are removed before --baseline and --fail-on-warn. The report shows "N warnings allowed by sarah-allow comments" (JSON: allowed); --no-allow shows them all.
  • Folder rules have no comment; override them per glob in sarah.toml.

Config: sarah.toml

Optional. Without it, sarah behaves as above. CLI flags win over the file.

preset = "rust"            # rust | python | go | ts: built-in folder rules, applied first
exclude = ["vendor/**"]    # added to --exclude (include = [...] too)

# Size limits per glob. Later matches override earlier ones, field by field.
[[files]]
path = "**"
max_fn_lines = 60
max_fn_depth = 4

[[files]]
path = "tests/**"
max_fn_lines = 200          # also: max_file_tokens, max_file_lines, max_fn_tokens,
                            # max_fn_params, max_fn_cyclomatic, max_fn_cognitive

# Folder rules. Later matches override earlier ones, field by field.
[[folder]]
path = "src/**"             # `*` = one path part, `**` = any; the root is "." (or "**")
max_files = 12              # files directly in the folder
max_dirs = 8                # sub-folders directly in the folder
max_depth = 4               # levels below the root; reported once per branch
max_tokens = 40000          # tokens in the whole subtree (one agent context)
min_files = 2               # direct entries (files + folders); fewer = flatten it
require = ["mod.rs"]        # each glob must match a file in the folder
forbid = ["utils*", "misc*"] # no file or folder name may match
allow_only = ["cli", "walk"] # sub-folder names must match one of these
naming = "snake_case"       # snake_case | kebab-case | camelCase | PascalCase
no_sibling_module = true    # no `x.rs` next to `x/`

[[folder]]
path = "src/generated/**"
max_files = 100
require = []                # an empty list turns a rule off

Folder warnings use kinds such as folder_too_many_files and folder_bad_name. They work with --fail-on-warn, --baseline, --changed-since and annotations.

--baseline matches warnings by path, kind and subject (function name or glob), and shows a warning again when its file, function or folder grew. --changed-since REV compares with the merge base of REV and HEAD, and counts uncommitted and untracked files.

Token counts use OpenAI encodings; Claude's tokenizer isn't public, so treat them as a close approximation.

Ratchet: sarah init and sarah ratchet

sarah init writes <PATH>/sarah.toml (or --config FILE) with one [[files]] rule for **. Each limit is today's maximum, so new code cannot be worse than the worst code today. It does not overwrite an existing file.

sarah ratchet lowers each max_* key of each [[files]] rule to the largest value that the rule governs today (the last matching rule that sets the key). It never raises a limit, never adds a key, and keeps comments and layout. Values that sarah-allow comments exempt are left out. --dry-run prints the changes only. Run it in CI or after a refactor, and commit the result.

MCP server: sarah mcp

sarah mcp serves three tools over stdio, so an agent can ask sarah directly:

Tool Returns
report The full report for path, as markdown (default) or JSON
warnings Only the warnings as JSON; changed_since limits them to a branch
check_file One file's sizes, functions (with every metric) and warnings, under the root's sarah.toml

Every tool takes args: extra CLI flags, for example ["--max-fn-lines", "60"]. Paths are relative to the server's working directory. For Claude Code:

claude mcp add sarah -- sarah mcp

Symlinks: every file is counted once, keyed by its realpath. File links are followed when their target is inside the root; directory links are never descended (an in-root target is walked by its real path anyway), so link cycles can't loop. Broken links, links leaving the root, directory links and duplicate targets are reported under skipped instead of disappearing.

GitHub Action

Installs the released binary for the runner (Linux, macOS, Windows; x64 and ARM64 where released), writes the markdown report to the job summary and exposes the JSON report.

- uses: actions/checkout@v7
- id: sarah
  uses: aiman2039/sarah@v0.1.3   # a release tag; @master installs the latest release
  with:
    path: .
    args: --max-fn-lines 60 --exclude 'vendor/**'
    fail-on-warn: true
- run: echo "${{ steps.sarah.outputs.tokens }} tokens, ${{ steps.sarah.outputs.warnings }} warnings"
Input Default
path . Directory to analyze
args Extra sarah arguments
fail-on-warn false Fail the step when limits are exceeded
summary true Write the markdown report to the job summary
annotations true Annotate the files (and lines) that have warnings
version action ref if it is a tag, else latest Release tag to install
token github.token Token used to download the release

Outputs: json (report path), tokens, warnings.

Development

Hooks install on the first cargo test (cargo-husky). pre-commit runs scripts/check.sh fmt and scripts/check.sh clippy. Clippy runs pedantic plus the safety lints in [lints.clippy] in Cargo.toml, with -D warnings.

sh scripts/check.sh          # fmt + clippy + tests + cargo-audit + cargo-deny (same as CI)

Releasing

scripts/bump-version.sh               # 0.1.0 -> 0.1.1 in Cargo.toml + Cargo.lock (no commit)
git commit -am "Release v0.1.1" && git push
scripts/release.sh                    # trigger the Release workflow for origin/master's version

scripts/git-release.sh --check-only   # run every check, change nothing
scripts/git-release.sh                # all of the above in one step: check, bump, commit, tag, push, release

The Release workflow (.github/workflows/release.yaml) runs CI on macOS/Linux/Windows, builds all five targets in parallel, uploads them to a draft release with SHA256SUMS.txt, publishes it, then runs the action end to end against the new release. CI runs only as part of Release.