litnik
Litnik embeds Lit integrated shell tests and FileCheck in a Rust test target. The test suite does not need the LLVM Lit or FileCheck executables.
Version 0.1.0 is yanked. It did not connect registered tools to command
execution, did not contain FileCheck, and accepted failed discovery as an empty
success. Version 0.2.0 is the replacement.
Set up a suite
Add Litnik as one development dependency:
[]
= "0.2"
[[]]
= "lit"
= "tests/lit.rs"
= false
Put the suite configuration in tests/lit.rs:
Write normal Lit syntax in the compiler test:
// RUN: butcher-opt %s | FileCheck %s
// REQUIRES: butcher-registered-target
// CHECK-LABEL: func.func @fold_me
// CHECK: arith.constant 42
Run it through Cargo:
cargo test --test lit
cargo test --test lit -- --list
FileCheck is an embedded command. Do not register a path for it. A tool that
is registered with Suite::tool takes precedence over PATH and can be used by
its exact name in each RUN: line.
Lit model
Litnik keeps test files in Lit's text format. Rust configures the suite around that format; it does not translate directives into Rust test cases.
The suite builder configures these Lit concepts:
- source and execution roots;
- test suffixes and excluded names;
- tool paths and environment variables;
- substitutions and available features;
- test time limits, retries, and pipeline failure behavior.
Discovery is recursive. An unreadable root, an unreadable directory entry, or zero discovered tests fail the harness.
Compatibility boundary
Litnik reads RUN:, DEFINE:, REDEFINE:, REQUIRES:, UNSUPPORTED:,
XFAIL:, ALLOW_RETRIES:, and END. from the test file. It supports Lit path,
line-number, recursive, and conditional substitutions. The shell runs commands,
pipelines, ordered redirections, &&, and || without a host shell. It
implements cd, pushd, popd, export, echo, mkdir, rm, umask, and
: in process. The cat and diff commands resolve through PATH. Litnik
rejects ulimit because it cannot apply that limit correctly yet.
The embedded FileCheck supports the main LLVM directives, string and numeric variables, expressions, check prefixes, comment prefixes, implicit negative checks, whitespace modes, full-line matching, case control, and variable scope.
Litnik does not execute Python lit.cfg files or delegate commands to a host
shell. Put the corresponding suite values in tests/lit.rs.
Embedded FileCheck is not byte-for-byte compatible with LLVM FileCheck yet.
It requires UTF-8 check files and input, uses Rust regular-expression behavior,
and does not expand response files. The pinned engine does not dump passing
input for --dump-input=always; it also does not implement
--dump-input-context. The -v option does not report successful matches,
--color has no effect, and diagnostics do not use LLVM's exact layout. These
differences are part of the current 0.2.0 compatibility boundary.
Platform and toolchain
Litnik requires Rust 1.92 or later. The runner supports Unix and Windows process execution.
The crate uses synchronous filesystem and process operations. Cargo's test harness can run independent test files in parallel. Time limits stop child processes. They do not preempt an embedded command such as FileCheck while that command is running.
The standard harness reports XFAIL and FLAKYPASS as successful tests because
libtest-mimic has no custom success statuses. The low-level TestResult API
preserves the exact Lit result code.