# cli-engine Rust Design
`cli_engine` is a Rust library for building consistent, domain-oriented command-line applications.
It is not a binary and does not assume one product's command set. Consumer CLIs provide their own
`main`, register domain modules, and let the framework handle shared CLI concerns.
The design priorities are:
- Make new commands easy to add by copying a nearby command and filling in command-specific details.
- Keep domain behavior close to the command that owns it.
- Centralize cross-cutting behavior: authentication, authorization, audit, activity, output rendering,
schemas, guides, search, command trees, and transport helpers.
- Preserve stable user-facing contracts such as command names, flag names, output envelopes, auth
provider JSON shapes, and colon-separated command paths.
- Follow normal Rust library and CLI practices: `clap` for argument parsing, `tokio` for async work,
`serde` for data, `schemars` for JSON Schema, `thiserror` for framework errors, and `reqwest`
for HTTP transport.
## Crate Shape
This crate lives in `cli-engine/`, a member of the repository's Cargo workspace alongside the
sibling `cli-engine-macros/` proc-macro crate (see the repo-root `AGENTS.md` for workspace-level
conventions):
```text
cli-engine/
Cargo.toml
docs/
auth.md
concepts.md
design.md
examples/
basic.rs
typed.rs
src/
lib.rs
cli.rs
command.rs
module.rs
middleware.rs
flags.rs
guide.rs
search.rs
tree.rs
tier.rs
error.rs
auth/
mod.rs
exec.rs
pkce.rs
...
output/
transport/
tests/
foundation.rs
derive_bridge.rs
```
The root module re-exports the common authoring surface so consumer modules can usually import from
`cli_engine::{...}` without knowing the internal file layout.
## Consumer Application Model
A consumer CLI should keep its binary entrypoint small:
```rust
use std::process::ExitCode;
use cli_engine::{BuildInfo, Cli, CliConfig};
mod modules;
#[tokio::main]
async fn main() -> ExitCode {
let cli = Cli::new(
CliConfig::new("my-cli", "Team CLI", "my-cli")
.with_build(BuildInfo::new(env!("CARGO_PKG_VERSION")))
.with_default_auth_provider("primary")
.with_modules(modules::all()),
);
cli.execute().await
}
```
Application code should be organized by domain or team ownership:
```text
src/
main.rs
modules/
mod.rs
project.rs
certificate.rs
```
Each module owns its command group, leaf commands, response types, output schemas, human views, and
module-local guides.
## CLI Assembly
`CliConfig` is the declarative root configuration. It contains:
- Root command name, short help, and optional long help.
- Build/version metadata.
- Application id.
- Default auth provider.
- Domain modules.
- Top-level commands.
- Guides and human output views.
- Auth providers.
- Lifecycle hooks for dependency initialization, custom global flags, pre-run behavior, metadata
resolution, shutdown, and extra search documents.
`Cli::new(config)` builds the `clap::Command` tree, registers framework global flags, mounts domain
modules, registers built-in commands, seeds schema and human-view registries, and prepares
middleware.
`Cli::execute()` is the normal binary entrypoint helper. Tests and generated integration harnesses
should prefer `Cli::run(args)` or `Cli::execute_from(args, stdout, stderr)` so stdout, stderr, and
exit status are asserted separately.
## Modules
Modules are domain-bounded collections of CLI functionality. Small modules can use a closure:
```rust
use cli_engine::{GroupSpec, Module, RuntimeGroupSpec};
pub fn module() -> Module {
Module::new("Platform Systems", |_context| {
RuntimeGroupSpec::new(GroupSpec::new("project", "Manage projects"))
})
}
```
Larger modules can implement `CommandModule` when named dependency ownership is clearer than a
closure. Both forms register through `ModuleContext`, which exposes middleware, schema registration,
human-view registration, and guide registration without exposing parser internals.
## Commands And Groups
Groups are noun-based containers. Commands are leaf actions. The framework derives colon-separated
paths from the command tree:
```text
my-cli project list -> project:list
```
Those colon paths are stable identifiers for policy, authorization, audit, activity, schemas,
search, and tree output.
Command definitions use `CommandSpec`; executable commands use `RuntimeCommandSpec`:
```rust
use clap::Arg;
use cli_engine::{CommandResult, CommandSpec, RuntimeCommandSpec};
use serde_json::json;
fn list_projects() -> RuntimeCommandSpec {
RuntimeCommandSpec::new(
CommandSpec::new("list", "List projects")
.with_system("projects-api")
.with_default_fields("id,name,status")
.with_arg(Arg::new("team").long("team").required(true)),
async |_credential, args| {
let team = args
.get("team")
.and_then(|value| value.as_str())
.unwrap_or_default();
Ok(CommandResult::new(json!([{ "id": "project-1", "team": team }])))
},
)
}
```
Use `RuntimeCommandSpec::new_with_context` only when a handler needs the colon command path,
user-supplied args, or a middleware snapshot.
Use `RuntimeCommandSpec::new_streaming` for commands that emit a sequence of events rather than a single result. The handler receives a `StreamSender` and writes individual `serde_json::Value` events. Each event is written to stdout as a newline-delimited JSON line as it arrives. The handler and the NDJSON writer run concurrently so the handler can keep sending while the writer flushes to stdout. If stdout is under backpressure the bounded channel can fill and the handler will wait on `send` until the writer catches up.
### Typed Arguments
When commands have many flags or already use `#[derive(clap::Args)]` structs, the typed path avoids
manual `Arg` construction and `ValueMap` extraction:
```rust
use cli_engine::{CommandResult, CommandSpec, Credential, RuntimeCommandSpec};
use serde_json::json;
#[derive(Debug, Clone, clap::Args)]
struct ListArgs {
#[arg(long)]
team: String,
#[arg(long, default_value = "10")]
limit: u32,
}
fn list_projects() -> RuntimeCommandSpec {
RuntimeCommandSpec::new_typed::<ListArgs, _, _, _>(
CommandSpec::from_args::<ListArgs>("list", "List projects")
.with_system("projects-api")
.with_default_fields("id,name,status"),
async |_credential: CredentialResolver, args: ListArgs| {
Ok(CommandResult::new(json!([
{"id": "p1", "name": "Portal", "team": args.team, "limit": args.limit}
])))
},
)
}
```
`CommandSpec::from_args::<T>()` calls `T::augment_args` to extract argument definitions.
`RuntimeCommandSpec::new_typed` deserializes parsed matches into the typed struct. Handlers that use
`RuntimeCommandSpec::new` or `new_with_context` can also call `context.typed_args::<T>()` for
on-demand deserialization.
The builder and derive paths are equivalent at runtime and can be mixed within a module.
Most real commands need more than `new_typed`'s `(CredentialResolver, T)` shape — they need the command path, middleware snapshot, or `--dry-run` via `CommandContext`. `RuntimeCommandSpec::new_typed_with_context` combines that with eager typed-arg deserialization: the engine parses `T` before calling the handler, so a `Fn(CommandContext, T) -> Fut` handler never needs `context.typed_args::<T>()` itself. `RuntimeCommandSpec::new_typed_streaming` does the same for streaming commands (`Fn(CommandContext, T, StreamSender) -> Fut`). Prefer these two over `new_with_context`/`new_streaming` + `context.typed_args()` whenever a command's args are already a `#[derive(clap::Args)]` struct — they give the same compile-time guarantee `new_typed` gives the credential-only case.
Command metadata should be explicit:
- `with_system` sets backend attribution for output and errors.
- `with_default_fields` sets default field projection for list-like output.
- `with_auth_provider` and `with_auth_metadata` select provider behavior.
- `with_tier` and `mutates` mark risk and dry-run behavior.
- `with_json_schema::<T>()` publishes JSON Schema for output.
- `with_arg` adds typed `clap::Arg` values.
- `with_arg_group` adds a `clap::ArgGroup` expressing "at least one of" or mutually-exclusive
relationships between args, without a `required_unless_present_any`/`conflicts_with` chain.
- `handles_dry_run` opts a mutating command out of the generic `--dry-run` short-circuit so its
handler runs instead (see [Dry-Run](#dry-run) below).
### Argument Relations
`CommandSpec::with_arg_group(ArgGroup)` registers a `clap::ArgGroup` alongside a command's `Arg`s, for example an "at least one of" constraint:
```rust
use clap::{Arg, ArgGroup};
CommandSpec::new("update", "Update an application")
.with_arg(Arg::new("label").long("label"))
.with_arg(Arg::new("description").long("description"))
.with_arg(Arg::new("status").long("status"))
.with_arg_group(
ArgGroup::new("update-fields")
.args(["label", "description", "status"])
.required(true)
.multiple(true),
);
```
`CommandSpec::from_args::<T>()` captures the same relationship automatically from a derive struct's struct-level `#[group(...)]` attribute — every `#[derive(clap::Args)]` struct registers one implicit group over its own fields (`multiple(true)`, `required(false)` by default, so a plain struct with no `#[group(...)]` attribute is a no-op); `#[group(required = true, multiple = true)]` customizes it into an "at least one of" constraint, and `multiple = false` into a mutually-exclusive one:
```rust
#[derive(clap::Args)]
#[group(required = true, multiple = true)]
struct UpdateArgs {
#[arg(long)] label: Option<String>,
#[arg(long)] description: Option<String>,
#[arg(long)] status: Option<String>,
}
```
**Flatten caveat.** `clap_derive` empties a struct's own implicit group's member list when the struct also has a `#[command(flatten)]` field, so `#[group(required = true)]` on a struct mixing flattened and literal fields silently enforces nothing. `CommandSpec::from_args` debug-asserts against shipping a required-but-empty group, but — like the `handles_dry_run` debug_asserts above —
treat that as a development-time safety net, not the actual guarantee: release builds won't catch it.
## Global Flags
Framework global flags populate middleware and apply consistently to every command:
| Flag | Purpose |
| --- | --- |
| `--output`, `-o` | Output format: `json`, `human`, or `toon`. |
| `--json` | Shorthand for `--output json`. |
| `--toon` | Shorthand for `--output toon`. |
| `--human` | Shorthand for `--output human`. |
| `--verbose` | Includes metadata; no value means all metadata. |
| `--dry-run` | Short-circuits mutating/destructive commands. |
| `--fields` | Selects comma-separated output fields. |
| `--filter` | Runs a JMESPath predicate against each list item. |
| `--expr` | Runs a JMESPath query against the whole result. |
| `--schema` | Renders command schema instead of running business logic. |
| `--reason` | Reason passed to authorization, audit, and activity. Only registered when `CliConfig` has an `authz`, `auditor`, or `activity` hook configured directly (not via `init_deps`, which runs after flag registration). |
| `--timeout` | Command deadline (e.g. `60s`, `5m`); default is no timeout (`0s`). |
| `--debug` | Debug selector for integrations that use it. |
| `--version`, `-v` | Prints version/build metadata. |
Applications can add their own global flags with `CliConfig::with_register_flags` and copy parsed
values into middleware with `CliConfig::with_apply_flags`.
`--limit`/`--offset` are the one exception: they are not global at all. A command registers them
for itself with `CommandSpec::with_pagination(PaginationConfig { default_limit, max_limit, ..Default::default() })`;
a command that never calls this has neither flag, in `--help` or on its command line.
`default_limit` applies when the user passes neither flag; `max_limit` (when non-zero) rejects an
explicit `--limit` above the cap.
## Middleware
Middleware owns the execution pipeline:
1. Resolve command metadata.
2. Resolve credentials unless the command is no-auth.
3. Run authorization if configured.
4. Short-circuit `--schema` or mutating `--dry-run` when applicable.
5. Run command business logic.
6. Audit and emit activity.
7. Apply the output pipeline.
8. Render success or error output.
Command handlers should not print directly. They return data or an error; middleware builds the
output envelope and renderer output. This keeps stdout machine-friendly and stderr reserved for
diagnostics in executable paths.
## Dry-Run
By default, `--dry-run` short-circuits any command with `dry_run_prompt` set (from `with_tier` or `mutates`) generically: the handler never runs, and the engine renders a fixed `{"command": ..., "action": "dry-run: would execute"}` envelope. This is cheap and side-effect free, but gives no command-specific preview — a command that would validate input (a token format, a file path, an environment name) can't run that validation under `--dry-run` either.
A command that wants a real preview instead calls `CommandSpec::handles_dry_run(true)`. This skips the generic short-circuit for that command only; the handler runs as normal (still subject to its declared `AuthRequirement` — a `Required` command still resolves its credential before the handler runs, even under `--dry-run`, once it opts in). The handler is expected to:
1. Run its real validation unconditionally (so bad input is still rejected under `--dry-run`).
2. Check `CommandContext::dry_run()` and skip only the mutating step (the write, the API call).
3. Return a `CommandResult` tagged with `CommandResult::with_dry_run()` when it took the preview branch, so middleware records a `dry-run` audit/activity outcome (instead of `ok`) and marks the envelope accordingly.
A handler that opts in but doesn't call `.with_dry_run()` on its result is treated as a normal `ok` outcome — the tagging is the handler's contract, not something middleware infers.
**Auth requirement stays as declared.** `handles_dry_run` only skips the generic short-circuit; it does not change the command's `AuthRequirement`. A `Required` command still resolves its credential *before the handler runs*, dry-run or not — so if the dry-run branch never needs a credential, `Required` forces an unnecessary (possibly interactive) auth flow purely to render a preview. Two cases:
- The dry-run branch itself needs a live authenticated call to build an accurate preview (e.g. fetching current state to diff against). Here `Required` and `Optional` behave identically — the handler needs the credential regardless of `dry_run()` — so keep the simpler, fail-closed `Required`.
- The dry-run branch can complete from validation alone, with no network call. Here call `CommandSpec::auth_optional()` and check `CommandContext::dry_run()` *before* calling `CommandContext::credential()`/`credential_with_scopes()`, so resolution — and any interactive login it might trigger — only happens on the paths that actually need it.
## Auth And Authorization
Auth providers implement `AuthProvider` and are registered with the CLI or during dependency
initialization. The dispatcher routes credential operations by provider name and supports the
built-in `auth login`, `auth status`, and `auth logout` commands.
`PkceAuthProvider` (behind the `pkce-auth` feature) is a built-in provider that implements the full browser-based OAuth 2.0 PKCE flow. It stores tokens in the system keychain and refreshes them automatically. Consumer CLIs that need a first-party browser login flow can use it directly without writing a provider binary.
Credential fields are serialized as provider-contract JSON and are used by transport injectors,
authorization, audit, and activity.
The provider process contract and transport injectors are described in
[Authentication and Transport](auth.md).
Authorization is optional and supplied by an `Authorizer` attached to middleware. The authorizer receives command path, effective args, a `CredentialResolver`, reason, and tier. Authentication policy is declared per command via `AuthRequirement` and defaults to `Required` (fail-closed): the engine resolves the credential before the handler runs and renders an `auth-error` if it cannot, so a command that should be gated cannot execute unauthenticated. The `CredentialResolver` memoizes a single resolution shared by the engine, the authorizer, and the handler. `Optional` commands defer resolution to the handler, `None` commands never authenticate, and `--schema`/`--dry-run` short-circuit before resolution so they never trigger an auth flow — except for a command that opted into `handles_dry_run`, which runs its handler (and any `Required` credential resolution) under `--dry-run` just as it would without the flag; see [Dry-Run](#dry-run).
Auditors and activity emitters are also pluggable traits. They receive enough context to record
success, auth failures, authorization denials, dry-runs, command errors, and command duration.
## Output
Handlers return JSON-serializable data and a system id. Middleware wraps the result in an envelope:
- `data`
- `pagination` (present when pagination was actually applied — an effective `--limit`/`--offset`
greater than zero — regardless of `--verbose`)
- `metadata`
- `error`
- `warnings`
- `next_actions`
- `fix` (optional recovery guidance on failed commands)
Metadata is omitted unless `--verbose` is requested. Selective metadata is supported with
comma-separated verbose fields. `pagination`, unlike `metadata`, is never gated by `--verbose` —
a caller relies on it to know whether more data exists at all. It's still conditional on
pagination actually running, though: a paginating command with `default_limit: 0` ("unlimited")
and neither flag passed produces no `pagination` field at all.
The output pipeline runs in this order:
1. `--filter`
2. `--limit` and `--offset`
3. `--expr`
4. `--fields`
5. `--output`
JSON is the default and preferred machine-readable format. Human output is designed for terminal
reading. TOON remains an optional output format.
Human views are keyed by schema id or command path:
```rust
use cli_engine::{HumanViewDef, TableColumn};
let view = HumanViewDef::new(
"project:list",
vec![
TableColumn::new("id", "ID"),
TableColumn::new("name", "Name"),
TableColumn::new("status", "Status"),
],
);
```
Custom human renderers can be registered when column output is not expressive enough.
## Schemas
Schemas exist for help output and agent comprehension. The preferred schema path is JSON Schema
from Rust types:
```rust
use schemars::JsonSchema;
use serde::Serialize;
#[derive(Debug, Serialize, JsonSchema)]
struct Project {
id: String,
name: String,
status: String,
}
```
Attach schemas with `CommandSpec::with_json_schema::<Project>()`. The framework also derives a
compact field summary for help text. Manual `OutputSchema` and `OutputField` definitions remain
available for simple or dynamic cases.
## Guides And Search
Guides are markdown documents registered globally or by module. They can come from filesystem paths,
embedded `(path, bytes)` pairs, or explicit `GuideEntry` values.
`search <query...> [--scope <path>]` is a built-in command that indexes command metadata, aliases, guide content, and extra registered search documents. `<query...>` accepts multiple words without quoting. `--scope` limits results to one command subtree (e.g. `--scope domain:list`).
## Transport
`transport::HttpClient` wraps `reqwest` for command implementations. It provides:
- Auth injection.
- Default headers and user-agent configuration.
- JSON request/response helpers.
- Raw response helpers.
- ETag and `If-Match` helpers.
- Multipart helpers.
- GraphQL helpers.
- Retry behavior.
- Structured transport errors that preserve code, system, and request id in output envelopes.
Auth injectors cover bearer tokens, provider-backed bearer tokens, cookies, basic auth, API keys,
OAuth2 client credentials, and no-op requests.
## Error Model
Framework code returns `cli_engine::Result<T>`. `CliCoreError` is the shared error enum for framework
failures, output failures, transport failures, and wrapped domain errors.
Use:
- `CliCoreError::message` for simple framework messages.
- `CliCoreError::message_for_system` for direct system-attributed messages.
- `CliCoreError::with_system` to wrap a source error with backend attribution.
- `CliCoreError::with_detailed_error` when a source error has structured code/system/request id.
- `CliCoreError::with_exit_code` when a specific process exit code must survive error wrapping.
- `CliCoreError::with_fix` when recovery guidance should appear as the envelope's top-level `fix`.
## Testing Design
The Rust crate uses integration tests in `tests/foundation.rs` to exercise the public
framework surface:
- CLI construction and built-ins.
- Command and group dispatch.
- Middleware sequencing.
- Auth provider routing.
- Output envelopes and renderers.
- Output pipeline behavior.
- Schemas and human views.
- Guides, search, and tree rendering.
- Transport clients and auth injectors.
Consumer CLIs should add their own integration tests around generated command trees. Prefer tests
that assert exit code, stdout, stderr, rendered JSON shape, and important command side effects.
Before handoff, run:
```sh
cargo fmt --all --check
cargo clippy --all-targets -- -D warnings
RUSTDOCFLAGS='-D warnings' cargo doc --no-deps
cargo test --all-targets
cargo test --doc
```
## Non-Goals
- This crate does not define product-specific commands.
- This crate does not own consumer binary entrypoints.
- This crate does not prescribe one guide-embedding crate.
- This crate does not require exact human table bytes across all implementations; the contract is
readable, stable terminal output.