# Detailed Roadmap: Auth, Environments, and Secrets
This document holds the detailed planning for Phase 2 from [ROADMAP.md](ROADMAP.md).
## Current Phase 2 State
Hen already has useful auth-adjacent building blocks, but they are still lower-level than the Phase 2 product goal:
- Milestone 2.1 is now complete: collections can declare preamble `env <name>` overlays for scalar variables, `hen run --env <name>` and MCP `run_hen { environment: ... }` share the same selection plumbing, `hen verify` reports available environments, structured run output reports the selected environment, and the docs plus example collection are in place.
- Milestone 2.2 is now complete at the scoped milestone boundary: collections can resolve local `secret.env("...")` and `secret.file("...")` references in scalar assignments, repeated lookups are cached once per run, `hen verify` remains purely structural, and the docs plus example collection are in place.
- Milestone 2.3 is now complete: secret-origin values are masked across shared output surfaces, built-in authorization and cookie header names are redacted by default, and collections can add `redact_header`, `redact_capture`, and `redact_body` rules that also propagate through downstream dependency reuse.
- Preamble variables, prompt placeholders, and dependency captures already flow through the shared request template resolver.
- CLI and MCP execution already share the same non-interactive run pipeline, so any environment or auth primitive can be exposed once and reused everywhere.
- Prompt inputs are validated up front for planned requests, which gives Phase 2 a clean place to reject missing environment or secret inputs before network calls start.
- Manual headers and dependency captures already make ad hoc bearer-token workflows possible.
- Session-backed execution already exists for MCP, SSE, and WebSocket flows, which is a useful precedent for stateful HTTP auth features.
- The current escape hatch for real secret retrieval and auth bootstrapping is still shell glue, manual prompt entry, or hand-written callback code.
- The next active planning slice is Milestone 2.4, because first-class secret inputs and safe shared redaction are now in place for later auth work.
That means Phase 2 should not invent a second configuration system. It should turn the existing variable, session, and reporting surfaces into first-class environment and auth primitives that remove shell-script dependencies without fragmenting CLI and MCP behavior.
## Auth Design Principles
- Reuse the existing variable and template model instead of creating a parallel auth-only DSL.
- Keep secret values out of committed collections by default; collections should describe where secrets come from, not inline the secrets themselves.
- Make precedence explicit across collection defaults, named environments, secret providers, explicit CLI or MCP inputs, and runtime dependency captures.
- Keep `hen verify`, non-interactive CLI runs, and MCP execution deterministic: syntax inspection must not trigger secret-provider network calls or interactive auth side effects.
- Apply redaction at the shared report and artifact layer so text, JSON, NDJSON, JUnit, and MCP all stay aligned.
- Prefer a small number of high-leverage built-ins over a long list of provider-specific auth wizards.
- Treat stateful auth as part of the same execution model Hen already uses for dependencies and sessions, not as a separate subsystem.
## Milestone 2.1: Environment Model and Input Precedence
Hen needs first-class named environments before higher-level auth flows become maintainable.
- Define how collections select named environments such as `local`, `staging`, and `prod` without rewriting request bodies or headers.
- Preserve existing preamble variables as collection defaults, then layer named-environment overrides, explicit CLI or MCP inputs, and runtime dependency captures on top of them with a documented precedence model.
- Add explicit environment selection to non-interactive CLI and MCP entry points so the same collection can run under automation without interactive prompts.
- Keep prompt placeholders meaningful within an environment-aware model: prompt inputs should remain the last-mile override for values the environment layer intentionally leaves unresolved.
- Make missing environment values fail before request execution begins.
### Milestone 2.1 Implementation Anchors
- Variable and request expansion in `src/request/template.rs`.
- Prompt placeholder resolution in `src/parser/context.rs`.
- Up-front run preparation and prompt validation in `src/automation.rs`.
- CLI argument handling in `src/cli.rs` and `src/cli_run.rs`.
- MCP run input handling in `src/mcp.rs`.
### Milestone 2.1 Dependencies and Risks
- Precedence can become hard to reason about if collection defaults, environment overrides, prompt values, and dependency captures all behave slightly differently.
- Environment file format sprawl is a risk; Phase 2 should choose a narrow, explicit model rather than accept every possible config source.
- Interactive prompt behavior must stay compatible with CI and MCP usage rather than becoming a hidden fallback for missing environment values.
### Milestone 2.1 Exit Check
The same collection can run against multiple environments with explicit selection and documented override precedence, without editing the collection or relying on shell-exported variables.
Status: complete.
### Milestone 2.1 Detailed Plan
The recommended 2.1 scope should stay intentionally narrow:
- Add named collection-level environment overlays for scalar variable values.
- Keep environments variable-only in 2.1; requests, headers, bodies, and protocol directives should continue to vary through existing variable interpolation rather than through environment-specific request blocks.
- Thread environment selection through the shared run-preparation surface so CLI and MCP stay aligned.
- Resolve environments before prompt validation so missing values still fail before any network execution starts.
- Leave secret-provider lookups, cookie persistence, OAuth flows, and signing out of scope until later milestones.
#### Recommended Authoring Surface
The cleanest fit with the current DSL is a preamble-only environment overlay model.
```text
name = Example Collection
$ API_ORIGIN = https://api.example.com
$ AUTH_AUDIENCE = public
$ CLIENT_ID = [[ client_id ]]
env local
$ API_ORIGIN = http://localhost:3000
$ AUTH_AUDIENCE = local
env staging
$ API_ORIGIN = https://staging.example.com
---
Get profile
GET {{ API_ORIGIN }}/profile
* X-Client-Id = {{ CLIENT_ID }}
```
- Environment declarations should only be valid in the collection preamble, before the first `---` request delimiter.
- Environment overrides should only target scalar variables in 2.1.
- Environment declarations should not introduce arrays, shell substitutions, or secret-provider syntax in 2.1.
- Environment names should be explicit and selected with a run-time flag or tool argument, not inferred from the machine or current shell.
- If no environment is selected, Hen should keep today’s behavior and use the collection-level defaults only.
#### Recommended Resolution Model
The plan should commit to one deterministic precedence order:
1. Collection preamble scalar assignments establish the base variable set.
2. The selected environment overlays matching scalar variables.
3. Explicit CLI `--input key=value` values or MCP `inputs` override unresolved prompt-backed values for that run.
4. Prompt defaults apply only when the selected environment and explicit run inputs still leave a placeholder unresolved.
5. Runtime dependency captures and callback exports continue to override static values for downstream requests during execution.
That keeps environment selection as a static run-preparation concern while preserving the current execution-time capture model.
#### Recommended Data-Flow Changes
- Extend the parsed collection model to retain declared environment names plus their scalar overrides.
- Add an optional selected environment field to the shared run request surface so CLI and MCP use the same plumbing.
- Apply the selected environment while building request templates, before the prompt-validation pass that currently reports missing `[[ prompt ]]` values.
- Surface available environment names in verify and summary output so automation and editors can discover them without executing a run.
- Surface the selected environment in machine-readable run output so CI artifacts say which configuration was used.
#### Delivery Slices
1. Parser and collection model
- Parse `env <name>` blocks in the preamble and reject them after requests begin.
- Reject duplicate environment names.
- Reject overrides of unknown scalar variables so typos fail early.
- Reject non-scalar overrides in 2.1 to keep mapped-request semantics unchanged.
1. Shared run-preparation plumbing
- Add `environment: Option<String>` to the shared run request shape.
- Resolve the selected environment during `prepare_run_path`.
- Fail early for unknown environment names.
- Reuse the existing prompt-validation pass after the environment merge so the current non-interactive guarantees stay intact.
1. CLI and MCP selection surfaces
- Add `hen run --env <name>` for explicit environment selection.
- Add an `environment` argument to the MCP `run_hen` tool.
- Keep environment selection out of `hen verify`; verify should parse and report environments, not require selecting one.
1. Reporting and discoverability
- Include `selectedEnvironment` in structured run output.
- Include `availableEnvironments` in verify output and collection summaries.
- Keep text output minimal; environment selection can appear in summary output rather than on every request line.
1. Documentation and examples
- Add one environment-focused example collection.
- Document precedence clearly in `README.md` and `syntax-reference.md`.
- Show one non-interactive CLI example and one MCP tool-call example using explicit environment selection.
#### Validation Targets
- Parser coverage for valid environment blocks and targeted authoring errors for duplicate names, unknown variables, misplaced blocks, and non-scalar overrides.
- Automation coverage showing missing prompt validation happens after environment merge rather than before it.
- CLI coverage for `--env` parsing and unknown-environment failure behavior.
- MCP coverage for `run_hen { environment: ... }` and machine-readable reporting of the selected environment.
- Integration coverage proving the same collection can run with different base URLs or client identifiers without editing the file.
#### Scope Boundaries for 2.1
- No environment inheritance or fallback chains beyond collection defaults plus one selected environment.
- No per-request environment blocks.
- No implicit loading from `.env`, shell profiles, or host-specific config files.
- No secret backends, OAuth token acquisition, cookie jars, or signing hooks yet.
- No environment-specific mutation of request graphs or dependency structure.
## Milestone 2.2: Local Secret Providers and Resolution Lifecycle
Once environments exist, Hen needs a safe way to resolve local secrets without shell callbacks or remote secret-backend complexity.
Hen now has the intended local-only provider model for this milestone: `secret.env("...")` and `secret.file("...")` work in scalar assignments, repeated lookups are cached once per run, `hen verify` accepts those references without resolving them, and docs plus an example collection are in place. Optional OS keychain work remains deferred and is not part of the milestone exit criteria.
- Add a provider abstraction for secrets sourced from environment variables and local files.
- Keep OS keychain support as a stretch goal rather than a core exit requirement for the milestone.
- Do not support remote secret backends in this phase; they add network I/O, provider auth, and failure modes that do not fit Hen's current scope.
- Resolve secret references as part of run preparation or request expansion, but keep `hen verify` purely structural so syntax inspection never performs secret-provider I/O.
- Cache resolved secrets within a run so repeated requests or retries do not repeatedly hit the same provider.
- Surface provider failures as explicit, actionable errors instead of leaving unresolved placeholders deep inside requests.
- Keep provider outputs compatible with named environments and higher-level auth flows so OAuth, cookie bootstrapping, or signing can reuse them.
#### Recommended 2.2 Provider Order
1. `env` provider as the lowest-friction CI and local-dev path.
2. `file` provider for mounted secrets, local certificates, and container workflows.
3. OS keychain only if the abstraction remains narrow and the implementation can stay explicitly optional by platform.
### Milestone 2.2 Implementation Anchors
- Variable expansion and request materialization in `src/request/template.rs`.
- Prompt and placeholder handling in `src/parser/context.rs`.
- Non-interactive run preparation in `src/automation.rs`.
- Shared CLI and MCP entry points in `src/cli_run.rs` and `src/mcp.rs`.
- This milestone likely merits a dedicated auth or secrets module rather than continuing to grow template-expansion logic inline.
### Milestone 2.2 Dependencies and Risks
- Even local secret-provider APIs introduce I/O, caching, and failure modes that do not belong in syntax verification.
- File-provider behavior needs explicit rules for path resolution, missing files, and newline handling so the results stay predictable.
- OS keychain integration is fragmented across macOS Keychain, Windows Credential Manager, and Linux Secret Service or libsecret, so it should stay additive and optional rather than drive the core design.
- Secret caching has to balance reproducibility and retry behavior without growing into a broader secret-refresh subsystem.
### Milestone 2.2 Exit Check
Collections can reference secrets from env vars and files without shell scripts, and verification still does not execute those lookups. OS keychain support, if added, is a bonus rather than a requirement for milestone completion.
Status: complete.
#### Scope Boundaries for 2.2
- No remote secret backends or hosted vault integrations.
- No provider-specific authentication flows.
- No implicit `.env` loading or host-profile discovery.
- No secret writes, rotation, or refresh workflows.
- No verify-time secret resolution.
## Milestone 2.3: Redaction and Safe Output
As soon as secrets become first-class inputs, Hen must stop leaking them into logs and artifacts.
- Define default redaction behavior for sensitive headers such as `Authorization`, `Cookie`, and `Set-Cookie`, plus token-like values captured in bodies or exported variables.
- Add configurable redaction rules for headers, body paths, captures, transcripts, retained artifacts, and machine-readable report fields.
- Ensure redaction happens before text output, JSON or NDJSON serialization, MCP responses, and any future artifact export surfaces.
- Preserve debugging value by replacing sensitive values with stable markers instead of dropping surrounding structure entirely.
- Keep redaction additive and deterministic so machine-readable output remains a stable automation contract.
### Milestone 2.3 Implementation Anchors
- Shared report serialization in `src/report/`.
- Text output projection in `src/cli_output.rs`.
- Execution artifact and transcript handling in `src/request/artifact.rs` and `src/request/executor.rs`.
- Capture and assertion surfaces that can currently echo values back in mismatches or diffs, especially `src/request/response_capture.rs` and `src/request/assertion.rs`.
### Milestone 2.3 Dependencies and Risks
- Over-redaction can make failures impossible to debug; under-redaction leaks secrets into CI logs.
- Diffs, assertion mismatch payloads, and transcript bodies are easy places to miss partial leaks.
- Redaction rules need to behave identically across CLI, MCP, and structured reporting.
### Milestone 2.3 Exit Check
Sensitive values are consistently masked in human-readable and machine-readable outputs, including transcripts, assertion failures, and partial-run reports.
Status: complete.
### Milestone 2.3 Detailed Plan
Initial implementation status: complete. The shared redaction slice now tracks values originating from `secret.*(...)`, applies built-in masking for authorization-style and cookie header names, and redacts those values in structured report bodies, run-level collection summaries, execution traces, transcripts, retained artifacts, JUnit assertion payloads, and CLI body, failure, or trace rendering. Collections can further broaden the policy with preamble `redact_header`, `redact_capture`, and `redact_body` directives, and captured sensitive exports continue to redact correctly when downstream dependent requests reuse them.
The recommended 2.3 scope should start by centralizing redaction before it adds more configuration syntax:
- Ship one shared redaction policy and projection layer that every output format uses.
- Start with strong built-in defaults for known sensitive material: authorization-style headers, cookies, values loaded through `secret.*(...)`, and captured token-like values that can appear in diffs or reports.
- Keep raw values available for request execution, dependency captures, and assertion evaluation; only outward-facing projections should be redacted.
- Use stable placeholders that preserve structure without leaking value length or partial substrings.
- Add user-configurable redaction rules only after the default policy is applied uniformly across text, JSON, NDJSON, JUnit, MCP, and retained artifacts.
#### Recommended 2.3 Delivery Order
1. Shared redaction core
- Introduce a small redaction module or shared report-layer helper that can mask by header name, exact value, capture name, and body path.
- Tag values that originate from `secret.env(...)` and `secret.file(...)` so later output surfaces can redact them consistently without guessing.
- Normalize a default sensitive-header set before any formatter-specific projection occurs.
1. Output-surface integration
- Redact request and response headers in text summaries, verbose traces, and structured report payloads.
- Redact request and response bodies, failure diffs, and assertion mismatch snippets before they are attached to artifacts or serialized.
- Redact retained artifacts and MCP responses from the same projection layer so CLI and agent usage stay aligned.
1. Configurable rule surface
- Add explicit configuration for additional headers, capture names, and JSON body paths once the defaults are stable.
- Keep rule matching additive: user rules can broaden protection but should not disable built-in masking by default.
- Keep `hen verify` structural; it can parse redaction configuration but should not need live secret values to validate it.
#### Recommended Coverage Targets
- Request headers: `Authorization`, `Proxy-Authorization`, `Cookie`, and API-key style headers.
- Response headers: `Set-Cookie` and any explicitly configured sensitive headers.
- Body payloads and diffs where access tokens, refresh tokens, session ids, or API keys may appear.
- Response captures, callback exports, and assertion failure messages that currently echo resolved values.
- Machine-readable outputs and partial-run reports so CI logs and artifacts never diverge from text output.
#### Validation Targets
- Unit coverage for redacting exact values, headers, captures, and body-path projections without altering surrounding structure.
- Integration coverage proving the same sensitive value is masked identically in text, JSON, NDJSON, JUnit, and MCP outputs.
- Regression coverage for failure diffs, assertion mismatch messages, and interrupted runs, since those are the easiest places to leak raw values.
- Golden-fixture coverage for default header and cookie masking plus one user-configured rule case.
#### Scope Boundaries for 2.3
- No in-memory secret scrubbing guarantees inside the executor process.
- No attempt to hide whether a field existed; redaction should mask values, not remove useful structural context.
- No provider-specific auth flows yet; OAuth, cookies, and signing remain separate milestones that should plug into the same redaction layer later.
- No per-formatter policy divergence; text and structured outputs should share the same masking rules.
## Milestone 2.4: Cookie Jars and Stateful HTTP Auth
Many real login flows are stateful even before OAuth enters the picture.
- Add HTTP cookie jar persistence across dependent requests and make the scope explicit.
- Reuse or extend the existing session model so cookie-backed HTTP auth does not become a one-off mechanism unrelated to the rest of execution planning.
- Support login flows that depend on a sequence of requests, redirects, CSRF values, and cookie reuse.
- Make cookie behavior deterministic under parallel execution by defining isolation boundaries clearly.
- Ensure cookie values are redacted in transcripts and reports by default.
### Milestone 2.4 Implementation Anchors
- Session storage in `src/request/session.rs`.
- HTTP execution in `src/request/executor.rs`.
- Request model and protocol-neutral execution surface in `src/request/mod.rs`.
- Dependency planning in `src/request/planner.rs` and `src/request/runner.rs`.
- Report serialization in `src/report/`.
### Milestone 2.4 Dependencies and Risks
- Cookie scope is easy to get wrong across hosts, redirects, retries, and parallel branches.
- Session reuse rules must stay understandable instead of silently sharing auth state across unrelated requests.
- Cookie-backed flows raise redaction and transcript-safety requirements immediately.
### Milestone 2.4 Exit Check
A typical form-login or session-cookie workflow can be modeled using ordinary dependent requests without custom callback code or external cookie tooling.
## Milestone 2.5: OAuth2 and OIDC Token Flows
OAuth is the highest-leverage auth feature once environments, secrets, and redaction are in place.
- Deliver first-class token acquisition for the most common API testing cases, starting with client-credentials and refresh-token flows.
- Add OIDC discovery support where it materially reduces endpoint boilerplate or token-endpoint configuration.
- Feed acquired tokens back into the normal request environment so later requests can reference them through headers, captures, or dependency context rather than a special auth-only pipeline.
- Cache and refresh tokens within a run when expiry makes that necessary, while keeping behavior deterministic under parallel execution.
- Redact tokens everywhere they can surface, including transcripts, diffs, and machine-readable reports.
### Milestone 2.5 Implementation Anchors
- Request execution and dependency context in `src/request/executor.rs`, `src/request/mod.rs`, and `src/request/runner.rs`.
- Variable expansion in `src/request/template.rs`.
- Session or credential caching in `src/request/session.rs`.
- Shared CLI and MCP invocation surfaces in `src/cli_run.rs` and `src/mcp.rs`.
- Structured output in `src/report/`.
### Milestone 2.5 Dependencies and Risks
- Token refresh can race under parallel execution if multiple requests try to renew the same credential at once.
- OIDC discovery introduces another network-backed dependency that must not leak into syntax verification.
- OAuth flows can expand quickly into provider-specific behavior if the initial scope is not kept tight.
### Milestone 2.5 Exit Check
A common client-credentials or refresh-token workflow can run without shell callbacks, and tokens can be reused safely by later requests in the same collection.
## Milestone 2.6: Signing and Transport Credentials
The last Phase 2 slice should cover auth mechanisms that modify transport behavior or canonical request signing.
- Add transport credential configuration for mTLS, including client certificate and key material sourced through the environment and secret layers.
- Add first-class request-signing support for common patterns such as HMAC and AWS SigV4.
- Ensure signing happens after template, environment, and secret resolution but before request dispatch, so retries and parallel runs stay deterministic.
- Keep the signing surface transport-aware but not provider-fragmented; the goal is a reusable primitive set, not a separate DSL for every vendor.
- Make signed requests observable enough to debug without leaking raw credentials.
### Milestone 2.6 Implementation Anchors
- Request execution in `src/request/executor.rs`.
- Request model and protocol-specific operation data in `src/request/mod.rs`.
- Template and environment expansion in `src/request/template.rs`.
- CLI and MCP configuration entry points in `src/cli.rs`, `src/cli_run.rs`, and `src/mcp.rs`.
- Output safety in `src/report/` and `src/cli_output.rs`.
### Milestone 2.6 Dependencies and Risks
- Signature algorithms are brittle when canonicalization or body hashing differs subtly from provider expectations.
- mTLS credential loading has to interact safely with the secret-provider layer and local file-system expectations.
- Signed requests become difficult to reproduce if retries or redirects change the input that was originally signed.
### Milestone 2.6 Exit Check
At least one certificate-based flow and one signature-based flow can run natively without shell glue, while keeping CLI and MCP behavior aligned.
## Remaining Delivery Order Within Phase 2
1. Environment model and override precedence.
2. Secret providers and resolution lifecycle.
3. Redaction and safe output.
4. Cookie jars and stateful HTTP auth.
5. OAuth2 and OIDC token flows.
6. Signing and transport credentials.
The order matters because milestones 4 through 6 become much easier to ship safely once environment selection, secret sourcing, and redaction are already stable.
## Phase 2 Non-Goals
- Storing raw production secrets directly in committed `.hen` collections.
- Falling back to interactive prompts silently in CI when a required secret or environment value is missing.
- A provider-specific auth wizard for every identity platform or cloud vendor.
- Divergent auth behavior between CLI and MCP execution.
- A second config system that bypasses the existing template, dependency, and reporting pipeline.