# OpenAPI Import Compatibility Plan
## Objective
- Make OpenAPI a first-class contract format for Hen's HTTP surface.
- Let users import OpenAPI specs without lossy translation for supported HTTP semantics.
- Keep import focused on OpenAPI contract semantics rather than inventing Hen-only execution or testing behavior.
- Grow Hen's schema, parameter, body, and auth capabilities until OpenAPI-defined HTTP APIs can materialize into executable Hen collections without contract loss.
## Current Status
Implemented in the current importer:
- `hen import <spec-path> [selector] --output <collection.hen>` with `all`, index, `operationId`, `METHOD /path`, and `tag:name` selectors.
- OpenAPI 3.0.x and 3.1.x JSON or YAML loading, including local multi-file refs.
- Path-item refs, component parameter/requestBody/response/securityScheme refs, and plain component schema aliases.
- Supported security lowering for API keys, basic, bearer, OAuth client credentials, reduced OpenID Connect bearer placeholders, and reduced non-client-credentials OAuth2 bearer placeholders.
- Server inheritance, cookie authoring, duplicate query preservation, supported parameter serialization, JSON-like `+json` media types, form-urlencoded, multipart text fields, object-only `allOf` flattening, named `oneOf` and `anyOf` unions, scalar `const(...)` tags, and `discriminator(...)` unions when the source shape is representable.
- VS Code command surface for importing OpenAPI specs into new `.hen` collections, plus example source and generated output under `examples/`.
Still intentionally out of scope or not yet implemented:
- remote refs
- richer multipart encoding-object and binary payload behavior
- full OpenID Connect metadata import and executable non-client-credentials OAuth flows
## What "Import-First OpenAPI Compatibility" Means
- Hen can consume OpenAPI 3.0.x and 3.1.x YAML or JSON, including multi-file refs.
- Hen can resolve operations, parameters, request bodies, responses, servers, examples, security requirements, links, and schema constraints from an OpenAPI spec.
- Hen can diagnose invalid or unsupported OpenAPI input during import with the same quality of diagnostics and required-input reporting it provides for `.hen` collections.
- Hen can materialize an editable `.hen` collection from a spec.
- Materialized-Hen execution should stay behaviorally aligned with the canonical OpenAPI contract for the same HTTP operation.
## Compatibility Boundaries
- First-class OpenAPI compatibility applies to HTTP API specs consumed by Hen.
- Hen's dedicated non-HTTP protocols such as `mcp`, `sse`, and `ws` remain Hen-native and cannot be represented in plain OpenAPI without extensions.
- GraphQL-over-HTTP can participate where a spec describes ordinary HTTP request and response contracts, but Hen's native GraphQL authoring stays additive.
- OpenAPI does not natively represent assertions, captures, dependencies, prompts, callbacks, or redaction rules. Import does not attempt to synthesize those Hen-only behaviors from the spec alone.
## Product Surface
Core command family:
```text
hen import <spec-path> [<selector>] --output <collection.hen>
```
Selector rules:
- Selectors apply to `hen import`.
- If no selector is provided, import all supported operations from the spec along with the shared components they require.
- `all` is equivalent to omitting the selector.
- Numeric indices resolve against a deterministic operation list ordered by normalized path, then HTTP method, then `operationId` when present.
- Support `operationId`.
- Support `METHOD /path`.
- Support tag filters.
- Support numeric indices and `all`.
Import behavior:
- Fail the import when the spec is invalid, references cannot be resolved, the selector matches nothing, the selected operation set contains no safely materializable operations, or the importer cannot emit syntactically valid `.hen`.
- Otherwise, import every selected operation and component that can be materialized safely.
- Emit explicit warnings for skipped operations, approximated semantics, unresolved values, and unsupported features.
## Architectural Direction
Do not build OpenAPI support as a one-off importer. Build one canonical HTTP contract model for the OpenAPI import pipeline, and keep it reusable if other Hen surfaces adopt it later.
Modularity constraints:
- Prefer directory-based module trees over large phase files.
- Keep top-level `mod.rs` or facade files thin; they should orchestrate pipelines, not hold detailed parsing, lowering, rendering, or validation logic.
- Split modules by stable domain boundaries such as schemas, operations, parameters, bodies, responses, security, examples, refs, and diagnostics.
- Avoid "god modules" in both the OpenAPI layer and the canonical contract layer.
- When one implementation area starts to span multiple independent concepts, split it immediately rather than allowing a single file to become the integration point for unrelated logic.
- Share common data models and helpers through small focused modules rather than reusing one catch-all utility file.
Suggested high-level module split:
- `src/contract/`: canonical HTTP contract IR for the OpenAPI import pipeline, split into focused modules such as schema, operation, parameter, body, response, security, examples, and source spans.
- `src/openapi/`: thin facade plus domain subtrees rather than broad single files.
Recommended shapes:
- `src/contract/{mod,schema,operation,parameter,body,response,security,example,source_span}.rs`
- `src/openapi/{mod,facade}.rs`
- `src/openapi/loader/{mod,parse,resolve,index}.rs`
- `src/openapi/lower/{mod,schema,operation,parameter,security,example}.rs`
- `src/openapi/diagnostics/{mod,capability,source_map,report}.rs`
- `src/openapi/materialize/{mod,schema,request,layout}.rs`
Pipeline ownership:
- A thin OpenAPI facade should own `import` orchestration.
- Loader modules should only parse, normalize, and resolve references.
- Lowering modules should only translate OpenAPI structures into the canonical contract IR.
- Materialize modules should only format canonical contract data into `.hen` surfaces.
- Diagnostics modules should own source mapping, capability checks, and user-facing diagnostic assembly.
Shared execution direction:
- OpenAPI -> contract IR -> import diagnostics and materialize surfaces.
- Contract IR -> `.hen` materializer.
Future reuse of the contract IR by existing Hen HTTP authoring is optional and not required for the first production slice.
This is the only scalable path if import diagnostics and materialization fidelity all matter.
## Required Core Work
### 1. Canonical HTTP Contract IR
- Define reusable contract types for operations, parameters, request bodies, responses, schemas, security requirements, servers, examples, and links.
- Give every operation and schema a stable canonical ID.
- Preserve source spans from both OpenAPI and `.hen` inputs so diagnostics stay actionable.
- Keep the IR rich enough that import diagnostics and materialization do not each need their own partial models, without requiring the existing Hen HTTP authoring path to adopt it in the first production slice.
### 2. Schema Engine Parity
Hen's schema system must grow past the current object, scalar, array, optional, and nullable model.
Chosen syntax direction:
- Keep `enum(...)` as the literal-membership syntax for scalar declarations.
- Add `const(...)` as the single-literal scalar syntax for tagged branches and discriminator fields.
- Add explicit schema combinators for composition and negation: `allOf(...)`, `oneOf(...)`, `anyOf(...)`, and `not(...)`.
- Prefer named branch schemas as the primary authoring model for unions, discriminators, and most conditional shapes.
- Keep object field syntax as `field: TypeRef` only. Do not add field-level schema expressions in the first pass.
- Add explicit discriminator syntax rather than inferring discriminator behavior from branch shapes.
- Keep combinator arguments to named schema or scalar references only in the first pass.
- Do not introduce a `where` block or ternary-style schema syntax in the first pass.
- Treat inline guarded-field syntax and native conditional keywords as possible future sugar, not the primary import-compatibility surface.
- Add an object rest-property syntax for `additionalProperties` parity:
- omit `...:` to keep Hen's current open-object default
- `...: never` for closed objects
- `...: TypeRef` for typed additional properties
Examples:
```text
scalar Food = enum("pizza", "tacos")
scalar Drink = enum("water", "soda")
scalar CardKind = const("card")
scalar BankKind = const("bank")
scalar Consumable = oneOf(Food, Drink)
scalar NonFood = not(Food)
schema CardCheckout {
method: CardKind
card: Card
billingAddress: Address
...: never
}
schema BankCheckout {
method: BankKind
bank: Bank
...: never
}
schema Checkout = discriminator(method,
"card": CardCheckout,
"bank": BankCheckout
)
schema Admin = allOf(User, AdminFields)
schema SearchResult = anyOf(UserResult, TeamResult, RepoResult)
schema Labels {
...: string
}
```
Authoring guidance:
- Use named branch schemas plus `oneOf(...)` as the canonical surface for discriminator-driven and branch-oriented contracts.
- Use `discriminator(field, value: Branch, ...)` for tagged unions when OpenAPI defines or implies a discriminator.
- Preserve Hen's existing `&` chaining where it already models conjunction well, especially for scalar refinement.
- Use `const(...)`-backed named branch schemas to model tagged object variants without field-level expressions.
- When importing OpenAPI conditionals, prefer lowering to named branch schemas and discriminated unions when the condition can be represented as a branch split.
- Leave native syntax for `if`/`then`/`else`, `dependentRequired`, and `dependentSchemas` open until real specs show cases that branch composition cannot express cleanly.
Locked combinator semantics:
- `oneOf(A, B, ...)` succeeds only when exactly one branch validates. Zero matches or multiple matches fail.
- `anyOf(A, B, ...)` succeeds when one or more branches validate. Multiple matches are allowed.
- `allOf(A, B, ...)` is schema intersection. For object branches, the implementation must converge on merged object semantics rather than treating them as unrelated passes once closed-object rules are in play.
- `not(A)` succeeds only when `A` does not validate.
- `discriminator(field, "tag": Branch, ...)` selects one branch by the value of `field`; unknown tag values fail, and branch validation continues against the selected branch.
Needed features include:
- `allOf` composition and merge rules.
- `oneOf` and `anyOf` union validation.
- `discriminator`-based branch selection.
- `not`.
- Lowering rules and capability diagnostics for OpenAPI conditional and dependent constraints, using named branch schemas and discriminated unions where representable.
- Closed-object rules such as `additionalProperties` and related 3.1 semantics where applicable.
- Validation keywords that materially affect runtime correctness such as `const`, `multipleOf`, min/max constraints, uniqueness, and string length or pattern rules already aligned with Hen's scalar system.
This may require expanding `src/schema/mod.rs`, adding a richer schema AST, and extending mismatch reporting so union and branch failures remain debuggable.
### 3. Parameter Serialization Parity
Hen needs first-class parameter serialization logic, not just simple `?` and `*` lowering.
Chosen request-cookie direction:
- Add first-class request-cookie authoring with `@ CookieName = value`.
- Treat `@ ...` lines as structured request cookies, not raw `Cookie` header text.
- Serialize all request cookies for a request into one outbound `Cookie` header.
- Support cookie declarations in both the preamble and individual requests, with request-level cookies overriding same-name preamble cookies.
- Reject mixing structured `@ ...` cookies with a manual `* Cookie = ...` header on the same request.
- Merge explicit `@ ...` cookies with session-jar cookies by name, with explicit request cookies winning for that request.
Examples:
```text
$ SESSION = [[ session_cookie ]]
@ theme = dark
---
Get profile
@ session = {{ SESSION }}
GET https://example.com/profile
```
Required coverage:
- `path`, `query`, `header`, and `cookie` parameters.
- `simple`, `form`, `label`, `matrix`, `spaceDelimited`, `pipeDelimited`, and `deepObject` styles.
- `explode` behavior.
- `allowReserved` behavior.
- Array and object parameter encoding.
Cookie-specific rules:
- OpenAPI `in: cookie` parameters should lower to `@ CookieName = ...` in materialized `.hen` output.
- OpenAPI `apiKey` security with `in: cookie` should also lower to structured `@ ...` cookie authoring rather than a raw `* Cookie` header.
The canonical request builder should use the same serialization rules whether the source was OpenAPI or `.hen` authoring.
### 4. Request Body and Media Type Parity
Hen's HTTP body model must support the OpenAPI surface more directly.
Required coverage:
- `application/json`.
- `text/plain`.
- `application/x-www-form-urlencoded`.
- `multipart/form-data`, including encoding-object behavior that materially changes wire format.
- Binary and file payloads.
- Wildcard and fallback content types where the spec requires them.
- `example` and `examples` selection rules for request generation.
### 5. Security and Auth Parity
Hen needs to model security requirements as part of the canonical HTTP contract.
Required coverage:
- Global, path-level, and operation-level security inheritance.
- `apiKey` in header, query, and cookie.
- `http` schemes such as basic and bearer, with an explicit decision on how to handle other HTTP auth schemes.
- OAuth2 flows, including client credentials, authorization code, device code if represented through extensions, and other OpenAPI-described flows Hen chooses to support.
- `openIdConnect` metadata handling.
- Non-interactive auth materialization rules so imported collections stay deterministic.
Cookie-auth authoring rules:
- Cookie-based API keys should use the same structured `@ CookieName = ...` surface as ordinary request cookies.
- Structured request cookies must participate in the same redaction and tooling surfaces as manual `Cookie` headers do today, without losing their parameter-level identity.
Auth materialization policy:
- Import must remain non-interactive and contract-only.
- When auth inputs can be materialized directly from concrete examples or defaults in the spec, import them.
- When auth inputs cannot be materialized concretely, generate unresolved Hen variables or secret placeholders in the imported collection and emit warnings.
- Do not prompt for credentials or attempt runtime secret resolution during import.
### 6. Import and Materialization
`hen import` should remain a supported surface, but it should be powered by the same canonical contract path as import diagnostics and materialization.
Materialized output should:
- Render idiomatic, editable `.hen`.
- Run existing Hen verification automatically after materialization.
- Surface warnings when the generated `.hen` does not verify cleanly, while still writing the imported collection when materialization succeeded.
- Preserve contract semantics, not just a rough approximation.
- Leave Hen-only assertions, captures, dependencies, prompts, and redaction rules to post-import authoring rather than inventing them from OpenAPI input.
- Materialize OpenAPI cookie parameters and cookie-based apiKey auth as `@ CookieName = ...` lines rather than raw `* Cookie` headers.
## Milestones
### Milestone 0: Import Contract and Design Decisions
- Define the exact compatibility promise for OpenAPI 3.0 and 3.1.
- Choose loader and resolver dependencies with proven 3.1 coverage.
- Define the exact contract-only import boundary and which Hen-only behaviors remain post-import authoring concerns.
- Define fail-vs-warn import rules for invalid specs, unsupported features, approximations, and skipped operations.
- Define deterministic auth placeholder rules for materialized collections.
- Decide where the canonical HTTP contract IR lives.
- Define capability diagnostics for partially supported features.
### Milestone 1: Loader and Resolver
- Parse YAML and JSON.
- Normalize OpenAPI 3.0 and 3.1 inputs into one internal view.
- Support in-document refs and local multi-file refs.
- Plan remote ref support with caching and offline behavior, even if remote refs land one slice later.
- Surface structured, source-mapped diagnostics for invalid specs, skipped operations, unsupported features, and approximations.
### Milestone 2: Canonical Contract IR
- Introduce the shared contract model for the OpenAPI import pipeline.
- Lower OpenAPI into the contract model.
- Add golden tests that prove lowering plus materialization preserve semantics across representative specs.
### Milestone 3: Schema Engine Expansion
- Add support for `allOf`, `oneOf`, `anyOf`, `discriminator`, `not`, plus lowering support for representable conditional and dependent OpenAPI constructs.
- Add the first user-facing schema composition syntax using named branch schemas plus `allOf(...)`, `oneOf(...)`, `anyOf(...)`, `not(...)`, `const(...)`, and `discriminator(...)`.
- Keep schema object fields on plain `TypeRef` syntax and do not add field-level schema expressions in this slice.
- Add object rest-property syntax for `additionalProperties` parity.
- Add closed-object and property-rule semantics required for correct 3.0 and 3.1 validation.
- Add capability diagnostics for conditional and dependent constructs that cannot yet be lowered through branch composition.
- Extend mismatch reporting so failures remain readable when multiple branches are involved.
- Update structured report payloads to preserve richer schema failure detail.
### Milestone 4: Parameter and Body Parity
- Implement full parameter serialization coverage.
- Add `application/x-www-form-urlencoded` support.
- Add richer multipart handling, including encoding rules that materially change request construction.
- Support example and default resolution for request generation.
### Milestone 5: Security and Auth
- Implement security inheritance.
- Support API keys in all OpenAPI-defined locations.
- Support HTTP auth schemes Hen chooses to treat as first-class.
- Expand OAuth and OIDC handling to the OpenAPI-described surface Hen will import faithfully.
- Materialize deterministic auth placeholders or concrete values without prompting during import.
- Thread auth placeholder diagnostics through CLI, automation, and reporting surfaces.
### Milestone 6: OpenAPI Import Surface
- Add the `hen import <spec-path> [<selector>] --output <collection.hen>` command surface.
- Add shared automation entrypoints in `src/automation.rs`.
- Reuse existing reporting surfaces so OpenAPI import diagnostics look like Hen tooling, not a separate product.
- Ensure import diagnostics carry operation metadata, required inputs, schemas, and security data for tooling.
- Define exit-code semantics for fail-fast conditions versus successful imports with warnings.
### Milestone 7: Importer and Materializer
- Rebuild `hen import` on top of the contract IR.
- Produce editable `.hen` that stays semantically aligned with the canonical contract extracted from the spec.
- Add stable materialization rules for names, schemas, examples, and security scaffolding.
### Milestone 8: Tooling and Docs
> Update the web docs and language support plugin (new schema features).
- Reuse the automation entrypoints in editor surfaces.
- Add fixtures for real-world 3.0 and 3.1 specs.
- Document import and materialization workflows.
- Document the compatibility boundaries for non-HTTP Hen features.
## Acceptance Criteria
- `hen import` can parse and diagnose representative 3.0 and 3.1 specs, including local multi-file refs and selector-scoped imports.
- Omitting the selector imports the full supported operation set for the spec.
- `all` is equivalent to omitting the selector, and numeric indices resolve against deterministic ordering by normalized path, HTTP method, and `operationId` when present.
- Invalid specs, unresolved refs, unmatched selectors, and syntactically unmaterializable outputs fail the import.
- Selected operation sets that match the spec but contain no safely materializable operations also fail the import.
- Partially supported specs still import every safely materializable operation and emit explicit warnings for skipped or approximated parts.
- Materialized `.hen` output can execute representative operations with correct parameter serialization, auth, and response validation for supported features.
- Materialized `.hen` execution stays behaviorally aligned with the canonical OpenAPI contract for the same operation.
- `hen import` runs post-materialization verification and surfaces warnings when generated `.hen` output does not verify cleanly.
- Auth values are imported from concrete examples/defaults when available, otherwise rendered as unresolved placeholders with warnings and without interactive prompts.
- Schema tests cover unions, discriminator flows, negation, and branch-composition lowering for representable conditional and dependent OpenAPI constructs, with focused regression coverage and explicit diagnostics for unsupported cases.
- Parameter-style, media-type, and security coverage is locked down with targeted tests.
## Validation Strategy
- Unit tests for schema lowering, schema validation, serialization, body construction, and security resolution.
- Integration tests for `hen import`, including omitted-selector imports and post-materialization verification warnings.
- Golden fixtures for imported `.hen` output.
- Semantic regression tests that compare representative OpenAPI fixtures with the generated `.hen` output and its verification results.
- End-to-end tests that import representative specs and then run the generated `.hen` collections against test servers, not just `hen verify`.
- Regression suites for unsupported or partially supported features so capability warnings stay explicit.
- Integration coverage for fail-fast cases versus successful imports with warnings.
- Integration coverage for auth example/default import, unresolved auth placeholders, and non-interactive import behavior.
- Real-spec fixtures from both OpenAPI 3.0 and 3.1 ecosystems.
Suggested validation commands after implementation:
- `cargo test openapi:: --lib`
- `cargo test contract:: --lib`
- `cargo test --test import_cli`
- `cargo test --test non_interactive_cli`
- `cargo test --test verify_cli`
- `cargo test`
## Risks and Tradeoffs
- OpenAPI 3.1 can require substantially richer JSON Schema support than Hen has today.
- Import-side parity requires core runtime and schema work, not just a new parser.
- Exact preservation of original spec structure in materialized `.hen` output is unrealistic; semantic fidelity is the right target.
- Users will expect upstream spec compatibility, so unsupported features must fail loudly or warn clearly.
- Users may expect imported collections to include Hen-only assertions, captures, or dependencies, so the contract-only import boundary must stay explicit in docs and diagnostics.
- Auth materialization can look incomplete when specs do not provide concrete examples or defaults, so placeholder behavior and warnings must be explicit.
## Still Out of Scope
- Treating non-HTTP Hen protocols as plain OpenAPI without extensions.
- Synthesizing assertions, captures, dependencies, prompts, or redaction rules from OpenAPI alone.
- Exact preservation of original spec layout, comments, or field ordering when materializing a `.hen` collection.
