# Dropshot API manager
This crate provides most of the scaffolding to manage OpenAPI documents generated by [Dropshot](https://docs.rs/dropshot), using its support for **API traits**.
For more information about API traits, see [Oxide RFD 479](https://rfd.shared.oxide.computer/rfd/0479).
> [!NOTE]
> The OpenAPI manager relies on symbolic links for some of its functionality. If you're on Windows, you'll need to ensure that symlink support is enabled.
>
> * [Enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development#activate-developer-mode), which allows non-administrators to create symlinks.
> * Run `git config --global core.symlinks true`.
>
> Also, disable CRLF conversions within Git by checking in a `.gitattributes` file with:
>
> ```
> # Disable CRLF conversions on Windows.
> * -text
> ```
## Who this is for
The Dropshot API manager was built for our needs at Oxide. But it is generally usable by many projects that use Dropshot. The manager is useful to you, if you:
* Use [Dropshot](https://docs.rs/dropshot) for your HTTP APIs
* Define your APIs using [API traits](https://docs.rs/dropshot/latest/dropshot/attr.api_description.html) rather than plain functions
* Organize your code using the archetypal strategy, where the API trait lives in its own crate
```mermaid
flowchart LR
dropshot_api_manager([<b>Dropshot API manager</b>]) --> api
types[base types]
api[Dropshot API trait] --> types
logic --> api
subgraph production_impl [implementation]
binary([server binary]) --> logic
logic[logic, database storage, etc]@{ shape: processes }
end
```
* Commit the generated OpenAPI documents to source control; Git is required if you use versioned APIs (see below)
* All generated OpenAPI documents live in the same directory
## Examples
A [fully working end-to-end example is available](https://github.com/oxidecomputer/dropshot-api-manager/tree/main/e2e-example). You're welcome to copy part or all of the example as desired.
## Basic usage
Dropshot's OpenAPI manager consists of two crates:
- [`dropshot-api-manager-types`](https://crates.io/crates/dropshot-api-manager-types): Base types for the OpenAPI manager. Crates defining API traits may depend on `dropshot-api-manager-types`.
- [`dropshot-api-manager`](https://crates.io/crates/dropshot-api-manager): The main implementation.
### Requirements
You need to provide:
- The API traits for your APIs (see [_API crates_](#api-crates) below). These may depend on `dropshot-api-manager-types`.
- A binary crate that integrates your API traits with `dropshot-api-manager`, called the **integration point**.
> [!TIP]
> We recommend you set up an easy way to run your binary. The quickest way is to add an alias to `.cargo/config.toml`:
>
> ```toml
> [alias]
> openapi = "run -p my-openapi-bin --"
> ```
>
> If you use the [`cargo xtask`](https://github.com/matklad/cargo-xtask) framework, you may choose to put the integration point within the xtask binary. However, at Oxide we've generally seen quicker compile times putting the integration point into a separate binary, and setting up a minimal xtask that just runs this binary.
>
> Your chosen alias is passed into the OpenAPI manager, and it will be displayed as part of user guidance. In the examples below, we'll use `cargo openapi`, as set up by the alias.
In addition, if you have versioned APIs (see below):
* The OpenAPI manager must be run within a Git repository.
* You must also ensure that `git` is available on the command line, or that the `GIT` environment variable is set to the location of the Git binary.
* Shallow clones don't work; full Git history is required. See [_CI and shallow clones_](#ci-and-shallow-clones) below.
#### API crates
The OpenAPI manager requires as inputs a set of *API crates*. An API crate is a Rust library that consists of the API trait, and possibly supporting types. In general, each OpenAPI document should have a separate API crate, though it is acceptable to have multiple closely-related OpenAPI documents within the same crate.
In general, to keep compile times down, the API crate should have as few dependencies as possible. For an archetypal way to organize code, see the dependency graph in [RFD 479's _Choosing between functions and traits_](https://rfd.shared.oxide.computer/rfd/0479#functions_vs_traits).
## Managing OpenAPI documents
For OpenAPI documents to be managed by this tool, the corresponding interfaces must be defined via **API traits** rather than traditional Dropshot function-based servers.
### Adding new APIs
If you're defining a new service fronted by OpenAPI, first create an API crate (see [_API crates_](#api-crates) above).
1. Add the API crate to the workspace's `Cargo.toml`.
2. Following the example in [RFD 479's _Trait definition_](https://rfd.shared.oxide.computer/rfd/0479#guide_trait_definition), define the API trait.
In the implementation crate:
1. Add a dependency on the API crate.
2. Following the example in [RFD 479's _API implementation_](https://rfd.shared.oxide.computer/rfd/0479#guide_api_implementation), provide an implementation of the trait.
Proceed to adding this API's OpenAPI document.
### Adding new OpenAPI documents
#### Deciding on a versioning strategy
When adding an API, you may need to decide on a versioning strategy for the document.
In simpler cases where clients are known to always match servers, the OpenAPI manager supports a **lockstep** versioning strategy. If you're unsure, start with a lockstep strategy.
For cases where clients and servers can potentially have be mismatched, such as when the server is updated to a new version but the client is not, Dropshot supports a more complex **versioned** strategy. See [RFD 532 ("Versioning for internal HTTP APIs")](https://rfd.shared.oxide.computer/rfd/0532) for a more complete discussion.
**For versioned APIs:** copy the following template into the **API crate** near the top, _above_ the `#[dropshot::api_description]`:
```rust
api_versions!([
// WHEN CHANGING THE API (part 1 of 2):
//
// +- Pick a new semver and define it in the list below. The list MUST
// | remain sorted, which generally means that your version should go at
// | the very top.
// |
// | Duplicate this line, uncomment the *second* copy, update that copy for
// | your new API version, and leave the first copy commented out as an
// | example for the next person.
// v
// (next_int, IDENT),
(1, INITIAL),
]);
// WHEN CHANGING THE API (part 2 of 2):
//
// The call to `api_versions!` above defines constants of type
// `semver::Version` that you can use in your Dropshot API definition to specify
// the version when a particular endpoint was added or removed. For example, if
// you used:
//
// (1, INITIAL)
//
// Then you could use `VERSION_INITIAL` as the version in which endpoints were
// added or removed.
```
**For both lockstep and versioned APIs:** once the API crate is defined, update the OpenAPI manager to manage the new OpenAPI document(s). Within this directory:
1. In your repository's integration point's `Cargo.toml`, add a dependency on the API crate.
2. Add the crate to the list of APIs managed by the OpenAPI manager. For versioned APIs, the `api_versions` macro defines a `supported_versions()` function, which you'll need to use.
> [!NOTE]
> The `api_versions!` macro also generates a `latest_version` function which returns the latest version (the first version on the list).
>
> When creating the server, you must configure a [`version_policy`](https://docs.rs/dropshot/latest/dropshot/struct.ServerBuilder.html#method.version_policy) to indicate that the API is versioned. If you use the [`ClientSpecifiesVersionInHeader`](https://docs.rs/dropshot/latest/dropshot/struct.ClientSpecifiesVersionInHeader.html) policy, set the `max_version` to the output of `latest_version()`.
To ensure everything works well, run `cargo openapi generate`. Your OpenAPI document should be generated on disk and listed in the output.
##### Versions crate
For long-term sustainability of managing types across versioned APIs, we recommend using a _versions crate_ as described in [RFD 619 Managing types across Dropshot API versions](https://rfd.shared.oxide.computer/rfd/619). Here's an archetypical crate graph:
```mermaid
flowchart TD
subgraph crates [crate dependencies]
versions["versions crate
(all published types)"]
types["types crate
(re-exports latest)"]
api["API trait"]
types --> versions
api --> versions
end
subgraph business [business logic]
stateless[stateless logic]
stateful[stateful logic]
stateful --> stateless
end
stateless --> types
subgraph boundary [boundary code]
real_impl[real API implementation]
test_impl[test API implementation]
end
client[Progenitor client]
real_impl --> api
real_impl --> stateful
real_impl -.-> |"prior versions"| versions
test_impl --> api
test_impl --> stateless
test_impl -.-> |"prior versions"| versions
client --> versions
subgraph binaries [binaries]
prod([production binary])
test([test binary])
end
prod --> real_impl
test --> test_impl
```
The key points:
- The **versions crate** is the source of truth for all published types.
- The **types crate** is a facade that re-exports from `latest`, used by business logic.
- The **API trait** depends only on the versions crate (not the types crate).
- **Business logic** depends only on the types crate, not the versions crate.
- **Boundary code** depends on the versions crate for prior version endpoints.
#### Performing validation
By default, the Dropshot API manager does not do any kind of validation or linting on the generated document, beyond the basic checks performed by Dropshot itself. If desired, the API manager can be configured to perform _global validation_ on all documents, as well as _extra validation_ on some of them.
For global validation, set the `validation` function on the `ManagedApis` struct. For an example, see the `validate` function within [the end-to-end example](https://github.com/oxidecomputer/dropshot-api-manager/blob/main/e2e-example/bin/src/main.rs).
For extra validation on some documents, it's recommended that you put them on the trait, within the API crate.
1. In the API crate, add dependencies on `openapiv3` and `dropshot-api-manager-types`.
2. Define a function with signature `fn validate_api(spec: &openapiv3::OpenAPI, mut cx: dropshot_api_manager_types::ValidationContext<'_>) which performs the extra validation steps.
3. Convert the `ManagedApiConfig` to a `ManagedApi` and call the `with_extra_validation` builder method with this function.
Currently, the validator can do two things:
1. Via the `ValidationContext::report_error` function, report validation errors.
2. Via the `ValidationContext::record_file_contents` function, assert the contents of other generated files.
### Iterating on lockstep APIs
Assuming you're starting from a fresh branch from `main`, the general workflow for making changes to a lockstep API looks like this:
1. Make whatever changes you want to the API crate (the trait definition)
2. In whichever order you want:
* Update the server(s) (the trait impl). You can immediately see what's needed with `cargo check`.
* Update the client. To do this, run `cargo openapi generate` to regenerate the OpenAPI document. Then `cargo check` will tell you how the client needs to be updated.
3. Repeat steps 1-2 as needed.
### Iterating on versioned APIs
See [guides/new-version.md](guides/new-version.md) for an overview and for detailed instructions using the versions crate pattern.
As of this writing, every API has exactly one Rust client package and it's always generated from the latest version of the API. Per RFD 532, this is sufficient for APIs that are server-side-only versioned. For APIs that will be client-side versioned, you may need to create additional Rust packages that use Progenitor to generate clients based on older OpenAPI documents. This has not been done before but is believed to be straightforward.
## Git stub storage
For versioned APIs, the Dropshot API manager can optionally store older API versions as *Git stubs* instead of full JSON files. A Git stub is a small text file (with a `.gitstub` extension) that points to the JSON content at a specific Git commit.
### Benefits
- **Meaningful diffs on GitHub.** New API versions appear as renames of the previous version, so GitHub shows what actually changed rather than thousands of added lines.
- **Blame works across versions.** `git blame` on the latest version traces history back through previous versions.
- **Smaller repository checkout on disk.** Git stubs are ~100 bytes each, replacing large JSON files in the working copy.
For more background, see [RFD 634](https://rfd.shared.oxide.computer/rfd/0634).
### Tradeoffs
- **Requires Git history.** Shallow clones won't work, so CI must use `fetch-depth: 0`. See [_CI and shallow clones_](#ci-and-shallow-clones) below.
- **Rename-rename conflicts.** Parallel branches adding versions to the same API produce merge conflicts. See [_Git stub merge conflicts_](#git-stub-merge-conflicts) below.
- **Tools must dereference Git stubs.** Tools that need older API versions must know how to read Git stubs, though most workflows only use the `-latest.json` symlink.
### Enabling Git stub storage
To enable Git stub storage for an API, use the `.with_git_stub_storage()` builder method when configuring the API in your integration point. You can also call `.with_git_stub_storage()` on a combined `ManagedApi` to turn Git stub storage on by default. Use `.disable_git_stub_storage()` to opt an API out of default Git stub storage.
For details on the file format and conversion rules, see [_Git stub storage details_](#git-stub-storage-details) below.
## More about versioned APIs
The idea behind versioned APIs is:
* This is an API where the client and server can be mismatched at runtime when the system is upgraded.
* Thus: for the system to keep working across an upgrade, the server _must_ support both the old and the new versions.
* To ensure that the server supports older versions, we check those OpenAPI documents into source control and this tool verifies that the server remains compatible with these older versions.
For much more on this, see [RFD 532 "Versioning for internal HTTP APIs"](https://rfd.shared.oxide.computer/rfd/0532).
For a versioned API, the set of all supported versions is defined by the `api_versions!` macro in the API crate. More precisely: in configuring the OpenAPI manager tool to know about a versioned API, you use the `supported_versions()` function defined by the macro. **This is critical: the OpenAPI documents in the `openapi` directory are _not_ the source of truth about what versions are supported. The Rust `api_versions!` call is.**
Each of these supported versions is either **blessed** (meaning it's been committed-to -- i.e., shipped, or potentially deployed on a system that we care about upgrading smoothly) or **locally-added**. Blessed versions are only allowed to change in provably wire-compatible ways.
Changes **allowed** in blessed versions include:
* Documentation updates
* Renaming types while keeping the overall structure the same
* Adding or removing newtype wrappers
Changes **not allowed** in blessed versions include:
* Adding or removing endpoints
* Adding or removing fields from a struct
* Adding or removing variants from an enum
* Changing string validation regexes
When you run `cargo openapi check` or `cargo openapi generate`, the tool loads OpenAPI documents from three sources:
* **blessed** documents: these are generally the OpenAPI documents in "main" in the `openapi` directory. (More precisely, by default, these are loaded from the merge-base between `HEAD` and `main`. You can override this.) By definition, these only contain blessed versions (since locally-added versions won't be present in "main").
* **local** documents: these are the OpenAPI documents in `openapi` in your working tree. These include both blessed versions and locally-added versions.
* **generated** documents: these are the OpenAPI documents generated by Dropshot from your API traits.
Putting all this together, the tool is pretty straightforward. For each supported version of a versioned API:
* If there is a blessed file for that version, then the version is blessed. The generated file must match the blessed one (up to wire-compatibility). If they don't, the tool cannot fix this. You have to undo whatever changes you made that affected the blessed version. (See above on how to make changes to the API trait without affecting older versions.)
* If there is no blessed file for that version, then the version is locally-added. There should be exactly one local file for it and it should exactly match the generated file. The tool can fix any problems here by removing all local files and generating a new one based on the generated one.
* The tool also ensures that a "latest" symlink exists and points to the highest-numbered OpenAPI document.
```mermaid
flowchart TD
HaveSupportedVersion["Have supported version<br/>(explicit list in Rust code)"]
QIsLockstep{"Is this a lockstep API?"}
IsLockstep["Make the local OpenAPI document match the generated one."]
QHaveBlessedSpec{"Is there an OpenAPI document in the blessed source (upstream)?"}
HaveBlessed["Verify that the generated OpenAPI document is compatible with the blessed document."]
NoBlessed["Make the local OpenAPI document match the generated one (and remove any others)"]
IsLatest{"Is it the latest version of this API?"}
NeedSymlink["Make the 'latest' symlink for this API refer to this version's OpenAPI document."]
HaveSupportedVersion --> QIsLockstep
QIsLockstep -->|"Yes"|IsLockstep
QIsLockstep -->|"No, it's versioned"|QHaveBlessedSpec
QHaveBlessedSpec -->|"Yes, it's a blessed version"| HaveBlessed
QHaveBlessedSpec -->|"No, it's a locally-added version"| NoBlessed
HaveBlessed --> IsLatest
NoBlessed --> IsLatest
IsLatest --> |"Yes"|NeedSymlink
```
You generally don't need to think about any of this to use the tool. Like with lockstep APIs, you just use `cargo openapi generate` to update the local files. The only ways you're likely to run into trouble are:
1. You forgot to define a new version. In this case, you'll get an error about having changed a blessed version. To fix this, you'll need to follow the steps above to make changes that don't affect older versions.
2. You defined a new version, but forgot to annotate the API endpoints with what version they were added or removed in. Again, you'll get an error about having changed a blessed version and you'll need to follow the steps above to fix it.
3. You merge with an upstream that adds new versions.
### CI and shallow clones
Versioned APIs require access to Git history: the tool loads blessed versions from the merge-base between `HEAD` and `main`, which requires that history to be available. Shallow clones (e.g., `git clone --depth 1`) typically lack the necessary history.
For GitHub Actions, use a full clone:
```yaml
- uses: actions/checkout@v6
with:
fetch-depth: 0
```
### Merging with upstream changes to versioned APIs
When you merge with commits that added one or more versions to the same API that you also changed locally:
* Git will report a merge conflict in the "latest" symlink. Just remove the symlink altogether (with `rm`). This will be regenerated correctly below.
* Git will report a merge conflict in the API crate in the `api_versions!` call. You will need to resolve this by hand. **This is the most important part to get right.** Generally, this is easy: you'll take all the versions that are present upstream, choose a new number for your locally-added version, and make sure that your locally-added one remains the latest one (first in the list).
* Less commonly: you may have other merge conflicts in the API crate or the server implementation. This would happen if specific endpoints were changed both upstream and locally. The details here are situation-dependent. You'll have to resolve these by hand.
Aside from merge conflicts from specific endpoints that were changed both upstream and in your local version, you generally should _not_ need to change any of the API crate as part of the merge. (This is why we use identifiers for the semvers that go in the `versions` argument -- so that the value can change after a merge without having to go update all the endpoints you changed.)
* When you've resolved all conflicts, run `cargo openapi generate` to regenerate files for locally-added versions and clean up any stale files.
Most commonly, this boils down to:
* `rm` the "latest" symlink
* fix up the `api_versions!` call in the API crate
* run `cargo openapi generate`
If you get any of this wrong, the tool should clearly report the problem. For example, if you mis-order the versions in the list, you'll get an error about them not being sequential. If you mismerge the API trait in such a way that changes a blessed version, as always, the tool will detect that and report it.
### Retiring old versions of versioned APIs
Of course, we don't need or want to support each version of an API forever. RFD 532 proposes supporting the one shipped in the last release, plus all the intermediate ones shipped in the current release. The specific policy doesn't really matter here.
To retire an old version:
1. Remove it from `api_versions!`.
2. Remove any references to its `VERSION_` identifier (these will show up as compile errors when you run `cargo openapi generate`).
3. Run `cargo openapi generate` to remove the old files.
### Converting lockstep APIs to be versioned
An existing lockstep API can be made versioned. You would do this when transitioning an API to support online update. We'll use a hypothetical example with a `dns-server` API defined in a `dns-server-api` crate:
1. Initially, its OpenAPI document is stored in `openapi/dns-server.json`.
2. Run `git rm -f openapi/dns-server.json`.
3. Run `mkdir openapi/dns-server`.
4. Update the API crate (`dns-server-api/src/lib.rs`) to use the new `api_versions!` macro. See the instructions under [_Adding new OpenAPI documents_](#adding-new-openapi-documents) above.
5. Within the integration point, update the OpenAPI manager configuration to specify that the API is now versioned. You'll use the `supported_versions()` function defined by the `api_versions!` macro.
6. Run `cargo openapi generate`. This will generate a new file under `openapi/dns-server` for your initial server version, along with a "latest" symlink.
You will probably see this warning:
```
Loading blessed OpenAPI documents from git revision "main" path "openapi"
Warning skipping file "dns-server.json": this API is not a lockstep API
```
This is okay. It's saying: this is a versioned API, but the file we found upstream (i.e., in "main") suggests it's lockstep. That's expected when you're doing this conversion.
7. Update references to the OpenAPI document elsewhere in the repo. For example, if you have a [Progenitor](https://docs.rs/progenitor/latest/progenitor/)-generated client for the OpenAPI document, it was likely reading form `openapi/dns-server.json`. The client should be updated to now generate it from `openapi/dns-server/dns-server-latest.json`.
That should be it! Now, when iterating on the API, you'll need to follow the procedure described above for versioned APIs (which is slightly more complicated than the one for lockstep APIs).
In principle, this process could be reversed to convert an API from versioned to lockstep, but this almost certainly has runtime implications that would need to be considered.
### Git stub storage details
#### What changes on disk
With Git stub storage enabled, the directory structure changes from:
```
openapi/sled-agent/
├── sled-agent-1.0.0-2da304.json
├── sled-agent-2.0.0-a3e161.json
├── sled-agent-3.0.0-f44f77.json
└── sled-agent-latest.json -> sled-agent-3.0.0-f44f77.json
```
To:
```
openapi/sled-agent/
├── sled-agent-1.0.0-2da304.json.gitstub
├── sled-agent-2.0.0-a3e161.json.gitstub
├── sled-agent-3.0.0-f44f77.json
└── sled-agent-latest.json -> sled-agent-3.0.0-f44f77.json
```
The latest version remains a full JSON file, and the `-latest.json` symlink continues to work. Older blessed versions become `.gitstub` files.
#### Git stub format
A `.gitstub` file contains a single line:
```
99c3f3ef97f80d1401c54ce0c625af125d4faef3:openapi/sled-agent/sled-agent-2.0.0-a3e161.json
```
The format is `<commit-hash>:<path>`, where the commit hash is when that version was introduced.
#### Reading Git stub contents
To view the contents of a Git stub:
```sh
git show $(cat sled-agent-2.0.0-a3e161.json.gitstub)
```
For Jujutsu:
```sh
IFS=: read -r commit path < sled-agent-2.0.0-a3e161.json.gitstub
jj file show -r "$commit" "root:$path"
```
#### When versions are converted
The API manager automatically converts versions between JSON and Git stub formats. A version is stored as a Git stub when all of the following are true:
- Git stub storage is enabled for the API.
- The version is blessed (present in the upstream branch).
- The version is not the latest.
- The version was not introduced in the same commit as the latest version.
When you add a new version locally, the previous latest version is converted to a Git stub. If you remove that new version, the conversion is reversed.
#### Git stub merge conflicts
With Git stub storage, Git detects new API versions as renames of the previous version. If parallel branches both add new versions, Git produces a rename-rename conflict.
To resolve, run `cargo openapi generate` (or your equivalent alias). The tool regenerates the correct files from your resolved `api_versions!` macro.
If you use Jujutsu, the `-latest.json` symlink becomes a regular file during conflicts. The API manager detects this and corrects it when you run `generate`.
#### Progenitor and client generation
Progenitor-generated clients should continue to reference the `-latest.json` symlink, which always points to a real JSON file. No changes are needed for typical client generation.
If you need to generate a client for an older version stored as a Git stub, you will currently need to disable Git stub storage for that API.
## Contributing
Bugfixes and other minor fixes are welcome! Before working on a major feature, please [open an issue](https://github.com/oxidecomputer/dropshot-api-manager/issues/new) to discuss it.
Tests must be run using [cargo-nextest](https://nexte.st/).