# dragoman
A web server for scholarly metadata with full [DOI Resolution](https://www.doi.org/the-identifier/resources/factsheets/doi-resolution-documentation) and [DOI content negotiation](https://citation.doi.org/docs.html). Send a DOI as the URL path; receive a redirect to the landing page or metadata in any supported format depending on the `Accept` header.
## Installation
### Prebuilt binary (recommended)
If you have [`cargo-binstall`](https://github.com/cargo-bins/cargo-binstall), install a prebuilt binary without compiling:
```bash
cargo binstall dragoman
```
Prebuilt binaries are published for **macOS arm64** (`aarch64-apple-darwin`) and **Linux amd64** (`x86_64-unknown-linux-gnu`). On any other platform, `cargo binstall` automatically falls back to building from source. The binary is installed to `~/.cargo/bin/dragoman`; make sure `~/.cargo/bin` is on your `PATH` (the Rust installer adds this automatically).
### Build from source
Requires Rust 1.75+ ([rustup.rs](https://rustup.rs)):
```bash
cargo install dragoman
```
This builds a release binary and installs it to `~/.cargo/bin/dragoman`.
## Local SQLite database
dragoman can serve metadata directly from a local SQLite database in the [commonmeta](https://commonmeta.org) format, bypassing the live Crossref/DataCite APIs. This dramatically reduces latency and API load for high-traffic deployments.
### Database format
The database is a SQLite3 file in the commonmeta store layout:
- `pid_records` holds each record as its provider delivered it — Crossref and DataCite works, ROR organizations, ORCID persons, and containers — keyed by bare pid (`10.5281/zenodo.1234`, `0000-0002-1825-0097`, `02nr0ka47`) and converted to commonmeta on read.
- `pid_relations` holds the links between them that `commonmeta enrich` derives: references and typed relations between works, and a work's persons (`hasPerson`), organizations (`hasOrganization`) and container (`hasContainer`).
- `pid_annotations` holds what `enrich` and the imports add per pid: the validated commonmeta, identifier columns such as `pmid`, `openalex` and `issn`, and the stored work counts of long works lists.
You can build a database from any commonmeta-supported source using the [commonmeta](https://codeberg.org/front-matter/commonmeta-rs) CLI.
## Running the server
### Start
```bash
# Default port 3456
dragoman start
# Custom port
dragoman start --port 8080
# With a local database
dragoman start --db /data/commonmeta-2026-06-15.sqlite3
# Write a PID file so the process can be stopped later
dragoman start --pid-file /tmp/dragoman.pid
# All options together
dragoman start --port 8080 --db /data/commonmeta-2026-06-15.sqlite3 --pid-file /tmp/dragoman.pid
```
### TLS
The `--domain` flag controls whether and how dragoman serves HTTPS.
| `--domain` value | Behaviour |
| --- | --- |
| *(not set)* | Plain HTTP on `127.0.0.1:<port>`. Use this behind a reverse proxy. |
| `localhost` | HTTPS on `127.0.0.1:<port>` using a locally-trusted certificate from [mkcert](https://github.com/FiloSottile/mkcert). |
| any public hostname | HTTPS on `0.0.0.0:<tls-port>` with automatic certificate provisioning via [Let's Encrypt](https://letsencrypt.org). Requires `--email`. |
| any public hostname, with `--tls-cert` and `--tls-key` | HTTPS on `0.0.0.0:<tls-port>` with the certificate in those PEM files, e.g. a CDN origin certificate. See [Behind a CDN](#behind-a-cdn). |
#### Local development (mkcert)
Install mkcert once and add its CA to your trust store:
```bash
brew install mkcert # macOS; see https://github.com/FiloSottile/mkcert for Linux/Windows
mkcert -install
```
Then start dragoman with `--domain localhost`:
```bash
dragoman start --domain localhost
```
dragoman runs `mkcert` automatically on the first start; subsequent starts reuse the cached certificate. Certificates are stored in `--acme-cache` (default `/var/lib/dragoman/acme`).
#### Production (Let's Encrypt)
dragoman provisions and renews certificates automatically using the [ACME TLS-ALPN-01 challenge](https://datatracker.ietf.org/doc/html/rfc8737). No separate HTTP challenge server or reverse proxy is needed.
```bash
dragoman start \
--port 80 \
--domain commonmeta.org \
--email admin@example.org
```
- Binds **HTTPS on `--tls-port`** (default 443) with automatic certificate provisioning.
- Binds **HTTP on `--port`** with a permanent redirect to `https://commonmeta.org/…`.
- Stores certificates in `--acme-cache` (default `/var/lib/dragoman/acme`) so they survive restarts.
Use `--acme-staging` while testing to avoid Let's Encrypt rate limits:
```bash
dragoman start \
--port 80 \
--domain commonmeta.org \
--email admin@example.org \
--acme-staging
```
#### Behind a CDN
A CDN that proxies traffic answers the TLS handshake itself, so Let's Encrypt's TLS-ALPN-01 challenge never reaches dragoman and renewal fails. Serve a certificate the CDN trusts instead, such as an origin certificate it issues, and tell dragoman which peers are the CDN's edges, so that [request limits](#request-limits) count the clients behind them rather than the edges:
```bash
dragoman start \
--port 80 \
--domain commonmeta.org \
--tls-cert /etc/dragoman/origin.pem \
--tls-key /etc/dragoman/origin-key.pem \
--trusted-proxies "$(scripts/cloudflare-trusted-proxies.sh)" \
--client-ip-header CF-Connecting-IP
```
`--trusted-proxies` takes the CDN's published edge networks and `--client-ip-header` the header the CDN puts the client's address in. [`scripts/cloudflare-trusted-proxies.sh`](scripts/cloudflare-trusted-proxies.sh) prints Cloudflare's full list from [its published ranges](https://www.cloudflare.com/ips/); `--env` prints it as a line for `/etc/dragoman/env`:
```bash
sudo sed -i '/^DRAGOMAN_TRUSTED_PROXIES=/d' /etc/dragoman/env
scripts/cloudflare-trusted-proxies.sh --env | sudo tee -a /etc/dragoman/env
```
Cloudflare changes its ranges rarely; rerun the script now and then, and restart dragoman when the list changes. The certificate files are also read at startup only, so restart dragoman after replacing them. See [Caching](#caching) for the headers a CDN keys on.
##### Cloudflare Tunnel
With [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/), `cloudflared` on the server dials out to Cloudflare and hands requests to dragoman locally, so the server accepts no inbound HTTP at all and needs no certificate. Run dragoman without `--domain`, on plain HTTP at `127.0.0.1`, point the tunnel's ingress at it (`service: http://127.0.0.1:3456`), and name the header Cloudflare sets:
```bash
dragoman start --port 3456 --client-ip-header CF-Connecting-IP
```
`cloudflared` connects from this machine, which dragoman always trusts, so `--trusted-proxies` is not needed. `--client-ip-header` is: without it dragoman would take `X-Real-IP` from the local peer, and Cloudflare passes on whatever `X-Real-IP` the client sent.
##### Firewall
Behind a CDN, only the CDN should reach dragoman. The [Hetzner Robot firewall](https://docs.hetzner.com/robot/dedicated-server/firewall/) of a dedicated server takes at most 10 rules, too few for Cloudflare's 22 ranges, so filter on the server instead — or use Cloudflare Tunnel and close the HTTP ports altogether. The Robot firewall then only needs SSH and, without a tunnel, 443.
With nftables, let SSH in from anywhere and 443 only from Cloudflare. `--nft` prints the ranges as nftables variables:
```bash
scripts/cloudflare-trusted-proxies.sh --nft | sudo tee /etc/nftables.d/cloudflare.nft
```
```nft
#!/usr/sbin/nft -f
flush ruleset
include "/etc/nftables.d/cloudflare.nft"
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
iif lo accept
ct state established,related accept
meta l4proto { icmp, ipv6-icmp } accept
tcp dport 22 accept
ip saddr $cloudflare_v4 tcp dport 443 accept
ip6 saddr $cloudflare_v6 tcp dport 443 accept
}
}
```
Unlike the Robot firewall, nftables tracks connections, so the replies to dragoman's own requests to Crossref and DataCite need no rule of their own.
Options can also be supplied as environment variables (flags take precedence):
```bash
PORT=8080 COMMONMETA_DB=/data/commonmeta-2026-06-15.sqlite3 RUST_LOG=dragoman=debug dragoman start
```
During development you can use `cargo run` in place of the installed binary:
```bash
# Run from the project root — the sqlite3 file in the root is loaded by filename
cargo run -- start --db commonmeta-2026-06-15.sqlite3
# Or with a full path
cargo run -- start --db /data/commonmeta-2026-06-15.sqlite3
```
### Error: port already in use
If the chosen port is already in use, the server logs an error and exits:
```text
ERROR dragoman: failed to bind port=3456 error=Address already in use (os error 48)
```
Choose a different port with `--port` or stop the process that holds the port.
### Database file not found
If the resolved database path does not exist, dragoman logs a warning and starts without a local database, falling back to the live API for all requests:
```text
WARN dragoman: sqlite file not found, running without local database path=…
```
To use a local database, place the SQLite file at the platform default path or set `COMMONMETA_DB` to its location.
### Stop
```bash
# Stop using the default PID file location (/tmp/dragoman.pid)
dragoman stop
# Stop using a custom PID file
dragoman stop --pid-file /var/run/dragoman.pid
```
`dragoman stop` sends `SIGTERM` to the running process. The server handles the signal gracefully: it finishes in-flight requests and removes the PID file before exiting. Pressing `Ctrl-C` has the same effect.
## CLI reference
```text
dragoman <COMMAND>
Commands:
start Start the server (runs in the foreground)
stop Stop a running server by sending SIGTERM to its PID file
import Import scholarly metadata into the local commonmeta database
convert Convert a metadata record from one format to another
list List records stored in the local commonmeta database
stats Show record counts and other statistics for the local database
settings Show key/value settings stored in the local SQLite database
help Print help
dragoman start [OPTIONS]
-p, --port <PORT> TCP port to listen on [env: PORT] [default: 3456]
-d, --db <PATH> Local commonmeta SQLite3 database [env: COMMONMETA_DB]
--pid-file <PATH> Write PID to this file on startup [env: DRAGOMAN_PID_FILE]
--domain <DOMAIN> Domain for HTTPS. Use `localhost` for local dev (mkcert) or a public hostname for Let's Encrypt [env: DRAGOMAN_DOMAIN]
--email <EMAIL> ACME contact email for Let's Encrypt (required for public domains) [env: DRAGOMAN_EMAIL]
--acme-cache <PATH> Directory for ACME certificate storage [env: DRAGOMAN_ACME_CACHE] [default: /var/lib/dragoman/acme]
--acme-staging Use Let's Encrypt staging (testing only) [env: DRAGOMAN_ACME_STAGING]
--tls-port <PORT> HTTPS port when --domain is set [env: DRAGOMAN_TLS_PORT] [default: 443]
--rate-pages <R/B> Per-client limit on entity pages [env: DRAGOMAN_RATE_PAGES] [default: 20/60]
--rate-lists <R/B> Per-client limit on works lists, facets, counts [env: DRAGOMAN_RATE_LISTS] [default: 5/20]
--rate-expensive <R/B> Per-client limit on search, handles, exports, conversion [env: DRAGOMAN_RATE_EXPENSIVE] [default: 2/20]
--max-in-flight <N> Requests served at once, across all clients [env: DRAGOMAN_MAX_IN_FLIGHT] [default: 256]
--request-timeout <S> Seconds a response may take [env: DRAGOMAN_REQUEST_TIMEOUT] [default: 60]
--header-timeout <S> Seconds for the TLS handshake and request headers [env: DRAGOMAN_HEADER_TIMEOUT] [default: 10]
--max-conns-per-ip <N> Open connections per client [env: DRAGOMAN_MAX_CONNS_PER_IP] [default: 32]
--trusted-proxies <NETS> Proxies that may name the client, e.g. a CDN's edges [env: DRAGOMAN_TRUSTED_PROXIES]
--client-ip-header <NAME> Header a trusted proxy names the client in [env: DRAGOMAN_CLIENT_IP_HEADER]
--tls-cert <PATH> PEM certificate for --domain instead of Let's Encrypt [env: DRAGOMAN_TLS_CERT]
--tls-key <PATH> PEM private key for --tls-cert [env: DRAGOMAN_TLS_KEY]
--reuse-port Share the listening ports with other dragoman processes [env: DRAGOMAN_REUSE_PORT]
dragoman stop [OPTIONS]
--pid-file <PATH> PID file to read [env: DRAGOMAN_PID_FILE] [default: /tmp/dragoman.pid]
```
Beyond `start` and `stop`, dragoman embeds the full [commonmeta](https://codeberg.org/front-matter/commonmeta-rs) CLI. `import`, `convert`, `list`, `stats` and `settings` are listed above for convenience, and **any other commonmeta subcommand** (`build`, `enrich`, `validate`, `push`, `migrate`, …) is forwarded verbatim — no need to install commonmeta separately. Each command's own flags are passed straight through:
```sh
dragoman import --from crossref --member 340
dragoman build --from crossref --journals
dragoman enrich --crossref
dragoman convert 10.5555/12345678 --to bibtex
dragoman stats
```
Run `dragoman <command> --help` for a command's full options.
## Environment variables
All flags can also be supplied as environment variables; flags take precedence. See [`env.example`](env.example) for an annotated template.
| Variable | Default | Description |
| --- | --- | --- |
| `PORT` | `3456` | TCP port to listen on. |
| `COMMONMETA_DB` | *(platform default)* | Path to a local commonmeta SQLite3 database. Served before falling back to the live API. Platform defaults: macOS → `~/Library/Application Support/commonmeta/commonmeta.sqlite3`; Linux → `/var/lib/commonmeta/commonmeta.sqlite3`. |
| `DRAGOMAN_CACHE_DB` | *(platform default)* | Path to the dragoman cache database for API fallback results. Platform defaults: macOS → `~/Library/Application Support/commonmeta/cache.sqlite3`; Linux → `/var/lib/commonmeta/cache.sqlite3`. |
| `DRAGOMAN_PID_FILE` | *(none)* | Path for the PID file written by `start` and read by `stop`. |
| `DRAGOMAN_DOMAIN` | *(none)* | Domain for HTTPS. `localhost` → mkcert; public hostname → Let's Encrypt. Unset → plain HTTP on `127.0.0.1` (reverse proxy mode). |
| `DRAGOMAN_EMAIL` | *(none)* | ACME contact email. Required when `DRAGOMAN_DOMAIN` is a public hostname. |
| `DRAGOMAN_ACME_CACHE` | *(platform default)* | Directory for ACME/mkcert certificates. Platform defaults: macOS → `~/Library/Application Support/dragoman/acme`; Linux → `/var/lib/dragoman/acme`. |
| `DRAGOMAN_ACME_STAGING` | *(false)* | Set to any non-empty value to use the Let's Encrypt staging environment. |
| `DRAGOMAN_TLS_PORT` | `443` / `3456` | HTTPS port when `DRAGOMAN_DOMAIN` is set. Defaults to `3456` for `localhost`, `443` for public domains. |
| `DRAGOMAN_VIKTORIALOGS_URL` | *(none)* | VictoriaLogs OTLP/HTTP endpoint. When unset, logs go to stdout only. |
| `DRAGOMAN_METRICS_ALLOW` | *(none)* | Comma-separated IPs allowed to scrape `/metrics`. When unset, the route is not registered. |
| `DRAGOMAN_RATE_PAGES` | `20/60` | Per-client limit on entity pages, as requests per second / burst. `0` turns it off. See [Request limits](#request-limits). |
| `DRAGOMAN_RATE_LISTS` | `5/20` | Per-client limit on works lists, facets and counts. |
| `DRAGOMAN_RATE_EXPENSIVE` | `2/20` | Per-client limit on search, handle lookups, exports, bibliographies and format conversion. |
| `DRAGOMAN_MAX_IN_FLIGHT` | `256` | Requests served at once across all clients; past it, `503`. `0` turns the cap off. |
| `DRAGOMAN_REQUEST_TIMEOUT` | `60` | Seconds a response may take before it is abandoned with `503`. `0` turns it off. |
| `DRAGOMAN_HEADER_TIMEOUT` | `10` | Seconds a client has to finish the TLS handshake, send its first request and each request's headers. HTTPS with a public domain only. |
| `DRAGOMAN_MAX_CONNS_PER_IP` | `32` | Open connections per client; an IPv6 client is its /64. HTTPS with a public domain only. `0` turns the cap off. |
| `DRAGOMAN_TRUSTED_PROXIES` | *(none)* | Comma-separated addresses and CIDR networks of proxies that may name the client, such as a CDN's edges; this machine is always trusted. Their connections are exempt from `DRAGOMAN_MAX_CONNS_PER_IP`. |
| `DRAGOMAN_CLIENT_IP_HEADER` | *(none)* | Header a trusted proxy names the client in, e.g. `CF-Connecting-IP`. |
| `DRAGOMAN_TLS_CERT` | *(none)* | PEM certificate (chain) to serve for `DRAGOMAN_DOMAIN` instead of one from Let's Encrypt. Requires `DRAGOMAN_TLS_KEY`. |
| `DRAGOMAN_TLS_KEY` | *(none)* | PEM private key for `DRAGOMAN_TLS_CERT`. |
| `DRAGOMAN_REUSE_PORT` | *(false)* | Set to `true` to share the listening ports with other dragoman processes (`SO_REUSEPORT`). See [Several processes on one port](#several-processes-on-one-port). |
| `RUST_LOG` | `dragoman=info` | Log filter (see [`tracing-subscriber`](https://docs.rs/tracing-subscriber)). Use `dragoman=debug` for per-request cache hits. |
## DOI Resolution
dragoman implements the [Handle System resolution protocol](https://www.doi.org/the-identifier/resources/factsheets/doi-resolution-documentation). Two entry points are available for each PID:
| Endpoint | Purpose |
| --- | --- |
| `/{path}` | Render the entity page (HTML · Svelte UI) or content-negotiate (API) |
| `/api/handles/{path}` | Handle resolver: redirect to URL of record, content negotiation, or `?noredirect` |
### Entity page — `/{path}`
Visiting a PID in a browser renders the dragoman entity page (Svelte UI with rich metadata). Sending an API `Accept` header to the same URL triggers content negotiation instead (see [Content negotiation](#content-negotiation)).
```bash
# Open entity page in a browser
open http://localhost:3456/10.5281/zenodo.1089100
# Works for ROR organisation IDs and ORCID person IDs too
open http://localhost:3456/027k65916
open http://localhost:3456/0000-0001-5727-2427
```
Paths that are not recognised PIDs redirect to the search page:
```bash
# Non-PID path → 307 to /?q=<path>
curl -v http://localhost:3456/machine-learning
```
### Handle resolver — `/api/handles/{path}`
The `/api/handles/` endpoint is the programmatic resolution interface. Three outcomes are possible depending on the request:
#### Redirect to URL of record (default)
Without a recognised `Accept` header, `/api/handles/{path}` redirects to the PID's URL of record:
| PID type | Redirect target |
| --- | --- |
| DOI | URL from metadata (`data.url`); fallback `https://doi.org/{doi}` |
| ROR | Organisation website (from database); fallback `https://ror.org/{id}` |
| ORCID | `https://orcid.org/{id}` |
```bash
# Follow the redirect to the landing page
curl -L http://localhost:3456/api/handles/10.5281/zenodo.1089100
# Inspect the redirect target without following
curl -s -o /dev/null -w "%{redirect_url}" http://localhost:3456/api/handles/10.5281/zenodo.1089100
# https://zenodo.org/record/1089100
```
#### View entity page — `?noredirect`
The `?noredirect` parameter redirects the browser to `/{path}` to render the entity page:
```bash
curl -v "http://localhost:3456/api/handles/10.5281/zenodo.1089100?noredirect"
# 307 → http://localhost:3456/10.5281/zenodo.1089100
```
#### Content negotiation at `/api/handles/{path}`
Send a recognised `Accept` header to receive metadata in a specific format.
##### DOI
```bash
curl -H "Accept: application/x-bibtex" \
http://localhost:3456/api/handles/10.5281/zenodo.1089100
curl -H "Accept: application/vnd.commonmeta+json" \
http://localhost:3456/api/handles/10.5281/zenodo.1089100
```
See [Supported formats](#supported-formats) for the full list. The `?format=` override is also accepted.
##### ROR (organisations)
| `Accept` header | `?format=` value | Output |
| --- | --- | --- |
| `application/json` | `ror` | ROR native JSON |
| `application/vnd.commonmeta+json` | `commonmeta` | commonmeta JSON |
```bash
# ROR native JSON
curl -H "Accept: application/json" \
http://localhost:3456/api/handles/027k65916
# commonmeta JSON
curl -H "Accept: application/vnd.commonmeta+json" \
http://localhost:3456/api/handles/027k65916
# ?format= override
curl "http://localhost:3456/api/handles/027k65916?format=ror"
```
##### ORCID (people)
| `Accept` header | `?format=` value | Output |
| --- | --- | --- |
| `application/vnd.orcid+json` | `orcid` | commonmeta JSON *(native ORCID JSON planned)* |
| `application/json` | `orcid` | commonmeta JSON *(native ORCID JSON planned)* |
| `application/vnd.commonmeta+json` | `commonmeta` | commonmeta JSON |
```bash
# ORCID JSON (currently returns commonmeta JSON; native ORCID JSON planned)
curl -H "Accept: application/vnd.orcid+json" \
http://localhost:3456/api/handles/0000-0001-5727-2427
# commonmeta JSON
curl -H "Accept: application/vnd.commonmeta+json" \
http://localhost:3456/api/handles/0000-0001-5727-2427
# ?format= override
curl "http://localhost:3456/api/handles/0000-0001-5727-2427?format=orcid"
```
## Usage
### Content negotiation
Send an `Accept` header to `/{path}` to receive metadata for a DOI without a browser redirect.
#### BibTeX
```bash
curl -H "Accept: application/x-bibtex" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### RIS
```bash
curl -H "Accept: application/x-research-info-systems" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### CSL (Citeproc) JSON
```bash
curl -H "Accept: application/vnd.citationstyles.csl+json" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### Crossref
```bash
curl -H "Accept: application/vnd.crossref+json" \
http://localhost:3456/10.1016/j.jaci.2019.09.015
```
#### Crossref XML
```bash
curl -H "Accept: application/vnd.crossref.unixref+xml" \
http://localhost:3456/10.1016/j.jaci.2019.09.015
```
#### DataCite
```bash
curl -H "Accept: application/vnd.datacite.datacite+json" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### Schema.org JSON-LD
```bash
curl -H "Accept: application/vnd.schemaorg.ld+json" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### InvenioRDM
```bash
curl -H "Accept: application/vnd.inveniordm.v1+json" \
http://localhost:3456/10.5281/zenodo.1089100
```
#### Formatted citation
`text/x-bibliography` accepts optional `style=` and `locale=` parameters. Style names come from the [CSL style repository](https://github.com/citation-style-language/styles); locale codes from the [CSL locales repository](https://github.com/citation-style-language/locales).
```bash
# APA (default)
curl -H "Accept: text/x-bibliography; style=apa" \
http://localhost:3456/10.5281/zenodo.1089100
# Vancouver in French
curl -H "Accept: text/x-bibliography; style=vancouver; locale=fr-FR" \
http://localhost:3456/10.5281/zenodo.1089100
```
### Query parameter overrides
Use `?format=` instead of an `Accept` header:
```bash
curl "http://localhost:3456/10.5281/zenodo.1089100?format=bibtex"
curl "http://localhost:3456/10.5281/zenodo.1089100?format=citation&style=apa&locale=de-DE"
```
### Format-specific URL prefixes
Two URL prefixes return a fixed format without requiring an `Accept` header or `?format=` parameter:
| Prefix | Output format | Content-Type |
| --- | --- | --- |
| `/dois/{doi}` | DataCite JSON | `application/vnd.datacite.datacite+json` |
| `/works/{doi}` | Crossref JSON | `application/vnd.crossref+json` |
```bash
curl http://localhost:3456/dois/10.1371/journal.pcbi.1000204
curl http://localhost:3456/works/10.1371/journal.pcbi.1000204
```
These routes are equivalent to `/{doi}?format=datacite` and `/{doi}?format=crossref` respectively. The `?source=` override is still accepted if you need to force a specific registration agency.
### Bulk export — `/api/export`
One request returns **every work of one entity** in a single document, in any of
the formats below. The entity is named by `id`, spelled exactly as its page path
spells it — a ROR id, an ORCID iD, or a container's ISSN, OpenAlex source id or
DOI:
```bash
# Every work of an organization as BibTeX
curl "http://localhost:3456/api/export?id=00pd74e08&format=bibtex"
# A person's works as RIS, a journal's as formatted citations
curl "http://localhost:3456/api/export?id=0000-0003-0530-4305&format=ris"
curl "http://localhost:3456/api/export?id=2749-9952&format=citation&style=apa&locale=de-DE"
```
This is what the entity page's Export and Copy buttons call. The works list on
the page is paged (`/api/{org,person}/{id}/works`, `/api/container/works`), so
exporting from the client would have covered only the pages loaded so far, at
one request per work.
| Response header | Meaning |
| --- | --- |
| `X-Total-Count` | The entity's full works total |
| `X-Exported-Count` | How many the body holds |
| `X-Export-Truncated` | Present when the export hit the ceiling |
The two counts also differ for an ordinary reason: a work superseded by a newer
version is left out. `X-Export-Truncated` is what distinguishes truncation —
an export writes at most 5000 works, or 1000 for `citation`, which renders every
record through the CSL processor.
## Supported formats
### DOI formats (works)
| `Accept` header | `?format=` value | Notes |
| --- | --- | --- |
| `application/x-bibtex` | `bibtex` | |
| `text/x-bibliography` | `citation` | `style=` and `locale=` params |
| `application/vnd.commonmeta+json` | `commonmeta` | |
| `application/vnd.crossref+json` | `crossref` | |
| `application/vnd.crossref.unixref+xml` | `crossref_xml` | |
| `application/vnd.crossref.unixsd+xml` | `crossref_xml` | alias |
| `application/vnd.citationstyles.csl+json` | `csl` | |
| `application/vnd.datacite.datacite+json` | `datacite` | |
| `application/vnd.datacite.datacite+xml` | `datacite_xml` | |
| `application/vnd.inveniordm.v1+json` | `inveniordm` | |
| `application/x-research-info-systems` | `ris` | |
| `application/vnd.schemaorg.ld+json` | `schemaorg` | |
| `text/html` / *(absent)* at `/{path}` | — | Svelte entity page |
| `text/html` / *(absent)* at `/api/handles/{path}` | — | 307 redirect to URL of record |
### ROR formats (organisations) — `/api/handles/{id}` only
| `Accept` header | `?format=` value | Notes |
| --- | --- | --- |
| `application/json` | `ror` | ROR native JSON |
| `application/vnd.commonmeta+json` | `commonmeta` | |
### ORCID formats (people) — `/api/handles/{id}` only
| `Accept` header | `?format=` value | Notes |
| --- | --- | --- |
| `application/vnd.orcid+json` | `orcid` | commonmeta JSON *(native ORCID JSON planned)* |
| `application/json` | `orcid` | commonmeta JSON *(native ORCID JSON planned)* |
| `application/vnd.commonmeta+json` | `commonmeta` | |
## HTTP status codes
| Code | Meaning |
| --- | --- |
| 200 | Metadata returned |
| 307 | Redirect (entity page, URL of record, or search) |
| 404 | PID not found or path not recognised |
| 406 | Requested content type not supported |
| 429 | Client over its [request limit](#request-limits); see `Retry-After` |
| 502 | Upstream API error |
| 503 | Server busy or response timed out; see `Retry-After` |
## Deployment (macOS)
#### Place the database
The platform default path on macOS is `~/Library/Application Support/commonmeta/commonmeta.sqlite3`. Place the database there and no further configuration is needed:
```bash
mkdir -p "$HOME/Library/Application Support/commonmeta"
cp commonmeta.sqlite3 "$HOME/Library/Application Support/commonmeta/commonmeta.sqlite3"
```
To use a different path, set `COMMONMETA_DB` in the launchd plist or pass `--db` on the command line.
#### Run as a background service (launchd)
```bash
# Start at login and keep alive
brew services start dragoman
# Check status
brew services info dragoman
# View logs
tail -f "$(brew --prefix)/var/log/dragoman.log"
# Stop the service
brew services stop dragoman
```
`brew services start` installs a launchd plist in `~/Library/LaunchAgents/` and starts the service immediately. It restarts automatically on crash and at login.
To run as a system-level daemon (starts at boot, not tied to a user login), use `sudo brew services start dragoman`. This installs the plist in `/Library/LaunchDaemons/` instead.
#### Configuration
To change the port or other settings, edit the service environment variables and restart:
```bash
# Open the generated plist for editing
open "$(brew --prefix)/opt/dragoman/homebrew.mxcl.dragoman.plist"
brew services restart dragoman
```
### Manual installation (without Homebrew)
```bash
# Install Rust if not already installed
curl https://sh.rustup.rs -sSf | sh
cargo install dragoman
sudo install -m 755 ~/.cargo/bin/dragoman /opt/homebrew/bin/dragoman
```
> **Intel Macs:** replace `/opt/homebrew` with `/usr/local` in all paths below.
#### Run as a launchd daemon
The bundled `com.front-matter.dragoman.plist` targets Apple Silicon paths.
```bash
sudo mkdir -p /opt/homebrew/var/dragoman /opt/homebrew/var/log
sudo cp commonmeta.sqlite3 /opt/homebrew/var/dragoman/commonmeta.sqlite3
sudo cp com.front-matter.dragoman.plist /Library/LaunchDaemons/
sudo launchctl load -w /Library/LaunchDaemons/com.front-matter.dragoman.plist
```
Check logs:
```bash
tail -f /opt/homebrew/var/log/dragoman.log
```
Stop and disable:
```bash
sudo launchctl unload -w /Library/LaunchDaemons/com.front-matter.dragoman.plist
```
### Updating
#### With Homebrew
```bash
brew upgrade dragoman
brew services restart dragoman
```
#### Manual
```bash
cargo install dragoman
sudo install -m 755 ~/.cargo/bin/dragoman /opt/homebrew/bin/dragoman
sudo launchctl kickstart -k system/com.front-matter.dragoman
```
---
## Deployment (Debian / systemd)
This section covers running dragoman as a persistent system service on a Debian 13 server.
### 1. Build the binary
On the server, install Rust and install the binary:
```bash
curl https://sh.rustup.rs -sSf | sh
source ~/.cargo/env
sudo $HOME/.cargo/bin/cargo install --root /usr/local dragoman
```
`--root /usr/local` writes the binary straight to `/usr/local/bin/dragoman`,
so there is no second copy to forget on an upgrade. It has to live outside a
home directory: the service runs as `User=dragoman` with `ProtectHome=true`, so
`/root` and `/home` are invisible to it and a binary under `~/.cargo/bin` could
not be executed at all.
Or cross-compile locally and copy the binary:
```bash
# macOS → Linux x86-64 (requires cross)
cargo install cross
cross build --release --target x86_64-unknown-linux-gnu
scp target/x86_64-unknown-linux-gnu/release/dragoman user@server:/tmp/dragoman
ssh user@server 'sudo install -m 755 /tmp/dragoman /usr/local/bin/dragoman'
```
### 2. Create system user and directories
```bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin dragoman
sudo mkdir -p /var/lib/commonmeta
sudo chown dragoman:dragoman /var/lib/commonmeta
```
### 3. Place the SQLite database
The platform default path on Linux is `/var/lib/commonmeta/commonmeta.sqlite3`. Place the database there and no further configuration is needed:
```bash
sudo cp commonmeta.sqlite3 /var/lib/commonmeta/commonmeta.sqlite3
sudo chown dragoman:dragoman /var/lib/commonmeta/commonmeta.sqlite3
```
To use a different path, set `COMMONMETA_DB` in the environment file instead.
### 4. Create the environment file
dragoman reads `/etc/default/commonmeta`, shared with the commonmeta importer
service. Copy the bundled example and edit it for your deployment:
```bash
sudo cp env.example /etc/default/commonmeta
sudo chmod 640 /etc/default/commonmeta
sudo chown root:dragoman /etc/default/commonmeta
sudo nano /etc/default/commonmeta # uncomment and fill in the variables you need
```
The cache database variable is `DRAGOMAN_CACHE_DB` — `CACHE_DB` is not read.
For a production server with Let's Encrypt, at minimum set:
```bash
DRAGOMAN_DOMAIN=commonmeta.org
DRAGOMAN_EMAIL=admin@example.org
```
The systemd unit grants write access only to `/var/lib/commonmeta`
(`ReadWritePaths`), where both databases live. The ACME certificate cache
defaults to `/var/lib/dragoman/acme`, which is outside that path — for TLS,
point it inside:
```bash
DRAGOMAN_ACME_CACHE=/var/lib/commonmeta/acme
```
See [Environment variables](#environment-variables) for the full reference.
### 5. Install and enable the systemd unit
`dragoman.service` is included in the source repository but not installed by `cargo install`. Download it directly:
```bash
sudo curl -fsSL https://codeberg.org/front-matter/dragoman/raw/branch/main/dragoman.service \
-o /etc/systemd/system/dragoman.service
sudo systemctl daemon-reload
sudo systemctl enable --now dragoman
```
Or, if you have the source checked out locally:
```bash
sudo cp dragoman.service /etc/systemd/system/dragoman.service
sudo systemctl daemon-reload
sudo systemctl enable --now dragoman
```
Check the service is running:
```bash
sudo systemctl status dragoman
sudo journalctl -u dragoman -f
```
### Several processes on one port
With `--reuse-port`, several dragoman processes bind the same ports and the kernel hands each new connection to one of them, so one machine serves from as many processes as it has use for. Each still does its own TLS; no proxy sits in front. [`dragoman@.service`](dragoman@.service) is a template unit that starts one process per instance:
```bash
sudo cp dragoman@.service /etc/systemd/system/
sudo systemctl disable --now dragoman # the single process cannot share its ports
sudo systemctl daemon-reload
sudo systemctl enable --now dragoman@{1..4}
```
Restart them one at a time and the others keep serving: `for i in 1 2 3 4; do sudo systemctl restart dragoman@$i; sleep 5; done`.
The processes share nothing but the ports and the databases:
- Each keeps its own [request limit](#request-limits) buckets. A client's connections land on different processes, so with four processes a client can get up to four times its limit; divide the `DRAGOMAN_RATE_*` values by the number of processes to keep the per-client budget.
- Each keeps its own redirect cache and its own connections to the databases.
- With Let's Encrypt, each process orders and renews its certificate on its own, and the TLS-ALPN-01 challenge connects to whichever process the kernel picks — most often not the one that asked, so renewals fail at random. Use `--tls-cert` and `--tls-key` (a CDN origin certificate, or one renewed elsewhere) with `--reuse-port`; dragoman warns at startup otherwise.
Linux spreads new connections evenly across the processes. macOS accepts `--reuse-port` but does not balance connections the same way, so measure on Linux.
### Updating the binary
```bash
sudo $HOME/.cargo/bin/cargo install --root /usr/local --force dragoman
sudo systemctl restart dragoman
```
`--force` is what lets `cargo install` overwrite the binary already there.
Restarting picks up the new server; the hourly import job takes it on its next
run, since both units point at the same path.
### Updating the database
The database file can be replaced while the service is running without restarting. dragoman opens a new read-only SQLite connection per request, so an atomic rename takes effect immediately for new requests while in-flight requests finish against the old file.
Copy the new file to a staging path on the same filesystem, then rename it over the live file:
```bash
sudo cp commonmeta-new.sqlite3 /var/lib/commonmeta/commonmeta_new.sqlite3
sudo chown dragoman:dragoman /var/lib/commonmeta/commonmeta_new.sqlite3
sudo mv /var/lib/commonmeta/commonmeta_new.sqlite3 /var/lib/commonmeta/commonmeta.sqlite3
```
If the source database uses WAL mode (has accompanying `-wal`/`-shm` files), produce a clean single-file snapshot first with `VACUUM INTO` before copying:
```bash
sqlite3 /path/to/source.sqlite3 "VACUUM INTO '/tmp/commonmeta_clean.sqlite3'"
```
### Hourly cache import (cron)
As it serves requests, dragoman writes newly fetched records into its cache
database. Run `dragoman build --from cache` periodically to fold them into the
main `commonmeta.sqlite3`: it copies the cached works, people, and organizations
into the main database and then empties the cache. This keeps later lookups fully
local and bounds the cache size. dragoman opens a fresh read-only connection per
request, so the job can write the live database with no restart — new requests
pick up the merged records immediately.
`build` is an `import` followed by an `enrich` pass over what it just imported,
which is what this job wants. The enrich derives `pid_relations` and
`pid_annotations`, and dragoman reads both: relations drive the references and
citations panels, and `pid_annotations` supplies the navigable container id —
ISSN, DOI, OpenAlex source id or DataCite client id — without which a container
is dropped from search results. A plain `import --from cache` folds the records
in but leaves them with no relation graph and no validated commonmeta overlay
until something else enriches them. `--from cache` enriches only the records it
imported, not the whole store; `--full` forces a whole-store rescan.
No extra binary is needed: dragoman embeds the commonmeta CLI and forwards
`build` to it verbatim, so the `dragoman` already installed at
`/usr/local/bin/dragoman` runs the job. It must run as a user that can
write both the main database and the cache — typically `root`, since
`commonmeta.sqlite3` is root-owned.
Set `COMMONMETA_DB` **and** `DRAGOMAN_CACHE_DB` in `/etc/default/commonmeta`
before enabling the job. `build` takes the destination from `COMMONMETA_DB`
only; left unset it falls back to the *per-user* data directory
(`/root/.local/share/commonmeta/commonmeta.sqlite3` under `root`), so the job
would succeed while quietly filling a database nothing serves.
```cron
# /etc/cron.d/dragoman-cache-import
SHELL=/bin/sh
# Fold dragoman's freshly-cached records into the main database every hour.
# `flock -n` skips a run while the previous one is still going, so a slow run
# never overlaps the next. This matters more for `build` than for a bare
# `import`: the enrich pass writes too, and an interrupted `build --from cache`
# is what left a 166 GB WAL on the production store, which then served
# `database is locked` for plain SELECTs.
# `export` matters: the env file assigns without exporting, so sourcing it alone
# leaves COMMONMETA_DB a shell variable the job would never see.
0 * * * * root . /etc/default/commonmeta && export COMMONMETA_DB && \
flock -n /run/dragoman-import.lock \
/usr/local/bin/dragoman build --from cache \
--cache-db "$DRAGOMAN_CACHE_DB" \
>> /var/log/dragoman-import.log 2>&1
```
The paths come from `/etc/default/commonmeta` — the same env file dragoman uses.
`build` reads the cache path from `CACHE_DB`, but the env file defines
`DRAGOMAN_CACHE_DB`, so the job passes it explicitly with `--cache-db` and the
file works unmodified.
On a systemd host a timer is the more idiomatic alternative: it reuses the same
`EnvironmentFile`, logs to the journal (`journalctl -u dragoman-import`),
serialises runs on its own, and catches up a missed run with `Persistent=true`.
```ini
# /etc/systemd/system/dragoman-import.service
[Unit]
Description=Import and enrich dragoman's cached records into the main commonmeta database
RequiresMountsFor=/var/lib/commonmeta
[Service]
Type=oneshot
EnvironmentFile=/etc/default/commonmeta
ExecStart=/usr/local/bin/dragoman build --from cache --cache-db ${DRAGOMAN_CACHE_DB}
```
```ini
# /etc/systemd/system/dragoman-import.timer
[Unit]
Description=Hourly dragoman cache import
[Timer]
OnCalendar=hourly
Persistent=true
[Install]
WantedBy=timers.target
```
Both unit files are bundled in the repository (`dragoman-import.service` and
`dragoman-import.timer`). Copy them into `/etc/systemd/system/` and enable the
timer:
```bash
sudo cp dragoman-import.service dragoman-import.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now dragoman-import.timer
sudo systemctl list-timers dragoman-import.timer # confirm the next run
```
These units were named `commonmeta-import.*` before, and the cron file
`commonmeta-cache-import`. On a host that still has the old ones, remove them
before enabling the new ones — two hourly importers writing the same database is
the overlap `flock` exists to prevent, and it can balloon the WAL:
```bash
sudo systemctl disable --now commonmeta-import.timer
sudo rm -f /etc/systemd/system/commonmeta-import.{service,timer} \
/etc/cron.d/commonmeta-cache-import
sudo systemctl daemon-reload
```
### Reverse proxy
When running behind a reverse proxy, do **not** set `--domain`. Let the proxy handle TLS termination; dragoman listens on plain HTTP at `127.0.0.1:3456` (or whatever `--port` you set). The proxy must set `X-Real-IP` (or append to `X-Forwarded-For`) so that [request limits](#request-limits) count each client rather than the proxy; connection caps and header timeouts are then the proxy's to set.
### Caching
The pid routes tell browsers and CDNs how long to keep their answers:
| Answer | `Cache-Control` |
| --- | --- |
| a record in a format (`200`) | `public, max-age=3600, s-maxage=86400, stale-while-revalidate=86400`: an hour in a browser, a day in a CDN |
| an entity page | `no-cache`: kept, but revalidated on every use |
| a `302` or `307`, e.g. a DOI to its landing page | `public, max-age=0, s-maxage=3600`: an hour in a CDN, revalidated by a browser every time |
| `404`, `406` | `public, max-age=300` |
| `5xx`, e.g. Crossref or DataCite failing | `no-store` |
| `429`, `503` from the [request limits](#request-limits) | `no-store` |
Every `200` carries a weak `ETag` of its body, and a request whose `If-None-Match` names it gets `304 Not Modified` with no body. An entity page still renders on the server; the 304 saves sending it.
Every answer from `/{pid}` and `/api/handles/{pid}` — pages, formats, the `302` to a landing page, the `404` — carries `Vary: Accept`, since each depends on the Accept header. That is the safety net. Many CDNs either ignore `Vary` or split their cache on every distinct Accept header, so prefer URLs that need no Accept header: `/{pid}?format=bibtex`, `/dois/{doi}` and `/works/{doi}` are cached by URL alone and carry no `Vary: Accept`. A format chosen by the Accept header names its own such URL in `Content-Location`, e.g. `Content-Location: /10.1038/171737a0?format=bibtex`.
### Request limits
dragoman limits what one client can ask of it, so it can face the internet without a proxy in front. A client is its IP address, an IPv6 client its /64. The peer address is used as is, unless the peer is a trusted proxy — this machine, or a network in `--trusted-proxies` — which names the client:
1. in `--client-ip-header`, if set (a CDN sets its own, overwriting the client's);
2. in `X-Real-IP`, from this machine only (a CDN passes on whatever the client sent);
3. in `X-Forwarded-For`, read from the right past the trusted proxies that appended themselves.
A trusted edge that names no client counts as one; a local request that names none is not limited.
Each client has a token bucket per tier. `RATE/BURST` means a client may send `BURST` requests at once and `RATE` per second after that; a request over the limit gets `429 Too Many Requests` with `Retry-After`.
| Tier | Requests | Default |
| --- | --- | --- |
| — | `/`, `/assets/*`, `/og-image.png`, `/commonmeta_v1.0.json`, `/api/status`, `/api/metrics` | not limited |
| Pages | entity pages and redirects (`/{pid}`, `/orcid/*`, `/ror/*`), and `?local_only=true` lookups — the records the UI shows on a page | `20/60` |
| Lists | `/api/person/*/works`, `/api/org/*/works`, `/api/container/works`, `/api/facets`, `/api/count`, `/api/random` | `5/20` |
| Expensive | `/api/search`, `/api/handles/*`, `/api/export`, `/api/bibliography`, `/dois/*`, `/works/*`, and a pid asked for in a format (`?format=` or an `Accept` header other than HTML) | `2/20` |
Across all clients, at most `--max-in-flight` requests are served at once and each response has `--request-timeout` seconds; past either, dragoman answers `503` with `Retry-After`.
With `--domain` set to a public hostname, the TLS listener also caps each client at `--max-conns-per-ip` open connections and gives it `--header-timeout` seconds to finish the TLS handshake, send its first request, and send each request's headers — so a client that connects and stays silent cannot hold a connection open.
Refusals are counted in the [metrics](#metrics--victoriametrics--grafana): `http_rate_limited_total{tier}`, `http_overloaded_total`, `http_timed_out_total` and `http_connections_refused_total`.
## Releasing prebuilt binaries
`cargo binstall dragoman` downloads target-named tarballs from [Codeberg Releases](https://codeberg.org/front-matter/dragoman/releases). The lookup URL is defined by `[package.metadata.binstall]` in `Cargo.toml`; each release tagged `v<version>` must carry these assets:
| Target | Asset |
| --- | --- |
| macOS arm64 (`aarch64-apple-darwin`) | `dragoman-aarch64-apple-darwin.tar.gz` |
| Linux amd64 (`x86_64-unknown-linux-gnu`) | `dragoman-x86_64-unknown-linux-gnu.tar.gz` |
Each tarball holds the bare `dragoman` binary at its root. Build and package both with:
```bash
# macOS arm64 builds natively; Linux amd64 cross-compiles via cargo-zigbuild
cargo install cargo-zigbuild && brew install zig # one-time, for the Linux target
scripts/package-release.sh
```
The tarballs and a `SHA256SUMS` file land in `dist/`. Create the `v<version>` release on Codeberg and upload both tarballs as assets.
## Telemetry
### Logs — VictoriaLogs
Set `--viktorialogs-url` (or `DRAGOMAN_VIKTORIALOGS_URL`) to the VictoriaLogs [OTLP/HTTP endpoint](https://docs.victoriametrics.com/victorialogs/data-ingestion/opentelemetry/). All `tracing` events are shipped in batches; the exporter is flushed on graceful shutdown.
```bash
dragoman start \
--viktorialogs-url http://localhost:9428/insert/opentelemetry/v1/logs
```
Without the flag, logs are written to stdout only (the default).
### Metrics — VictoriaMetrics / Grafana
Set `--metrics-allow` (or `DRAGOMAN_METRICS_ALLOW`) to a comma-separated list of IP addresses permitted to scrape the `/api/metrics` endpoint. The route is only registered when this flag is set; all other callers receive `403 Forbidden`.
```bash
dragoman start --metrics-allow 127.0.0.1,10.0.0.5
# curl http://localhost:3456/api/metrics # works from 127.0.0.1
```
The `/api/metrics` endpoint is served on the same port as the main application. The client IP is the peer address; only when the peer is on this machine — a reverse proxy such as nginx, Caddy or Traefik — is it taken from `X-Real-IP`, then the last address in `X-Forwarded-For`, the one the proxy appended.
Metrics recorded:
| Metric | Labels | Description |
| --- | --- | --- |
| `http_requests_total` | `method`, `route`, `status` | Request count by route and HTTP status code |
| `http_request_duration_seconds` | `method`, `route` | Request latency histogram |
Routes are normalised to templates (`/dois/:doi`, `/works/:doi`, `/orcid/:id`, `/{pid}`) to keep cardinality low.
## Benchmarking
[`scripts/bench.sh`](scripts/bench.sh) samples DOIs from the store's `pid_records` and runs [oha](https://github.com/hatoo/oha) against a running dragoman at concurrency 1 to 64, so a release can be compared with the one before. Run it on the server, where requests from the machine itself are not rate limited:
```bash
scripts/bench.sh -l 0.5.3 # before upgrading
scripts/bench.sh -l 0.5.4 # after
scripts/bench.sh -k page -c "4 16" # entity pages, two levels
column -t bench/results.tsv
```
Each level's full oha report is kept as JSON under `bench/`, and `bench/results.tsv` gains one line per level: requests per second, p50 and p99 latency, success rate and status codes. `-k` picks the URLs: `redirect` (`/{doi}`, the default), `handles`, `page` or `bibtex`. `-h` lists the other options.
## Keyboard shortcuts
| Shortcut | Action |
| --- | --- |
| `⌘K` / `Ctrl-K` | Focus search bar; press again to cycle the entity type |
| `⌘Y` / `Ctrl-Y` | Toggle History & Bookmarks panel (History tab) |
| `⌘B` / `Ctrl-B` | Toggle History & Bookmarks panel (Bookmarks tab) |
| `⌥↑` / `⌥↓`, `Alt-↑` / `Alt-↓` | Move the selected bookmark up or down |
| `Escape` | Close panel or cancel search |
| `↑` / `↓` | Navigate search suggestions |
| `Enter` | Navigate to selected search result |
## License
MIT