# defter API
defter serves a subset of the [Zotero Web API v3](https://www.zotero.org/support/dev/web_api/v3/start)
— what a stock Zotero client needs to sync, plus the read/query API a CLI
uses — and the parts of Zotero's local API that `commonmeta sync` relies on.
Every Zotero endpoint is served twice: at the root (the Zotero client's
`extensions.zotero.api.url`) and under `/api` (where commonmeta expects
Zotero's local API, `http://localhost:23119/api`). `<lib>` is `users/<id>`, the
key's own user's library: any other answers `403`. `users/0` is accepted as an
alias for it, as in the local API. `<lib>` is also `groups/<id>`, the library of
a group the key's user is a member of: anyone else gets `403`. In a group, a
member may write when the group lets members edit (`libraryEditing`), admins and
the owner always, and may upload files as `fileEditing` says; otherwise `403`.
## Supported client
defter requires the **Zotero 10** desktop client; earlier versions are not
supported. Zotero 10 has no setting for a sync server in its interface: it
reads `extensions.zotero.api.url` (Config Editor) and falls back to
`https://api.zotero.org/`. It appends paths such as `keys/sessions` to that
URL directly, so the URL must end in `/`. With
`extensions.zotero.streaming.enabled` off, Zotero polls instead of using a
streaming server, which defter does not have.
Zotero 10's login (checked against the source of 10.0.7:
`syncAPIClient.js`, `syncRunner.js`, `preferences_account.js`):
- `POST keys/sessions`, with a JSON body `{userID}` when the profile was
linked to an account before (defter ignores it); Zotero opens `loginURL`
in the browser.
- It polls `GET keys/sessions/<token>` for up to 10 minutes, treating `404`
as "session not found" and `410` as "expired".
- A completed session must carry `apiKey`, `userID` and `username`; if
storing the key fails, Zotero deletes it with `DELETE keys/current`.
The sync contract follows zhost's `server/SPEC.md`, which records what live
testing against Zotero 9 showed; the differences are noted below.
## Authentication
Data endpoints need a key in `Zotero-API-Key` (or `Authorization: Bearer`);
an unknown key is `403`. Read-only keys get `403` on `POST`/`PUT`/`PATCH`/`DELETE`.
Keys configured on the server are the owner's. Every other key is minted when a
user signed in through the OpenID Connect provider approves a Zotero login, and
only its sha256 is stored (see the README).
| Endpoint | Auth | Notes |
|---|---|---|
| `GET /api/` | — | `Zotero-Server-ID` (stable per installation) and `X-Defter-Version` headers; `/` carries them too |
| `GET /keys/current` | key | `{key, userID, username, displayName, access}` of the key's user; `access.user.write` reflects the key, and `access.groups.all` covers every group they are in, as their role there allows |
| `DELETE /keys/current` | key | `204`; Zotero sends it when the account is unlinked. A minted key is revoked; one configured on the server stays |
| `POST /keys/sessions` | — | `201 {sessionToken, loginURL}` |
| `GET /keys/sessions/<token>` | — | `{status: "pending"}` until approved, then once `{status: "completed", apiKey, userID, username}`: the configured key the session was approved with, or the key minted for the user who approved it |
| `DELETE /keys/sessions/<token>` | — | cancel |
| `GET /login?session=<token>` | — | server-rendered consent page with a form for an API key, no JavaScript needed; never authorizes by itself. `400` without a session, `404` when unknown or expired. Not cacheable or frameable, strict CSP; its URL (with the token) goes as a referrer to defter only (`Referrer-Policy: same-origin`) |
| `POST /login` (form `session`, `key`) | — | approve with an API key that may write; a wrong or read-only key gets the form again with `403`, after a second's delay. Also rejects a foreign `Origin`, and checks the SSO identity with `--login-authorized-user` (`403` page). `303` to `/login?approved=1` |
Anyone can create a session; only a key that may write approves one, so
`/login` is safe to expose. The web UI needs a key too (see below).
## Versions and preconditions
A single library version is bumped by every write and returned in
`Last-Modified-Version`; each object carries the version it last changed at.
- `?since=<v>` reads return objects with a greater version, always `200`
(never `304`: the client's version reads loop on it). Only `GET /settings`
answers `304` to `If-Modified-Since-Version`.
- Multi-object writes: `If-Unmodified-Since-Version` is the **library**
version → `412` when stale. It is optional on `POST`/`PATCH` (as in the Web
API; commonmeta sends none when creating); without it, an object whose
`version` property differs from the stored one fails on its own with `412`
in `failed`. Deletes require it (`428` without).
- Single-object writes (`PUT`/`PATCH`/`DELETE …/<key>`): the precondition is the
**object's** version, from the header or the body's `version` (`428` when
neither is given). `PUT` with version `0` creates.
Write bodies may be gzip-compressed (`Content-Encoding: gzip`).
## Library data
| Endpoint | Notes |
|---|---|
| `GET <lib>/{items,collections,searches}?format=versions&since=<v>` | `{key: version}` |
| `GET <lib>/{items,collections,searches}?{item,collection,search}Key=a,b` | `[{key, version, meta?, data}]` |
| `POST`/`PATCH <lib>/{items,collections,searches}` | ≤ 50 objects; merge semantics; `{successful, success, unchanged, failed}` keyed by batch index; `successful` holds full objects |
| `DELETE <lib>/{items,collections,searches}?<kind>Key=a,b` | `204` |
| `GET`/`PUT`/`PATCH`/`DELETE <lib>/{items,collections,searches}/<key>` | single object; `GET` carries the object's version in `Last-Modified-Version`; writes `204` |
| `GET <lib>/items/<key>/children` | attachments and notes (a listing, see below) |
| `GET <lib>/items/top`, `/items/trash` | listings; `items/top` also answers the `format=versions` and `itemKey` reads |
| `GET <lib>/collections/<key>/items[/top]` | listings |
| `GET <lib>/tags` | `[{tag, meta: {type, numItems}}]`, trash excluded |
| `DELETE <lib>/tags?tag=a \|\| b` | strips the tags from every item and logs them |
| `GET`/`POST`/`DELETE <lib>/settings` | `{key: {value, version}}`; delete by `settingKey=a,b`. A `POST` of an unchanged value makes no new version, as on zotero.org |
| `GET <lib>/deleted?since=<v>` | `{collections, searches, items, settings, tags}` |
| `GET <lib>/fulltext?since=<v>` | `{itemKey: version}` |
| `GET <lib>/items/<key>/fulltext` | `{content, indexedChars, …}` with the row's own version |
| `POST <lib>/fulltext` | batch of `{key, content, …}` |
| `GET users/<id>/groups` | the groups the user is a member of: with `format=versions` `{groupID: version}`, which a client compares with its own (a group missing from it is one the user was taken out of, or deleted), otherwise their JSON as below. The `Last-Modified-Version` is the user's library's |
| `GET groups/<id>` | the group's JSON, as zotero.org's: `{id, version, links, meta: {created, lastModified, numItems}, data: {id, version, name, owner, type: "Private", description, url, libraryEditing, libraryReading: "members", fileEditing, admins, members}}`, with the owner, admins and members as user IDs (the owner and admins are not repeated among the members), and the group's version in `Last-Modified-Version`: every change to the group or its members makes a new one |
| `GET /schema` | Zotero's item schema, fetched from `api.zotero.org/schema`, stored, revalidated daily |
Object keys are 8 characters from `23456789ABCDEFGHIJKLMNPQRSTUVWXYZ`;
others fail with `400`. A keyless object gets a fresh key. Item data is
stored verbatim; attachments and annotations come back with `linkMode` /
`annotationType` first. Top-level items carry `meta.numChildren`. In a group
library, objects also carry `meta.createdByUser` and `meta.lastModifiedByUser`,
each `{id, username, name}`: the member who added the object and the one who
last changed it, as zotero.org's do.
### Listings
Item listings accept:
| Parameter | Meaning |
|---|---|
| `q`, `qmode` | case-insensitive substring; `titleCreatorYear` (default) searches title, date and creators; `everything` searches every field and tag (DOI, `extra`, URL, …) and stored full text |
| `itemType` | `book \|\| journalArticle`, `-attachment` |
| `tag` | repeat for AND, `\|\|` for OR |
| `sort`, `direction` | `dateModified` (default), `dateAdded`, `title`, `creator`, `date`, `itemType`; `asc`/`desc` |
| `limit`, `start` | page of ≤ 100 (default 25); `Total-Results` and `Link: <…>; rel="next"` on the public URL |
| `includeTrashed` | include items in the trash |
| `format=keys` | every matching key, unpaged, as plain text |
## Files
Attachment bytes live in object storage, under `users/<id>/files/<item key>`,
`groups/<id>/files/<item key>` for a group's, which count against its owner's
quota, or
`files/<item key>` for the library of a server from before there were several;
the database keeps their md5, size and mtime.
1. **Authorize** — `POST <lib>/items/<key>/file`, form `md5, filename,
filesize, mtime` (ms), with `If-None-Match: *` (new) or `If-Match: <md5>`
(replace); `428` without either, `404` for an unknown item. Answers
`{exists: 1}` when the same file is stored, `412` when another is (or when
`If-Match` names a different md5), `413` past `--max-upload-mb`, `413` with
`Zotero-Storage-Usage` and `Zotero-Storage-Quota` (MB) and
`Zotero-Storage-UserID` when the file would take the user past their quota (a
file it replaces does not count), otherwise
`{url, uploadKey, contentType, prefix: "", suffix: ""}` with an absolute
`url`. As on zotero.org, `exists` and `412` carry the item's version in
`Last-Modified-Version`, except a `412` for `If-Match` on an item with no
stored file, which carries none: Zotero then retries with
`If-None-Match: *`. Files added through Zotero's local API (as
`commonmeta sync --pdf` does) arrive that way.
Snapshots (`text/*` attachments) and attachments with several files come
as a ZIP of the attachment directory, with `zipMD5` and `zipFilename` added:
`md5` is then the main file's, `filesize` the ZIP's, and the upload is
checked against `zipMD5`. defter stores the ZIP as it is.
2. **Upload** — `POST <url>` with the raw bytes (no key: the token in the URL
is the capability). Streamed to `pending/<token>` while hashing; a md5 or
size mismatch is `400` and the bytes are dropped. `201`.
3. **Register** — `POST <lib>/items/<key>/file`, form `upload=<uploadKey>`:
moves the file to `files/<key>`, records it and stamps `md5`/`mtime` into the
item. `204`.
**Download** — `GET <lib>/items/<key>/file` answers `302` with
`Zotero-File-MD5`, `Zotero-File-Modification-Time`, `Zotero-File-Compressed`
(`Yes` for a stored ZIP, which Zotero unpacks) and `Location`: a pre-signed
bucket URL valid for `--presign-ttl` seconds, or, without signing (local
storage, `--proxy-downloads`), a `/downloads/<token>` link valid for five
minutes that streams the file through defter.
Deleting an attachment item deletes its file.
## Web UI
| Endpoint | Notes |
|---|---|
| `GET /` | signed in: the user's dashboard (their library, storage, profile and devices; for admins also where files are stored, backups and a link to `/ui/admin`), and with `HX-Request: true` (htmx's 30-second poll) only its status block (`Vary: HX-Request`). Otherwise `401` with a sign-in form, or for htmx's poll an empty `401` with `HX-Refresh: true`, which reloads the page. Not cacheable |
| `POST /ui/session` (form `key`) | sign in as the owner with an API key configured on the server, read-only included: sets the `defter_session` cookie (HttpOnly, SameSite=Lax, Secure with an HTTPS public URL, 7 days) and `303` to `/`. A wrong key gets the form again with `403` after a second's delay; a foreign `Origin` is rejected |
| `GET /ui/oidc/login?session=` | with `--oidc-issuer`: sends the browser to the provider, with a state tied to it by the `defter_oidc` cookie. `session` is a Zotero login to return to. `404` without `--oidc-issuer`, `502` if the provider cannot be read |
| `GET /ui/oidc/callback?state=&code=` | the provider's redirect back: for a user's ORCID iD, an invited one, which becomes a user, or an admin's (`--admin-orcids`), sets the `defter_session` cookie and `303` to `/` or the Zotero login. `403` with the sign-in form for anyone else or a disabled user |
| `POST /ui/logout` | ends the session, clears the cookie, `303` to `/`. A session from an OpenID Connect provider gets a page that goes on to the provider's logout instead, which returns to `/` |
| `GET /<orcid>?q=&year=&type=&featured=&content=&lang=&page=` | the public profile of the user with that ORCID iD, without a key. `q` searches title, creators and date; `year`, `language`, `type` and `license` can repeat: values of one are ORed, different ones ANDed. `language` is an ISO 639-1 code and `license` an SPDX ID, read from Zotero's Language and Rights fields (`en-US`, `English`, `Deutsch` and `ger` are all `en` or `de`; `CC BY 4.0` and its URL are `CC-BY-4.0`); a work whose field names neither is left out of those facets. With none of these, or with `featured=1`, it shows up to five works with the manual tag `featured` (`featured=0` for all works; a bare address shows all of them if none is featured). `content=pdf` to works with a PDF link. `lang` is `en` or `de`, otherwise `Accept-Language` decides. With `HX-Request: true` only the list of works (`Vary: HX-Request, Accept-Language`). `404` for any other path, while hidden, and for a disabled user |
| `GET /<orcid>/export?format=&style=&…` | the works the list shows, under the same filters and without paging, as a file, or with `key` the one public work with that key (`404` for any other): `bibtex`, `commonmeta` (default), `crossref`, `crossref_xml`, `csl`, `datacite`, `datacite_xml`, `citation`, `inveniordm`, `ris` or `schemaorg`. `style` is for `citation`: `apa` (default), `chicago-author-date`, `harvard-cite-them-right`, `ieee`, `modern-language-association` or `vancouver`. `400` for anything else |
| `POST /ui/profile` (form `hidden`, `true` or `false`) | hide or show the signed-in user's public profile: `303` to `/`. Needs a session signed in with a read/write key (`401`, `403`); a foreign `Origin` is rejected, as on every `POST` below |
| `POST /ui/groups` (form `name`) | create a group the signed-in user owns: `303` to its page |
| `GET /ui/groups/<id>` | a group's settings and members, for its members: `404` for others |
| `POST /ui/groups/<id>/settings` (form `name`, `description`, `library_editing`, `file_editing`) | for the owner and admins |
| `POST /ui/groups/<id>/members` (form `orcid`, or `user` for a member already, and `role`, `member` or `admin`) | add a user of the server by their ORCID iD, or change a member's role: the owner and admins add members, only the owner makes or unmakes admins. `?error=orcid` or `?error=user` for no iD or no user |
| `POST /ui/groups/<id>/remove` (form `user`, the signed-in user's own without) | take a member out, or leave: admins take out members, the owner anyone; the owner cannot leave |
| `POST /ui/groups/<id>/delete` | for the owner: members' clients are told the group is gone; its library is kept |
| `POST /ui/keys/revoke` (form `id`) | revoke one of the signed-in user's devices' keys: `303` to `/`, `404` for a key that is not theirs |
| `GET /ui/admin` | for admins: invitations, users with their storage, devices and quota, and backups. `404` for others, `303` to `/` when not signed in |
| `POST /ui/admin/invite` (form `orcid`, `note`) | invite an ORCID iD (or ORCID URL): `303` to `/ui/admin`, with `?error=orcid` for no iD and `?error=user` for a user's. Admins with a read/write session only |
| `POST /ui/admin/uninvite` (form `orcid`) | withdraw an invitation not used yet |
| `POST /ui/admin/user` (form `id`, and `quota_mb`, empty for the default, or `disabled`, `true` or `false`) | set a user's quota (`?error=quota` for no number), or disable or enable them. Disabling revokes their keys and ends their sessions; admins cannot disable themselves, nor the owner, whose keys are configured on the server (`400`) |
| `GET /assets/<name>` | the stylesheet and script (htmx), named by content hash and cached for a year; public |
| `GET /ui/status` | the dashboard's data as JSON, for monitoring, about the user of the API key (`Zotero-API-Key` or `Authorization: Bearer`) or the session cookie, `401` otherwise: their library, `quota` (bytes, `null` for none) and `storage.maxUpload`. For admins also `storage.location`, `storage.signedDownloads` and `backup`, which holds `enabled`, and the last run's `lastAt`, `manifest`, `error`, and `warning` when it succeeded with a problem, such as old backups it could not remove |
Sessions are kept in memory, so a restart signs everyone out.
Pages are in English or German: `?lang=en` or `?lang=de`, which the `defter_lang` cookie
then remembers for a year, otherwise the browser's `Accept-Language`, otherwise English.
With `--oidc-issuer` and without `--api-key` (`DEFTER_API_KEY`), a key can't be typed in:
the forms are not offered, and `POST /ui/session` and a `POST /login` from a session not
signed in through the provider answer `403`. Keys keep working for the data API and
`GET /ui/status`.
Pages are rendered on the server from the askama templates in `templates/`
and work without JavaScript. They are sent with a Content-Security-Policy
that allows scripts and styles only from defter (plus a per-response nonce
for the inline theme script) and forbids framing; htmx is configured not to
use `eval` or inline styles, so it runs under that policy.
## Out of scope
Group libraries, the streaming (websocket) server, binary-diff uploads
(`PATCH …/file`), and styled bibliographies.