# defter
A self-hosted Zotero sync and file storage server: keep one Zotero library in
sync across your machines — metadata **and** attachment files such as PDFs —
without [Zotero Storage](https://www.zotero.org/storage). Metadata lives in SQLite, files in S3-compatible object
storage (Hetzner Object Storage, AWS S3, Cloudflare R2, RustFS, …). Stock Zotero clients sync against
it, and [commonmeta](https://codeberg.org/front-matter/commonmeta-rs)'s `sync`
command can enrich the library through the same API.
**Status: early development (0.3).** The sync protocol, file storage and query
API are implemented and tested; interfaces may still change.
defter builds on [zhost](https://github.com/mulatta/zhost) (MIT, see
`NOTICE`), which worked out the subset of the Zotero Web API v3 a client
needs. defter swaps PostgreSQL for SQLite (rusqlite), `rust-s3` for the
`object_store` crate, and adds a web UI and the endpoints `commonmeta sync`
uses.
## How it works
- **Client redirect, no patching.** Zotero reads its sync server from the
hidden pref `extensions.zotero.api.url`; pointing it at defter redirects
sync and login.
- **Server.** A Rust/[axum](https://github.com/tokio-rs/axum) service. Library
objects are stored as opaque JSON keyed by `(kind, key)`, with the columns
queries need derived on every write. One library version counter, bumped by
each write, keeps the client's `since` reads and `If-Unmodified-Since-Version`
writes coherent.
- **Files.** Uploads stream through defter, which checks the md5 while writing
them to the bucket; downloads redirect to a short-lived pre-signed URL, so
the bytes come straight from the bucket.
- **Auth.** One user, one library. API keys are configured, each read/write or
read-only. The Zotero client gets a read/write key through its browser login,
which you approve on defter's login page by entering that key.
- **Web UI.** A dashboard (library statistics, storage, connection
instructions) and the login consent page, rendered on the server from
[askama](https://askama.rs) templates and styled with Tailwind CSS.
[htmx](https://htmx.org) refreshes the dashboard in place; every page works
without JavaScript and is served under a strict Content-Security-Policy.
The full contract is in [`docs/api.md`](docs/api.md).
## Running
```sh
cargo install --path . # or: cargo install defter
defter start --public-url https://zotero.example.org \
--s3-bucket <your-bucket> \
--s3-endpoint https://fsn1.your-objectstorage.com --s3-region fsn1 \
--s3-access-key-id-file /etc/defter/s3-access-key-id \
--s3-secret-access-key-file /etc/defter/s3-secret-access-key
```
Every flag has a `DEFTER_*` environment variable; see `env.example` and
`defter start --help`. S3 credentials also fall back to the standard `AWS_*`
variables. Without `--s3-bucket`, files go to `<data-dir>/files` — fine for
development, not meant for production.
### API keys
The simplest setup is one read/write key in the environment, e.g. in
`/etc/defter/env`:
```sh
DEFTER_API_KEY=<output of: openssl rand -hex 32>
```
When Zotero logs in, it opens defter's login page in the browser; you approve
the login by entering that key, and Zotero receives it. Anyone can start a
login, so the key is what protects the library: generate it, don't choose it
(defter warns about keys shorter than 32 characters), and keep the file
readable by root and the `defter` group only.
To give scripts or agents read-only access as well, provision keys as files
instead, each read/write or read-only:
```sh
openssl rand -hex 32 > app-key # read/write, for Zotero
openssl rand -hex 32 > cli-key # read-only
defter start --keys rw:/path/app-key,ro:/path/cli-key …
```
With no key configured at all, defter generates a read/write key in
`<data-dir>/api-key`; `defter key` prints it.
### Deployment
defter listens on `127.0.0.1:8189`; put it behind a reverse proxy with HTTPS
and set `--public-url` to the proxy's address. The data API is protected by
keys and safe to expose, and so is `/login`, which hands out a key only to
whoever enters one. The dashboard (`/`, `/ui/status`) shows library
statistics and the storage location without a key; put it behind SSO or a
private network if that matters. Behind an SSO proxy, add
`--login-authorized-user you@example.org` to also require its identity on
login approval.
Clients download from pre-signed URLs on `--s3-endpoint`, so the bucket
endpoint must be reachable from them; otherwise add `--proxy-downloads`.
### Debian with systemd
As root, with Rust installed through [rustup](https://rustup.rs):
```sh
# the binary, straight to /usr/local/bin (the service cannot run one from ~)
cargo install --locked --root /usr/local defter
useradd --system --no-create-home --shell /usr/sbin/nologin defter
install -d -m 0750 -o root -g defter /etc/defter
# env.example and defter.service from this repository:
install -m 0640 -o root -g defter env.example /etc/defter/env # then edit it
# the S3 keys, one line each, root:defter 0640:
# /etc/defter/s3-access-key-id, /etc/defter/s3-secret-access-key
install -m 0644 defter.service /etc/systemd/system/defter.service
systemctl daemon-reload && systemctl enable --now defter
journalctl -u defter -f
```
To upgrade: `cargo install --locked --root /usr/local defter` again, then
`systemctl restart defter`.
[`defter.service`](defter.service) runs defter as the `defter` user with its
data in `/var/lib/defter` (systemd's `StateDirectory`, which defter picks up
without `--data-dir`) and the rest of the file system read-only.
### Sharing port 443 with dragoman
On a machine where [dragoman](https://codeberg.org/front-matter/dragoman)
already serves HTTPS on port 443 behind a CDN, defter needs no port, proxy or
certificate of its own: dragoman's `--sni-route` passes connections for
defter's hostname, still encrypted, to defter, which serves HTTPS on a
loopback port with the same CDN origin certificate (if it covers defter's
hostname, e.g. `*.example.org`).
```sh
# dragoman (/etc/dragoman/env), then restart it:
DRAGOMAN_SNI_ROUTES=defter.example.org=127.0.0.1:8443
# defter (/etc/defter/env):
DEFTER_PORT=8443
DEFTER_PUBLIC_URL=https://defter.example.org
DEFTER_TLS_CERT=/etc/defter/origin.pem
DEFTER_TLS_KEY=/etc/defter/origin-key.pem
```
Give defter its own copy of the certificate and key, readable by its group:
```sh
install -m 0644 -o root -g defter /etc/dragoman/origin.pem /etc/defter/origin.pem
install -m 0640 -o root -g defter /etc/dragoman/origin-key.pem /etc/defter/origin-key.pem
```
With the hostname proxied by the CDN, its limits apply to uploads through
defter (Cloudflare: 100 MB per request on the Free and Pro plans), so set
`DEFTER_MAX_UPLOAD_MB=100` there; downloads come from the bucket directly.
### Hetzner Object Storage
[Hetzner Object Storage](https://www.hetzner.com/storage/object-storage/)
is the intended storage, for a defter running at home as well as for one
later deployed on a Hetzner Cloud server.
1. In Hetzner Console, create a project, a **private** Bucket in one location
(`fsn1` Falkenstein, `nbg1` Nuremberg or `hel1` Helsinki), and S3
credentials for the project (shown once: save both keys). Keys are valid
for every Bucket of the project, so give defter a project of its own.
2. Start defter with the location as both endpoint and region:
```sh
defter start \
--s3-bucket <your-bucket> \
--s3-endpoint https://fsn1.your-objectstorage.com --s3-region fsn1 \
--s3-access-key-id-file ~/.config/defter/s3-access-key-id \
--s3-secret-access-key-file ~/.config/defter/s3-secret-access-key
```
defter addresses Hetzner Buckets by host name
(`https://<bucket>.fsn1.your-objectstorage.com`), which its pre-signed
download URLs need.
3. Check the round trip: `DEFTER_KEY=$(defter key) scripts/smoke-s3.sh`.
Costs (Hetzner's prices, November 2026, excl. VAT): €6.49 a month includes
1 TB of storage and 1 TB of outgoing traffic; uploads and S3 requests are
free. Downloads by pre-signed URL count as outgoing traffic. Objects under
64 KB are billed as 64 KB.
For a later deployment on a Hetzner Cloud server, put the server in an EU
location: it is then in the Bucket's network zone (`eu-central`), and traffic
between them is free. With `--proxy-downloads`, files then reach clients
through the server, whose traffic allowance is separate from the Bucket's.
### Connecting Zotero
In Zotero, Settings → Advanced → Config Editor:
```
extensions.zotero.api.url = https://zotero.example.org/api/
extensions.zotero.streaming.enabled = false
```
Then log in under Settings → Sync and approve the request in the browser
window that opens.
### commonmeta sync
`commonmeta sync` enhances the items of a Zotero library with what the
commonmeta store knows about them. defter serves the API it uses under
`/api`; [`docs/commonmeta-rs-changes.md`](docs/commonmeta-rs-changes.md)
lists the commonmeta-rs changes needed for it to target a remote server (it
currently only talks to `http://localhost:23119/api`).
## Development
```sh
cargo test # unit and API tests
cargo run -- start # http://127.0.0.1:8189, files on local disk
```
Building needs only Rust. Pages are askama templates in `templates/`,
compiled into the binary, so a template change needs a rebuild (`cargo run`
does it). Their stylesheet, `ui/dist/app.css`, is built from `ui/app.css` by
the standalone [Tailwind CLI](https://tailwindcss.com/docs/installation/tailwind-cli)
and committed. To change styling, install it (`brew install tailwindcss`, or
a release binary with `TAILWINDCSS=/path/to/tailwindcss`); the build then
regenerates the stylesheet, and warns when it can't. The script is the
vendored htmx (`ui/vendor/`, see its README for updating) plus `ui/main.js`.
To run against real S3 storage locally, use [RustFS](https://rustfs.com):
```sh
docker run -d --rm --name rustfs -p 9000:9000 \
-e RUSTFS_ACCESS_KEY=defter -e RUSTFS_SECRET_KEY=defter-secret rustfs/rustfs
curl --aws-sigv4 "aws:amz:us-east-1:s3" --user defter:defter-secret \
-X PUT http://127.0.0.1:9000/zotero # create the bucket
DEFTER_S3_BUCKET=zotero DEFTER_S3_ENDPOINT=http://127.0.0.1:9000 \
DEFTER_S3_REGION=us-east-1 DEFTER_S3_ACCESS_KEY_ID=defter \
DEFTER_S3_SECRET_ACCESS_KEY=defter-secret cargo run -- start
DEFTER_KEY=$(cargo run -q -- key) scripts/smoke-s3.sh # file round trip
```
### Layout
| `src/main.rs` | CLI and startup |
| `src/app.rs` | state, router, gzip and logging middleware |
| `src/zotero.rs` | Zotero API handlers |
| `src/auth.rs` | keys, login sessions, expiring tokens |
| `src/store.rs`, `migrations/` | library storage in SQLite |
| `src/query.rs` | item listing parameters |
| `src/db.rs` | connections and migrations |
| `src/storage.rs` | attachment files via `object_store` |
| `src/ui.rs`, `templates/` | web UI pages (askama) |
| `ui/` | the pages' stylesheet (Tailwind) and script (vendored htmx) |
## License
MIT. Includes code adapted from zhost (MIT), see `NOTICE`.