defter 0.3.1

Self-hosted Zotero sync and file storage server, with metadata in SQLite and attachments in S3.
defter-0.3.1 is not a library.

defter

A self-hosted Zotero sync and file storage server: keep one Zotero library in sync across your machines — metadata and attachment files such as PDFs — without Zotero Storage. Metadata lives in SQLite, files in S3-compatible object storage (Hetzner Object Storage, AWS S3, Cloudflare R2, RustFS, …). Stock Zotero 10 clients sync against it, and commonmeta's sync command can enrich the library through the same API.

Status: early development (0.3). The sync protocol, file storage and query API are implemented and tested; interfaces may still change.

defter builds on zhost (MIT, see NOTICE), which worked out the subset of the Zotero Web API v3 a client needs. defter swaps PostgreSQL for SQLite (rusqlite), rust-s3 for the object_store crate, and adds a web UI and the endpoints commonmeta sync uses.

How it works

  • Client redirect, no patching. Zotero reads its sync server from the hidden pref extensions.zotero.api.url; pointing it at defter redirects sync and login.
  • Server. A Rust/axum service. Library objects are stored as opaque JSON keyed by (kind, key), with the columns queries need derived on every write. One library version counter, bumped by each write, keeps the client's since reads and If-Unmodified-Since-Version writes coherent.
  • Files. Uploads stream through defter, which checks the md5 while writing them to the bucket; downloads redirect to a short-lived pre-signed URL, so the bytes come straight from the bucket.
  • Auth. One user, one library. API keys are configured, each read/write or read-only. The Zotero client gets a read/write key through its browser login, which you approve on defter's login page by entering that key.
  • Web UI. A dashboard (library statistics, storage, connection instructions), shown after signing in with an API key, and the login consent page for Zotero, rendered on the server from askama templates and styled with Tailwind CSS. htmx refreshes the dashboard in place; every page works without JavaScript and is served under a strict Content-Security-Policy.

The full contract is in docs/api.md.

Running

cargo install --path .          # or: cargo install defter
defter start --public-url https://zotero.example.org \
  --s3-bucket <your-bucket> \
  --s3-endpoint https://fsn1.your-objectstorage.com --s3-region fsn1 \
  --s3-access-key-id-file /etc/defter/s3-access-key-id \
  --s3-secret-access-key-file /etc/defter/s3-secret-access-key

Every flag has a DEFTER_* environment variable; see env.example and defter start --help. S3 credentials also fall back to the standard AWS_* variables. Without --s3-bucket, files go to <data-dir>/files — fine for development, not meant for production.

API keys

The simplest setup is one read/write key in the environment, e.g. in /etc/defter/env:

DEFTER_API_KEY=<output of: openssl rand -hex 32>

When Zotero logs in, it opens defter's login page in the browser; you approve the login by entering that key, and Zotero receives it. Anyone can start a login, so the key is what protects the library: generate it, don't choose it (defter warns about keys shorter than 32 characters), and keep the file readable by root and the defter group only.

To give scripts or agents read-only access as well, provision keys as files instead, each read/write or read-only:

openssl rand -hex 32 > app-key   # read/write, for Zotero
openssl rand -hex 32 > cli-key   # read-only
defter start --keys rw:/path/app-key,ro:/path/cli-key …

With no key configured at all, defter generates a read/write key in <data-dir>/api-key; defter key prints it.

Deployment

defter listens on 127.0.0.1:8189; put it behind a reverse proxy with HTTPS and set --public-url to the proxy's address. The data API is protected by keys and safe to expose, and so is /login, which hands out a key only to whoever enters one. The dashboard (/) shows a sign-in form until an API key is entered (a read-only key is enough), and /ui/status takes the key as a header. Behind an SSO proxy, add --login-authorized-user you@example.org to also require its identity on login approval.

Clients download from pre-signed URLs on --s3-endpoint, so the bucket endpoint must be reachable from them; otherwise add --proxy-downloads.

Debian with systemd

As root, with Rust installed through rustup:

# the binary, straight to /usr/local/bin (the service cannot run one from ~)
cargo install --locked --root /usr/local defter

useradd --system --no-create-home --shell /usr/sbin/nologin defter
install -d -m 0750 -o root -g defter /etc/defter
# env.example and defter.service from this repository:
install -m 0640 -o root -g defter env.example /etc/defter/env   # then edit it
# the S3 keys, one line each, root:defter 0640:
#   /etc/defter/s3-access-key-id, /etc/defter/s3-secret-access-key
install -m 0644 defter.service /etc/systemd/system/defter.service
systemctl daemon-reload && systemctl enable --now defter
journalctl -u defter -f

To upgrade: cargo install --locked --root /usr/local defter again, then systemctl restart defter.

defter.service runs defter as the defter user with its data in /var/lib/defter (systemd's StateDirectory, which defter picks up without --data-dir) and the rest of the file system read-only.

Sharing port 443 with dragoman

On a machine where dragoman already serves HTTPS on port 443 behind a CDN, defter needs no port, proxy or certificate of its own: dragoman's --sni-route passes connections for defter's hostname, still encrypted, to defter, which serves HTTPS on a loopback port with the same CDN origin certificate (if it covers defter's hostname, e.g. *.example.org).

# dragoman (/etc/dragoman/env), then restart it:
DRAGOMAN_SNI_ROUTES=defter.example.org=127.0.0.1:8443

# defter (/etc/defter/env):
DEFTER_PORT=8443
DEFTER_PUBLIC_URL=https://defter.example.org
DEFTER_TLS_CERT=/etc/defter/origin.pem
DEFTER_TLS_KEY=/etc/defter/origin-key.pem

Give defter its own copy of the certificate and key, readable by its group:

install -m 0644 -o root -g defter /etc/dragoman/origin.pem /etc/defter/origin.pem
install -m 0640 -o root -g defter /etc/dragoman/origin-key.pem /etc/defter/origin-key.pem

With the hostname proxied by the CDN, its limits apply to uploads through defter (Cloudflare: 100 MB per request on the Free and Pro plans), so set DEFTER_MAX_UPLOAD_MB=100 there; downloads come from the bucket directly.

Hetzner Object Storage

Hetzner Object Storage is the intended storage, for a defter running at home as well as for one later deployed on a Hetzner Cloud server.

  1. In Hetzner Console, create a project, a private Bucket in one location (fsn1 Falkenstein, nbg1 Nuremberg or hel1 Helsinki), and S3 credentials for the project (shown once: save both keys). Keys are valid for every Bucket of the project, so give defter a project of its own.

  2. Start defter with the location as both endpoint and region:

    defter start \
  --s3-bucket <your-bucket> \
  --s3-endpoint https://fsn1.your-objectstorage.com --s3-region fsn1 \
  --s3-access-key-id-file ~/.config/defter/s3-access-key-id \
  --s3-secret-access-key-file ~/.config/defter/s3-secret-access-key

    defter addresses Hetzner Buckets by host name (https://<bucket>.fsn1.your-objectstorage.com), which its pre-signed download URLs need.

  3. Check the round trip: DEFTER_KEY=$(defter key) scripts/smoke-s3.sh.

Costs (Hetzner's prices, November 2026, excl. VAT): €6.49 a month includes 1 TB of storage and 1 TB of outgoing traffic; uploads and S3 requests are free. Downloads by pre-signed URL count as outgoing traffic. Objects under 64 KB are billed as 64 KB.

For a later deployment on a Hetzner Cloud server, put the server in an EU location: it is then in the Bucket's network zone (eu-central), and traffic between them is free. With --proxy-downloads, files then reach clients through the server, whose traffic allowance is separate from the Bucket's.

Connecting Zotero

defter requires Zotero 10 (desktop). Earlier versions are not supported.

defter syncs My Library only. Group libraries stay on zotero.org: a Zotero that syncs with defter no longer syncs its groups, and may offer to remove their local copies. To keep both, use a separate Zotero profile for defter (zotero -P → Create Profile, with its own data directory under Settings → Advanced → Files and Folders).

  1. Back up the data directory first (Settings → Advanced → Files and Folders → Show Data Directory; copy it while Zotero is closed).

  2. If the profile is linked to a zotero.org account, unlink it under Settings → Sync before changing the server, so the key is removed there.

  3. Settings → Advanced → Config Editor (accept the warning), then set:

    extensions.zotero.api.url = https://defter.example.org/api/
extensions.zotero.streaming.enabled = false

    Keep the trailing slash: Zotero appends paths such as keys/sessions directly. defter has no streaming server, hence the second setting.

  4. Restart Zotero.

  5. Settings → Sync → Log In. Zotero opens defter's login page in the browser; enter an API key that may write and click Approve. Zotero picks up the key within seconds; the login request expires after 10 minutes.

  6. If the library was synced with another account before, Zotero asks before combining it with the defter library; continue to upload it.

  7. Keep file syncing set to Zotero (Settings → Sync → File Syncing): attachment files then go to defter, and through it to the bucket.

To go back to zotero.org, reset both settings in the Config Editor (right-click → Reset), restart Zotero and log in again.

commonmeta sync

commonmeta sync enhances the items of a Zotero library with what the commonmeta store knows about them. defter serves the API it uses under /api; docs/commonmeta-rs-changes.md lists the commonmeta-rs changes needed for it to target a remote server (it currently only talks to http://localhost:23119/api).

Development

cargo test                 # unit and API tests
cargo run -- start         # http://127.0.0.1:8189, files on local disk

Building needs only Rust. Pages are askama templates in templates/, compiled into the binary, so a template change needs a rebuild (cargo run does it). Their stylesheet, ui/dist/app.css, is built from ui/app.css by the standalone Tailwind CLI and committed. To change styling, install it (brew install tailwindcss, or a release binary with TAILWINDCSS=/path/to/tailwindcss); the build then regenerates the stylesheet, and warns when it can't. The script is the vendored htmx (ui/vendor/, see its README for updating) plus ui/main.js.

To run against real S3 storage locally, use RustFS:

docker run -d --rm --name rustfs -p 9000:9000 \
  -e RUSTFS_ACCESS_KEY=defter -e RUSTFS_SECRET_KEY=defter-secret rustfs/rustfs
curl --aws-sigv4 "aws:amz:us-east-1:s3" --user defter:defter-secret \
  -X PUT http://127.0.0.1:9000/zotero          # create the bucket
DEFTER_S3_BUCKET=zotero DEFTER_S3_ENDPOINT=http://127.0.0.1:9000 \
DEFTER_S3_REGION=us-east-1 DEFTER_S3_ACCESS_KEY_ID=defter \
DEFTER_S3_SECRET_ACCESS_KEY=defter-secret cargo run -- start
DEFTER_KEY=$(cargo run -q -- key) scripts/smoke-s3.sh   # file round trip

Layout

Path
src/main.rs CLI and startup
src/app.rs state, router, gzip and logging middleware
src/zotero.rs Zotero API handlers
src/auth.rs keys, login sessions, expiring tokens
src/store.rs, migrations/ library storage in SQLite
src/query.rs item listing parameters
src/db.rs connections and migrations
src/storage.rs attachment files via object_store
src/ui.rs, templates/ web UI pages (askama)
ui/ the pages' stylesheet (Tailwind) and script (vendored htmx)

License

MIT. Includes code adapted from zhost (MIT), see NOTICE.