defter

A self-hosted Zotero sync and file storage server: keep one Zotero library in sync across your machines — metadata and attachment files such as PDFs — without Zotero Storage. Metadata lives in SQLite, files in S3-compatible object storage (AWS S3, Cloudflare R2, RustFS, …). Stock Zotero clients sync against it, and commonmeta's sync command can enrich the library through the same API.

Status: early development (0.1). The sync protocol, file storage and query API are implemented and tested; interfaces may still change.

defter builds on zhost (MIT, see NOTICE), which worked out the subset of the Zotero Web API v3 a client needs. defter swaps PostgreSQL for SQLite (rusqlite), rust-s3 for the object_store crate, and adds a web UI and the endpoints commonmeta sync uses.

How it works

• Client redirect, no patching. Zotero reads its sync server from the hidden pref extensions.zotero.api.url; pointing it at defter redirects sync and login.
• Server. A Rust/axum service. Library objects are stored as opaque JSON keyed by (kind, key), with the columns queries need derived on every write. One library version counter, bumped by each write, keeps the client's since reads and If-Unmodified-Since-Version writes coherent.
• Files. Uploads stream through defter, which checks the md5 while writing them to the bucket; downloads redirect to a short-lived pre-signed URL, so the bytes come straight from the bucket.
• Auth. One user, one library. API keys are configured, each read/write or read-only. The Zotero client gets a read/write key through its browser login, which defter releases only after you approve it in the web UI.
• Web UI. A dashboard (library statistics, storage, connection instructions) and the login consent page, rendered on the server from askama templates and styled with Tailwind CSS. htmx refreshes the dashboard in place; every page works without JavaScript and is served under a strict Content-Security-Policy.

The full contract is in docs/api.md.

Running

cargo install --path .          # or: cargo install defter
defter start --public-url https://zotero.example.org \
  --s3-bucket zotero \
  --s3-endpoint https://<account>.r2.cloudflarestorage.com --s3-region auto \
  --s3-access-key-id-file /run/credentials/defter/s3-access-key-id \
  --s3-secret-access-key-file /run/credentials/defter/s3-secret-access-key

Every flag has a DEFTER_* environment variable; see env.example and defter start --help. S3 credentials also fall back to the standard AWS_* variables. Without --s3-bucket, files go to <data-dir>/files — fine for development, not meant for production.

API keys

With no key configured, defter generates a read/write key in <data-dir>/api-key; defter key prints it. For real deployments provision keys as files:

openssl rand -hex 32 > app-key   # read/write, handed to Zotero at login
openssl rand -hex 32 > cli-key   # read-only, for scripts and agents
defter start --keys rw:/path/app-key,ro:/path/cli-key …

Deployment

defter listens on 127.0.0.1:8189; put it behind a reverse proxy with HTTPS and set --public-url to the proxy's address. The data API is protected by keys and safe to expose. /login (which releases a key once approved), /ui/* and / should be behind SSO (e.g. oauth2-proxy; add --login-authorized-user you@example.org for defence in depth) or reachable only on a private network.

Clients download from pre-signed URLs on --s3-endpoint, so the bucket endpoint must be reachable from them; otherwise add --proxy-downloads.

Connecting Zotero

In Zotero, Settings → Advanced → Config Editor:

extensions.zotero.api.url = https://zotero.example.org/api/
extensions.zotero.streaming.enabled = false

Then log in under Settings → Sync and approve the request in the browser window that opens.

commonmeta sync

commonmeta sync enhances the items of a Zotero library with what the commonmeta store knows about them. defter serves the API it uses under /api; docs/commonmeta-rs-changes.md lists the commonmeta-rs changes needed for it to target a remote server (it currently only talks to http://localhost:23119/api).

Development

cargo test                 # unit and API tests
cargo run -- start         # http://127.0.0.1:8189, files on local disk

Building needs only Rust. Pages are askama templates in templates/, compiled into the binary, so a template change needs a rebuild (cargo run does it). Their stylesheet, ui/dist/app.css, is built from ui/app.css by the standalone Tailwind CLI and committed. To change styling, install it (brew install tailwindcss, or a release binary with TAILWINDCSS=/path/to/tailwindcss); the build then regenerates the stylesheet, and warns when it can't. The script is the vendored htmx (ui/vendor/, see its README for updating) plus ui/main.js.

To run against real S3 storage locally, use RustFS:

docker run -d --rm --name rustfs -p 9000:9000 \
  -e RUSTFS_ACCESS_KEY=defter -e RUSTFS_SECRET_KEY=defter-secret rustfs/rustfs
curl --aws-sigv4 "aws:amz:us-east-1:s3" --user defter:defter-secret \
  -X PUT http://127.0.0.1:9000/zotero          # create the bucket
DEFTER_S3_BUCKET=zotero DEFTER_S3_ENDPOINT=http://127.0.0.1:9000 \
DEFTER_S3_REGION=us-east-1 DEFTER_S3_ACCESS_KEY_ID=defter \
DEFTER_S3_SECRET_ACCESS_KEY=defter-secret cargo run -- start
DEFTER_KEY=$(cargo run -q -- key) scripts/smoke-s3.sh   # file round trip

Layout

License

MIT. Includes code adapted from zhost (MIT), see NOTICE.