# defter
A self-hosted Zotero sync and file storage server: keep one Zotero library in
sync across your machines — metadata **and** attachment files such as PDFs —
without [Zotero Storage](https://www.zotero.org/storage). Metadata lives in SQLite, files in S3-compatible object
storage (AWS S3, Cloudflare R2, RustFS, …). Stock Zotero clients sync against
it, and [commonmeta](https://codeberg.org/front-matter/commonmeta-rs)'s `sync`
command can enrich the library through the same API.
**Status: early development (0.1).** The sync protocol, file storage and query
API are implemented and tested; interfaces may still change.
defter builds on [zhost](https://github.com/mulatta/zhost) (MIT, see
`NOTICE`), which worked out the subset of the Zotero Web API v3 a client
needs. defter swaps PostgreSQL for SQLite (rusqlite), `rust-s3` for the
`object_store` crate, and adds a web UI and the endpoints `commonmeta sync`
uses.
## How it works
- **Client redirect, no patching.** Zotero reads its sync server from the
hidden pref `extensions.zotero.api.url`; pointing it at defter redirects
sync and login.
- **Server.** A Rust/[axum](https://github.com/tokio-rs/axum) service. Library
objects are stored as opaque JSON keyed by `(kind, key)`, with the columns
queries need derived on every write. One library version counter, bumped by
each write, keeps the client's `since` reads and `If-Unmodified-Since-Version`
writes coherent.
- **Files.** Uploads stream through defter, which checks the md5 while writing
them to the bucket; downloads redirect to a short-lived pre-signed URL, so
the bytes come straight from the bucket.
- **Auth.** One user, one library. API keys are configured, each read/write or
read-only. The Zotero client gets a read/write key through its browser login,
which defter releases only after you approve it in the web UI.
- **Web UI.** A dashboard (library statistics, storage, connection
instructions) and the login consent page, rendered on the server from
[askama](https://askama.rs) templates and styled with Tailwind CSS.
[htmx](https://htmx.org) refreshes the dashboard in place; every page works
without JavaScript and is served under a strict Content-Security-Policy.
The full contract is in [`docs/api.md`](docs/api.md).
## Running
```sh
cargo install --path . # or: cargo install defter
defter start --public-url https://zotero.example.org \
--s3-bucket zotero \
--s3-endpoint https://<account>.r2.cloudflarestorage.com --s3-region auto \
--s3-access-key-id-file /run/credentials/defter/s3-access-key-id \
--s3-secret-access-key-file /run/credentials/defter/s3-secret-access-key
```
Every flag has a `DEFTER_*` environment variable; see `env.example` and
`defter start --help`. S3 credentials also fall back to the standard `AWS_*`
variables. Without `--s3-bucket`, files go to `<data-dir>/files` — fine for
development, not meant for production.
### API keys
With no key configured, defter generates a read/write key in
`<data-dir>/api-key`; `defter key` prints it. For real deployments provision
keys as files:
```sh
openssl rand -hex 32 > app-key # read/write, handed to Zotero at login
openssl rand -hex 32 > cli-key # read-only, for scripts and agents
defter start --keys rw:/path/app-key,ro:/path/cli-key …
```
### Deployment
defter listens on `127.0.0.1:8189`; put it behind a reverse proxy with HTTPS
and set `--public-url` to the proxy's address. The data API is protected by
keys and safe to expose. `/login` (which releases a key once approved),
`/ui/*` and `/` should be behind SSO (e.g. oauth2-proxy; add
`--login-authorized-user you@example.org` for defence in depth) or reachable
only on a private network.
Clients download from pre-signed URLs on `--s3-endpoint`, so the bucket
endpoint must be reachable from them; otherwise add `--proxy-downloads`.
### Connecting Zotero
In Zotero, Settings → Advanced → Config Editor:
```
extensions.zotero.api.url = https://zotero.example.org/api/
extensions.zotero.streaming.enabled = false
```
Then log in under Settings → Sync and approve the request in the browser
window that opens.
### commonmeta sync
`commonmeta sync` enhances the items of a Zotero library with what the
commonmeta store knows about them. defter serves the API it uses under
`/api`; [`docs/commonmeta-rs-changes.md`](docs/commonmeta-rs-changes.md)
lists the commonmeta-rs changes needed for it to target a remote server (it
currently only talks to `http://localhost:23119/api`).
## Development
```sh
cargo test # unit and API tests
cargo run -- start # http://127.0.0.1:8189, files on local disk
```
Building needs only Rust. Pages are askama templates in `templates/`,
compiled into the binary, so a template change needs a rebuild (`cargo run`
does it). Their stylesheet, `ui/dist/app.css`, is built from `ui/app.css` by
the standalone [Tailwind CLI](https://tailwindcss.com/docs/installation/tailwind-cli)
and committed. To change styling, install it (`brew install tailwindcss`, or
a release binary with `TAILWINDCSS=/path/to/tailwindcss`); the build then
regenerates the stylesheet, and warns when it can't. The script is the
vendored htmx (`ui/vendor/`, see its README for updating) plus `ui/main.js`.
To run against real S3 storage locally, use [RustFS](https://rustfs.com):
```sh
docker run -d --rm --name rustfs -p 9000:9000 \
-e RUSTFS_ACCESS_KEY=defter -e RUSTFS_SECRET_KEY=defter-secret rustfs/rustfs
curl --aws-sigv4 "aws:amz:us-east-1:s3" --user defter:defter-secret \
-X PUT http://127.0.0.1:9000/zotero # create the bucket
DEFTER_S3_BUCKET=zotero DEFTER_S3_ENDPOINT=http://127.0.0.1:9000 \
DEFTER_S3_REGION=us-east-1 DEFTER_S3_ACCESS_KEY_ID=defter \
DEFTER_S3_SECRET_ACCESS_KEY=defter-secret cargo run -- start
DEFTER_KEY=$(cargo run -q -- key) scripts/smoke-s3.sh # file round trip
```
### Layout
| `src/main.rs` | CLI and startup |
| `src/app.rs` | state, router, gzip and logging middleware |
| `src/zotero.rs` | Zotero API handlers |
| `src/auth.rs` | keys, login sessions, expiring tokens |
| `src/store.rs`, `migrations/` | library storage in SQLite |
| `src/query.rs` | item listing parameters |
| `src/db.rs` | connections and migrations |
| `src/storage.rs` | attachment files via `object_store` |
| `src/ui.rs`, `templates/` | web UI pages (askama) |
| `ui/` | the pages' stylesheet (Tailwind) and script (vendored htmx) |
## License
MIT. Includes code adapted from zhost (MIT), see `NOTICE`.