defter 0.1.0

# defter

A self-hosted Zotero sync and file storage server: keep one Zotero library in
sync across your machines — metadata **and** attachment files such as PDFs —
without zotero.org. Metadata lives in SQLite, files in S3-compatible object
storage (AWS S3, Cloudflare R2, RustFS, …). Stock Zotero clients sync against
it, and [commonmeta](https://codeberg.org/front-matter/commonmeta-rs)'s `sync`
command can enrich the library through the same API.

**Status: early development (0.1).** The sync protocol, file storage and query
API are implemented and tested; interfaces may still change.

defter builds on [zhost](https://github.com/mulatta/zhost) (MIT, see
`NOTICE`), which worked out the subset of the Zotero Web API v3 a client
needs. defter swaps PostgreSQL for SQLite (rusqlite), `rust-s3` for the
`object_store` crate, and adds a web UI and the endpoints `commonmeta sync`
uses.

## How it works

- **Client redirect, no patching.** Zotero reads its sync server from the
  hidden pref `extensions.zotero.api.url`; pointing it at defter redirects
  sync and login.
- **Server.** A Rust/[axum](https://github.com/tokio-rs/axum) service. Library
  objects are stored as opaque JSON keyed by `(kind, key)`, with the columns
  queries need derived on every write. One library version counter, bumped by
  each write, keeps the client's `since` reads and `If-Unmodified-Since-Version`
  writes coherent.
- **Files.** Uploads stream through defter, which checks the md5 while writing
  them to the bucket; downloads redirect to a short-lived pre-signed URL, so
  the bytes come straight from the bucket.
- **Auth.** One user, one library. API keys are configured, each read/write or
  read-only. The Zotero client gets a read/write key through its browser login,
  which defter releases only after you approve it in the web UI.
- **Web UI.** A Svelte + Tailwind CSS dashboard (library statistics, storage,
  connection instructions), embedded in the binary. The login consent page is
  rendered by the server, works without JavaScript and shares the stylesheet.

The full contract is in [`docs/api.md`](docs/api.md).

## Running

```sh
cargo install --path .          # needs pnpm to build the UI, or a built ui/dist
defter start --public-url https://zotero.example.org \
  --s3-bucket zotero \
  --s3-endpoint https://<account>.r2.cloudflarestorage.com --s3-region auto \
  --s3-access-key-id-file /run/credentials/defter/s3-access-key-id \
  --s3-secret-access-key-file /run/credentials/defter/s3-secret-access-key
```

Every flag has a `DEFTER_*` environment variable; see `env.example` and
`defter start --help`. S3 credentials also fall back to the standard `AWS_*`
variables. Without `--s3-bucket`, files go to `<data-dir>/files` — fine for
development, not meant for production.

### API keys

With no key configured, defter generates a read/write key in
`<data-dir>/api-key`; `defter key` prints it. For real deployments provision
keys as files:

```sh
openssl rand -hex 32 > app-key   # read/write, handed to Zotero at login
openssl rand -hex 32 > cli-key   # read-only, for scripts and agents
defter start --keys rw:/path/app-key,ro:/path/cli-key …
```

### Deployment

defter listens on `127.0.0.1:8189`; put it behind a reverse proxy with HTTPS
and set `--public-url` to the proxy's address. The data API is protected by
keys and safe to expose. `/login` (which releases a key once approved),
`/ui/*` and `/` should be behind SSO (e.g. oauth2-proxy; add
`--login-authorized-user you@example.org` for defence in depth) or reachable
only on a private network.

Clients download from pre-signed URLs on `--s3-endpoint`, so the bucket
endpoint must be reachable from them; otherwise add `--proxy-downloads`.

### Connecting Zotero

In Zotero, Settings → Advanced → Config Editor:

```
extensions.zotero.api.url = https://zotero.example.org/api/
extensions.zotero.streaming.enabled = false
```

Then log in under Settings → Sync and approve the request in the browser
window that opens.

### commonmeta sync

`commonmeta sync` enhances the items of a Zotero library with what the
commonmeta store knows about them. defter serves the API it uses under
`/api`; [`docs/commonmeta-rs-changes.md`](docs/commonmeta-rs-changes.md)
lists the commonmeta-rs changes needed for it to target a remote server (it
currently only talks to `http://localhost:23119/api`).

## Development

```sh
pnpm --dir ui install
cargo test                 # builds the UI, then runs unit and API tests
cargo run -- start         # http://127.0.0.1:8189, files on local disk
pnpm --dir ui dev          # UI with hot reload, proxying to the server above
```

To run against real S3 storage locally, use [RustFS](https://rustfs.com):

```sh
docker run -d --rm --name rustfs -p 9000:9000 \
  -e RUSTFS_ACCESS_KEY=defter -e RUSTFS_SECRET_KEY=defter-secret rustfs/rustfs
curl --aws-sigv4 "aws:amz:us-east-1:s3" --user defter:defter-secret \
  -X PUT http://127.0.0.1:9000/zotero          # create the bucket
DEFTER_S3_BUCKET=zotero DEFTER_S3_ENDPOINT=http://127.0.0.1:9000 \
DEFTER_S3_REGION=us-east-1 DEFTER_S3_ACCESS_KEY_ID=defter \
DEFTER_S3_SECRET_ACCESS_KEY=defter-secret cargo run -- start
DEFTER_KEY=$(cargo run -q -- key) scripts/smoke-s3.sh   # file round trip
```

### Layout

| Path | |
|---|---|
| `src/main.rs` | CLI and startup |
| `src/app.rs` | state, router, gzip and logging middleware |
| `src/zotero.rs` | Zotero API handlers |
| `src/auth.rs` | keys, login sessions, expiring tokens |
| `src/store.rs`, `migrations/` | library storage in SQLite |
| `src/query.rs` | item listing parameters |
| `src/db.rs` | connections and migrations |
| `src/storage.rs` | attachment files via `object_store` |
| `src/ui.rs`, `ui/` | web UI: login page, assets, Svelte dashboard |

## License

MIT. Includes code adapted from zhost (MIT), see `NOTICE`.