cosmic-pim

The shared substrate under the COSMIC personal-information suite: storage, sync, credentials, and the iCalendar/vCard layer.

This repository holds no user interface. It is the layer three applications sit on so that a sync bug is fixed once rather than three times.

The suite

Names: Slate holds what's on your slate; Circle is your circle of people; Envelope is the universal mail symbol as a word.

The crates

Dependencies point downward only. See ARCHITECTURE.md for the diagram, the invariants, and where new code belongs.

Accounts

Signing in works two ways, and an application does not have to care which.

Most servers — Fastmail, Nextcloud, Migadu, a university, a Synology box — take a URL and an app password. Google and Microsoft withdrew password authentication and take OAuth, so the suite runs the authorization-code flow itself: PKCE, a loopback redirect, and a refresh token renewed when it expires.

use cosmic_pim_accounts::{AccountStore, Registry};

let registry = Registry::load();
let provider = registry.get("fastmail").expect("built in");

// Everything the manifest knows — CalDAV, CardDAV, mail — filled in.
let account = provider.account_for("ada@fastmail.com");
accounts.add(account, &app_password)?;

An OAuth provider is the same shape with the flow in between:

let oauth = provider.oauth.as_ref().expect("this provider uses OAuth");
let pending = cosmic_pim_auth::begin(oauth)?;
open_in_the_users_browser(pending.authorize_url());

let grant = pending.exchange(&pending.wait()?, oauth)?;
// The grant says whose it is, so nobody typed an address first.
let address = grant.identity.expect("the manifest asks for `openid`").email;
accounts.add_oauth(provider.account_for(&address), &provider.id, &grant.credential)?;

From there nothing distinguishes the two. cosmic_pim_auth::resolve hands back the secret of the moment — a password, or an access token it renewed and re-stored on the way — and the CalDAV client sends Basic or Bearer, the IMAP session LOGIN or AUTHENTICATE XOAUTH2, without knowing which.

Who signed in is read from the provider, not from a form. The identity comes from the ID token in the token endpoint's own response, and is believed only when the endpoint is https, the token was issued to this application's client id, it has not expired, and the provider does not mark the address unverified. A token that fails one of those stops the sign-in.

An add-account window does not have to assemble any of this. cosmic_pim_sync::setup takes a typed address and answers, with no network, which provider it belongs to and every way of signing in that can work on this installation, best first — the provider's own sign-in where a client id is configured, GNOME Online Accounts where it is running, a password otherwise. Then one call finishes the job and stores the account for every application in the suite:

use cosmic_pim_sync::setup::{self, Route};

let plan = setup::plan(&registry, "ada@fastmail.com", online_accounts_running)
    .expect("an address");
let id = match plan.routes.first() {
    // Finds the servers, tries the password, stores only what was accepted.
    Some(Route::Password { .. }) => {
        setup::add_with_password(&mut accounts, &registry, &plan, "Ada", &password)?
    }
    // The browser flow; the account is named after whoever signed in.
    Some(Route::SignIn) => setup::sign_in(&mut accounts, &plan, open_in_the_users_browser)?,
    _ => return Ok(()), // Online Accounts, or no way in on this installation
};

When the address is not enough, the window asks, and three more calls take the answer and still try it before storing anything: setup::add_with_mail_servers for mail servers the person named, setup::add_with_dav_url for a typed CalDAV or CardDAV address (and setup::add_with_dav_login for a server form with a login and no address), and setup::sign_in_with_provider for a Google Workspace or Microsoft 365 address on a domain of its own, where the person picks the provider.

Providers are data. Google, Microsoft, Fastmail, iCloud, Yahoo, AOL, Proton Mail (through Proton Mail Bridge), mailbox.org, Posteo and GMX ship compiled in, each with the mail domains that are its own, so an address is enough to find it. A manifest adds a provider or overrides a field of one, and is read from three places, each overlaying the one before:

No OAuth client id is shipped — one identifies the application asking, and there is none this project could publish that would be right for a downstream package — so Google and Microsoft need a two-line manifest before they can be used. A distribution ships it once for every user:

# /usr/share/cosmic-pim/providers/google.toml
id = "google"
[oauth]
client_id = "…apps.googleusercontent.com"

A server on loopback is not asked for a real certificate. Proton Mail Bridge serves IMAP and SMTP on 127.0.0.1 with a certificate it signed itself, as any local server must. For a host written as a loopback address the TLS handshake accepts it; for every other host, including the name localhost, the certificate is verified as usual.

An account added in one application reaches the others while they run. The account list is one file for the whole suite, and a running application notices that something else rewrote it by keeping the file's FileStamp — its size and modification time, one stat — and comparing it every ACCOUNTS_FILE_CHECK_INTERVAL:

use cosmic_pim_accounts::FileStamp;
use cosmic_pim_accounts::account::default_config_path;

let mut stamp = FileStamp::accounts_file(); // when the list is first read
// …then on every tick:
if stamp.changed(&default_config_path()) {
    accounts.reload()?;
}

Using these

On crates.io. Depend on the crates you need:

[dependencies]
cosmic-pim-core = "3"
cosmic-pim-sync = "3"

# A contacts or calendar app with no mail: keep the mail and OpenPGP stack
# out of the build.
# cosmic-pim-sync = { version = "3", default-features = false }

# Uncomment to develop against a sibling checkout.
# [patch.crates-io]
# cosmic-pim-core = { path = "../cosmic-pim/crates/cosmic-pim-core" }

CHANGELOG.md lists every public API change, with what to call instead.

Sending

A message that could not be sent waits in the account's outbox (cosmic_pim_mail::Outbox) and leaves on a later pass. Three things about it are deliberate:

• A drain reports what it did even when it stops early. drain returns a DrainOutcome, not a Result: a local failure part-way is a field beside the messages already sent, so their ids cannot be lost with an error.
• Why a message stopped is a type. Queued::failure is a SendFailure — refused by the server (with its reason), possibly delivered, or simply not reached — stored on the record, so an application words each case itself. A send that may have been delivered is never retried automatically.
• A reply remembers what it answers. Answers rides on the queued record and comes back with the entry a drain reports sent: the Message-ID, which survives a renumbering, and the mailbox and UID, which are exact while the numbering stands. Index::locate turns the id back into a place.

What you get for free

Reading a calendar, with sync, is about this much:

use cosmic_pim_core::store::Store;

let store = Store::open_default()?;
let today = chrono::Local::now().date_naive();
let days = store.occurrences_by_day(today, today + chrono::Duration::days(7), &hidden)?;
let tasks = store.todos(&hidden);

An address book:

use cosmic_pim_core::store::contacts::ContactStore;

let book = ContactStore::open_default()?;
let matches = book.search("lovelace");

One sync pass over every enabled account — calendars and address books together, failing per collection rather than per run:

let reports = cosmic_pim_sync::sync_all(&mut accounts, &registry, &calendar_root, &contacts_root);

A local edit and the upload it needs, as one step, so that a sync pass can neither overwrite the edit before it is queued nor miss it:

let saved = cosmic_pim_sync::save_and_queue(
    &calendar_root,
    &event.calendar_id,
    &[&event.file_name],
    || store.save(&event),
)?;
if let Err(why) = saved.queued {
    // Saved on this device, but not queued for the server.
}

Mail, in whichever protocol the account uses — IMAP, JMAP, POP3, the Gmail API or Microsoft Graph. Every one of them lands in the same maildir, because each provider API is used as a change feed while the message bytes still come from its raw endpoint:

let secret = cosmic_pim_auth::resolve(&mut accounts, &registry, &account.id)?;
let report = cosmic_pim_sync::sync_account_mail(
    &account,
    &cosmic_pim_sync::credentials_for(&secret),
    &mail_root,
    Default::default(),
    now_ms,
)?;

When the server and this device changed the same event, the pass first tries to settle it without asking: an app that saves through save_and_queue (which captures the pre-edit bytes) gets non-overlapping changes three-way merged and re-queued automatically. Only genuinely overlapping edits are recorded — local, remote, and base intact — for the user to decide:

for (collection, conflict) in cosmic_pim_sync::conflicts(&calendar_root) {
    // conflict.local and conflict.remote are both here; the user chooses.
    cosmic_pim_sync::conflict::take_remote(&calendar_root, &collection, &conflict.href)?;
}

An ICS subscription — a holiday calendar, a timetable, anything published as a webcal:// URL — becomes an ordinary read-only collection, refreshed with conditional requests and split into one file per event so every vdir reader sees it:

use cosmic_pim_caldav::feed;

let meta = feed::subscribe(&calendar_root, "Holidays", url, color, None)?;
feed::refresh(&meta.path, now_ms)?;

And mail can be pushed rather than polled, where the server offers it — IMAP IDLE, or JMAP's event source — with one blocking primitive on each session:

match imap_session.watch("INBOX", Duration::from_secs(25 * 60))? {
    Watched::Changed => { /* run a sync pass */ }
    Watched::TimedOut => { /* watch again */ }
}

Interoperability

The on-disk layout is vdir — a directory per collection, one .ics or .vcf per item. That is not an implementation detail we happened to land on; it is the point. khal, khard, vdirsyncer, and Thunderbird read the same files, and a user can walk away with their data as plain text at any moment.

Building

cargo test --workspace

Licensing

MPL-2.0. The applications are GPL-3.0-only and link these crates without absorbing them. LICENSING.md explains why that split exists, why not MIT and not LGPL, and the one trap (MPL Exhibit B) that would break it.

The iCalendar, CalDAV, and mail layers derive from the Meltemi project, which now grants MPL-2.0 on the donor files explicitly — publishing here is no longer licence-blocked. Details in LICENSING.md.