readstor 0.2.0

ReadStor is a simple CLI for exporting user-generated data from Apple Books. The goal of this project is to facilitate data-migration from Apple Books to any other platform. Currently Apple Books provides no simple way to do this. Exporting is possible but not ideal and often times truncates long annotations.

Version 0.1.x contained the core functionality: (1) save all annotations and notes as JSON (2) render them via a custom (or the default) template using the Tera syntax or (3) backup the current Apple Books databases. See Output Structure for more information.

Note that this repository is a heavy work-in-progress and things are bound to change.

Installation

Using Homebrew

$ brew tap tnahs/readstor
$ brew install readstor

$ readstor --version

Using Cargo

$ cargo install readstor

CLI

$ readstor --help

readstor 0.2.0
A CLI for Apple Books annotations

USAGE:
    readstor [OPTIONS] <SUBCOMMAND>

OPTIONS:
    -o, --output <OUTPUT>    Sets the OUTPUT path [default: ~/.readstor]
    -f, --force              Runs even if Apple Books is open
    -v                       Sets the logging verbosity
    -h, --help               Print help information
    -V, --version            Print version information

SUBCOMMANDS:
    export    Exports Apple Books' data to OUTPUT
    render    Renders annotations via a template to OUTPUT
    backup    Backs-up Apple Books' databases to OUTPUT
    help      Print this message or the help of the given subcommand(s)

Version Support

The following versions have been verified as working.

Note that using iCloud to "Sync collections, bookmarks, and highlights across devices" is currently unverified and might produce unexpected results.

• macOS Monterey 12.x
  • Apple Books 4.1
  • Apple Books 4.2
• macOS Big Sur 11.x
  • Apple Books 3.2

Output Structure

export

[output] ── [default: ~/.readstor]
 │
 └─ data
     │
     ├─ Author - Title
     │   │
     │   ├─ data
     │   │   ├─ book.json
     │   │   └─ annotations.json
     │   │
     │   └─ resources
     │       ├─ .gitkeep
     │       ├─ Author - Title.epub   ─┐
     │       ├─ cover.jpeg             ├─ These are not exported.
     │       └─ ...                   ─┘
     │
     ├─ Author - Title
     │   └─ ...
     │
     └─ ...

render

[output] ── [default: ~/.readstor]
 │
 └─ renders
     │
     ├─ default ── (omitted if a custom template is used)
     │   ├─ Author - Title.[template-ext]
     │   ├─ Author - Title.txt
     │   └─ ...
     │
     ├─ [template-name]
     │   ├─ Author - Title.[template-ext]
     │   ├─ Author - Title.txt
     │   └─ ...
     │   
     └─ ...

backup

[output] ── [default: ~/.readstor]
 │
 └─ backups
     │
     ├─ 2021-01-01-000000 v3.2-2217 ── [YYYY-MM-DD-HHMMSS VERSION]
     │   │
     │   ├─ AEAnnotation
     │   │   ├─ AEAnnotation*.sqlite
     │   │   └─ ...
     │   │
     │   └─ BKLibrary
     │       ├─ BKLibrary*.sqlite
     │       └─ ...
     │
     │─ 2021-01-02-000000 v3.2-2217
     │   └─ ...
     │
     └─ ...

1.x Target

USAGE:
    readstor [OPTIONS] <SUBCOMMAND>

OPTIONS:
    -o, --output <OUTPUT>    Sets the OUTPUT path [default: ~/.readstor]
    -f, --force              Runs even if Apple Books is open
    -v                       Sets the logging verbosity
    -h, --help               Print help information
    -V, --version            Print version information

SUBCOMMANDS:
    export            Exports Apple Books' data to OUTPUT
    render            Renders annotations via a template to OUTPUT
    backup            Backs-up Apple Books' databases to OUTPUT
    help              Print this message or the help of the given subcommand(s)
    dump              Runs 'save', 'export' and 'backup'
    save              Saves Apple Books' database data to OUTPUT
    export            Exports annotations/books via templates to OUTPUT
    backup            Backs-up Apple Books' databases to OUTPUT
    sync              Adds new annotations/books from AppleBooks to the USER-DATABASE
    add               Adds an annotation/book to the USER-DATABASE
    search <QUERY>    Searches the USER-DATABASE
    random            Returns a random annotation from the USER-DATABASE
    check             Prompts to delete unintentional annotations from the USER-DATABASE
    info              Prints ReadStor info

# `~/.readstor/config.toml`

output = "./output"
templates = "./templates"
user-database = "./database.sqlite"
backup = true
extract-tags = true

Creating a Custom Template

Syntax

The templating syntax is based on Jinja2 and Django templates. In a nutshell, values are accessed by placing an attribute between {{ }} e.g. {{ book.title }}. Filters can manipulate the accessed values e.g. {{ name | capitalize }}. And statements placed between {% %} e.g. {% if my_var %} ... {% else %} ... {% endif %}, can be used for control flow. For more information, see the Tera documentation.

Attributes

Every template has access to two object: the current book as book and its annotations as annotations.

Book

book {
    title
    author
    metadata {
        id
        last_opened
    }
}

Book Attributes

Book Example

Here the date filter is used to format a datetime object into a human-readable date.

title: {{ book.title }}
author: {{ book.author }}
last-opened: {{ book.metadata.last_opened | date }}

Annotations

annotations [
    annotation {
        body
        style
        notes
        tags
        metadata {
            id
            book_id
            created
            modified
            location
            epubcfi
        }
    },
    ...
]

Annotations Attributes

Annotation Example

Here the join_paragraph filter concatenates a list of strings with line-breaks and the join filter does the same but with a specific separator passed to the sep keyword. This example also shows how to loop over the annotations using the {% for %} ... {% endfor %} statement.

{% for annotation in annotations %}

{{ annotation.body | join_paragraph }}

notes: {{ annotation.notes }}
tags: {{ annotation.tags | join(sep=" ") }}

{% endfor %}