Crate peermerge

Introduction

Peermerge is a Rust crate for managing JSON-like documents with multiple writers without a central authority. Concurrent writing is made possible with the CRDT implementation of automerge, and communication between peers with hypercore-protocol. Optional support for p2panda and other peer protocols, instead of hypercore, is planned in the future.

Peermerge can be built with WASM for browser support and uses either the tokio or async-std runtimes.

This crate exposes a low-level API that the following crates from the same family utilize:

• peermerge-tcp: TCP/IP bindings to an existing peermerge
• peermerge-hub: configurable (tokio) runner that utilizes peermerge-tcp to host a proxy peermerge and adds back-up and logging
• peermerge-server: configurable axum web server built on top of peermerge-hub, which adds websocket bindings and static HTML file serving.

Design

The main abstraction is Peermerge, which is a repository that stores documents. For a single peer (device or server) there usually isn’t a need to create and/or open more than one Peermerge instance.

Peermerge is designed to be cheap to clone and sent to spawned threads, and data within it can be either stored in memory or on disk. Disk storage is naturally more common, but in-memory storage can also be useful for example for WASM builds where a peer (i.e. browser) doesn’t have a file system. (To prevent a new peer from being created on every browser refresh, see reattach_secrets.)

Peermerge exposes the full interface of automerge’s AutoCommit via the methods transact and transact_mut, for reading and writing, respectively.

Connecting to a peer is done via [AsyncRead]/[AsyncWrite] supported FeedProtocol passed onto either connect_protocol_disk or connect_protocol_memory.

Features

hypercore-feed (default, mandatory)

Future-compatible, currently mandatory, feature to use the hypercore feed.

tokio-runtime (default)

Use the tokio runtime, on by default. Either this or async-std-runtime is mandatory.

async-std-runtime

Use the async-std runtime. Either this or tokio-runtime is mandatory.

wasm-bindgen (default)

Enable support for WASM compilation and runtime.

log

Enable support for log from tracing.

channel-writer

Export a convenient ChannelWriter wrapper and the bytes crate and .

Example

TODO

See also

Peermerge is heavily inspired by the original Javascript hypermerge package, which targeted now obsolete versions of both hypercore and automerge.

Re-exports

• pub use automerge;
• pub use uuid;

Structs

• AttachDocumentDiskOptions
  Options for attaching an existing document to a disk Peermerge
• AttachDocumentDiskOptionsBuilder
  Builder for AttachDocumentDiskOptions.
• AttachDocumentMemoryOptions
  Options for attaching an existing document to an in-memory Peermerge
• AttachDocumentMemoryOptionsBuilder
  Builder for AttachDocumentMemoryOptions.
• CreateNewDocumentDiskOptions
  Options for creating new disk document.
• CreateNewDocumentDiskOptionsBuilder
  Builder for CreateNewDocumentDiskOptions.
• CreateNewDocumentMemoryOptions
  Options for creating new in-memory document.
• CreateNewDocumentMemoryOptionsBuilder
  Builder for CreateNewDocumentMemoryOptions.
• DocumentInfo
  Information about a document
• DocumentSharingInfo
  Document sharing information
• DynamicDocumentInfo
  Dynamic document information. Can change over the normal course of the document.
• FeedDiskPersistence
  Main constructor.
• FeedMemoryPersistence
  In-memory storage for random access
• FeedProtocol
  A Protocol stream.
• NameDescription
  Name and optional description
• OpenDiskOptions
  Options for opening an existing peermerge
• OpenDiskOptionsBuilder
  Builder for OpenDiskOptions.
• Peermerge
  Peermerge is the main abstraction and a store for multiple documents.
• PeermergeDiskOptions
  Disk Peermerge options
• PeermergeDiskOptionsBuilder
  Builder for PeermergeDiskOptions.
• PeermergeMemoryOptions
  In-memory Peermerge options
• PeermergeMemoryOptionsBuilder
  Builder for PeermergeMemoryOptions.
• ProtocolBuilder
  Build a Protocol instance with options.
• RandomAccessDisk
  Main constructor.
• RandomAccessMemory
  In-memory storage for random access
• StateEvent
  State event from a document
• StaticDocumentInfo
  Static information about the document. Will never change over the course of the document’s lifetime.
• UrlDocumentInfo
  Document information parsed from a document URL. NB: This is a limited set of information compared to what is received when manipulating documents in a live peermerge, i.e. DocumentInfo. Most notably, the URL will not contain a full list of child documents because that list can grow infinitely and thus cause the URL to exceed the maximum character limit.

Enums

• AccessType
  Access type to a document.
• AttachDocumentDiskOptionsBuilderError
  Error type for AttachDocumentDiskOptionsBuilder
• AttachDocumentMemoryOptionsBuilderError
  Error type for AttachDocumentMemoryOptionsBuilder
• CreateNewDocumentDiskOptionsBuilderError
  Error type for CreateNewDocumentDiskOptionsBuilder
• CreateNewDocumentMemoryOptionsBuilderError
  Error type for CreateNewDocumentMemoryOptionsBuilder
• FeedType
  Type of feed.
• OpenDiskOptionsBuilderError
  Error type for OpenDiskOptionsBuilder
• PeermergeDiskOptionsBuilderError
  Error type for PeermergeDiskOptionsBuilder
• PeermergeError
  Common error type for the peermerge interface
• PeermergeMemoryOptionsBuilderError
  Error type for PeermergeMemoryOptionsBuilder
• StateEventContent
  Content of a StateEvent

Constants

• DEFAULT_MAX_ENTRY_DATA_SIZE_BYTES
  Use the max size of 1 MiB also as the default value for a single data chunk in a feed.
• DEFAULT_MAX_WRITE_FEED_LENGTH
  Use 2^8 as the default maximum length of a write feed. After this length is reach, a new write feed created which replaces the old.
• MAX_ENTRY_DATA_SIZE_BYTES
  Use 1 MiB as the absolute max limit for a single data chunk in a feed.

Traits

• FeedPersistence
  Type for a feed’s persistence
• IO
  Input/Output trait using AsyncRead/AsyncWrite
• RandomAccess
  Interface for reading from, writing to and deleting from a randomly accessible storage of bytes.

Functions

• decode_base64_nopad
  Decode with Base64 URL-compatible without padding. Convenience method to create byte arrays from automerge keys.
• encode_base64_nopad
  Encode with Base64 URL-compatible without padding. Convenience method to create short strings from byte arrays to use as automerge keys. strings from byte arrays to use as automerge keys.
• get_document_info
  Get document information from document URL and optional document secret.
• new_uuid_v4
  Generate new v4 UUID. Convenience method to generate UUIDs using just peermerge without importing the uuid crate and configuring WASM support.

Type Aliases

• AutomergeDoc
  Automerge document type, set to AutoCommit.
• DocumentId
  Id of a document. Derived by hashing from FeedDiscoveryKey.
• FeedDiscoveryKey
  Discovery key of the feed. Derived by hashing from FeedPublicKey.
• FeedPublicKey
  Public key of the feed.
• PeerId
  Id of a peer