Crate modyne

Expand description

Utilities for working with DynamoDB, particularly focusing on an opinionated approach to modeling data using single-table design.

In order to effectively use this library, you should first have completed a single-table design, identifying your entities and the access patterns that you will need to implement.

§Tables

Tables are defined via a trait implementation. To do this, we’ll need to expose a means for modyne to access the table’s name, a properly configured client, and the relevant keys for the table.

Below, we define a database that has one global secondary index in addition to the default primary key.

use modyne::{keys, Table};

struct Database {
    table_name: String,
    client: aws_sdk_dynamodb::Client,
}

impl Table for Database {
    type PrimaryKey = keys::Primary;
    type IndexKeys = keys::Gsi1;

    fn table_name(&self) -> &str {
        &self.table_name
    }

    fn client(&self) -> &aws_sdk_dynamodb::Client {
        &self.client
    }
}

§Primary keys and indexes

While a default primary key and generic indexes are provided, it is possible to define your own primary key or index if desired. These types must be serde-serializable.

use modyne::keys::{
    GlobalSecondaryIndexDefinition, IndexKey, Key, KeyDefinition,
    PrimaryKey, PrimaryKeyDefinition, SecondaryIndexDefinition
};

#[derive(Debug, serde::Serialize)]
struct SessionToken {
    session_token: String,
}

impl PrimaryKey for SessionToken {
    const PRIMARY_KEY_DEFINITION: PrimaryKeyDefinition = PrimaryKeyDefinition {
        hash_key: "session_token",
        range_key: None,
    };
}

impl Key for SessionToken {
    const DEFINITION: KeyDefinition =
        <Self as PrimaryKey>::PRIMARY_KEY_DEFINITION.into_key_definition();
}

#[derive(Debug, serde::Serialize)]
struct UserIndex {
    user_id: String,
}

impl IndexKey for UserIndex {
    const INDEX_DEFINITION: SecondaryIndexDefinition = GlobalSecondaryIndexDefinition {
        index_name: "user_index",
        hash_key: "user_id",
        range_key: None,
    }.into_index();
}

§Entities

Entities are the heart of the data model. An instance of an entity represents a single item in a DynamoDB table. An entity will always have the same primary key as the assoicated table, but may also participate in zero or more secondary indexes.

For more information on setting up an entity, see EntityDef and Entity.

use modyne::{keys, Entity, EntityDef};

#[derive(Debug, EntityDef, serde::Serialize, serde::Deserialize)]
struct Session {
    user_id: String,
    session_token: String,
}

impl Entity for Session {
    type KeyInput<'a> = &'a str;
    type Table = Database;
    type IndexKeys = keys::Gsi1;

    fn primary_key(input: Self::KeyInput<'_>) -> keys::Primary {
        keys::Primary {
            hash: format!("SESSION#{}", input),
            range: format!("SESSION#{}", input),
        }
    }

    fn full_key(&self) -> keys::FullKey<keys::Primary, Self::IndexKeys> {
        keys::FullKey {
            primary: Self::primary_key(&self.session_token),
            indexes: keys::Gsi1 {
                hash: format!("USER#{}", self.user_id),
                range: format!("SESSION#{}", self.session_token),
            },
        }
    }
}

Entities can be interacted with through utility methods on the EntityExt trait.

use modyne::EntityExt;

let mk_session = || Session {
    session_token: String::from("session-1"),
    user_id: String::from("user-1"),
};

let upsert = mk_session().put();
let create = mk_session().create();
let replace = mk_session().replace();
let delete = Session::delete("session-1");
let get = Session::get("session-1");
let update = Session::update("session-1");

§Projections

A projection is a read-only view of some subset of an entity’s attributes. Every entity is trivially its own projection. Projections can be defined manually or by using the Projection derive macro.

use modyne::{EntityDef, Projection};

#[derive(Debug, EntityDef, serde::Serialize, serde::Deserialize)]
struct Session {
    user_id: String,
    session_token: String,
}

#[derive(Debug, Projection, serde::Deserialize)]
#[entity(Session)]
struct SessionTokenOnly {
    session_token: String,
}

The derive macro for projections includes a minimal amount of verification to ensure that the field names match names know about from the projected entity. Note that if the entity or the projection use the flatten attribute, then this detection algorithm will not be able to identify misnamed fields. As an example, the following will fail to compile.

#[derive(Debug, Projection, serde::Deserialize)]
#[entity(Session)]
struct SessionTokenOnly {
    session: String,
}

However, serde field attributes can be used to rename fields so that they will appropriately match up if a different field name in the struct is desired.

#[derive(Debug, Projection, serde::Deserialize)]
#[entity(Session)]
struct SessionTokenOnly {
    #[serde(rename = "session_token")]
    session: String,
}

§Aggregates and queries

The most efficient way to pull data out of a DynamoDB table is using range queries that can extract many entities in one operation. To support this, we provide means of ergonomically processing the variety of entities that might be returned in a single query through an aggregate.

use modyne::{
    expr, keys, projections, read_projection,
    Aggregate, Error, Item, QueryInput, QueryInputExt,
};

struct UserInfoQuery<'a> {
    user_id: &'a str,
}

impl QueryInput for UserInfoQuery<'_> {
    type Index = keys::Gsi1;
    type Aggregate = UserInfo;

    fn key_condition(&self) -> expr::KeyCondition<Self::Index> {
        let partition = format!("USER#{}", self.user_id);
        expr::KeyCondition::in_partition(partition)
    }
}

#[derive(Debug, Default)]
struct UserInfo {
    session_tokens: Vec<String>,
    metadata: Option<UserMetadata>,
}

projections! {
    enum UserInfoEntities {
        Session,
        UserMetadata,
    }
}

impl Aggregate for UserInfo {
    type Projections = UserInfoEntities;

    fn merge(&mut self, item: Item) -> Result<(), Error> {
        match read_projection!(item)? {
            Self::Projections::Session(session) => {
                self.session_tokens.push(session.session_token)
            }
            Self::Projections::UserMetadata(user) => {
                self.metadata = Some(user)
            }
        }
        Ok(())
    }
}

impl Database {
    async fn get_user_info_page(&self, user_id: &str) -> Result<UserInfo, Error> {
        let result = UserInfoQuery { user_id: "test" }
            .query()
            .execute(self)
            .await?;

        let mut info = UserInfo::default();
        info.reduce(result.items.unwrap_or_default())?;
        Ok(info)
    }
}

§Non-standard tables

Modyne is generally opinionated about how to manage entities in a single-table design. However, certain users may want to manage that information in a slightly different way for interoperability with already existing ecosystems migrating into Modyne. For these cases, the Table trait has a few items that can be overriden.

§Using a non-standard entity type attribute name

If the attribute that holds the entity type name is not entity_type, then you can specify the appropriate attribute name by overriding the ENTITY_TYPE_ATTRIBUTE constant.

use modyne::{keys, Table};

impl Table for Database {
    const ENTITY_TYPE_ATTRIBUTE: &'static str = "et";

    type PrimaryKey = keys::Primary;
    type IndexKeys = keys::Gsi1;

    fn table_name(&self) -> &str {
        &self.table_name
    }

    fn client(&self) -> &aws_sdk_dynamodb::Client {
        &self.client
    }
}

This table will now store and retreive the entity type name from the et attribute on an item, which it will expect to be a DynamoDB string value.

§Using a non-standard entity type attribute value

If the attribute that holds the entity type name does not use a DynamoDB string value, then you can override the behavior for embedding and extracting the entity type name by overriding the serialization functions for the table.

use modyne::{keys, AttributeValue, EntityTypeNameRef, MalformedEntityTypeError, Table};

impl Table for Database {
    type PrimaryKey = keys::Primary;
    type IndexKeys = keys::Gsi1;

    fn table_name(&self) -> &str {
        &self.table_name
    }

    fn client(&self) -> &aws_sdk_dynamodb::Client {
        &self.client
    }

    fn deserialize_entity_type(
        attr: &AttributeValue,
    ) -> Result<&EntityTypeNameRef, MalformedEntityTypeError> {
        let values = attr
            .as_ss()
            .map_err(|_| MalformedEntityTypeError::Custom("expected a string set".into()))?;
        let value = values
            .first()
            .expect("a DynamoDB string set always has at least one element");
        Ok(EntityTypeNameRef::from_str(value.as_str()))
    }

    fn serialize_entity_type(entity_type: &EntityTypeNameRef) -> AttributeValue {
        AttributeValue::Ss(vec![entity_type.to_string()])
    }
}

The above example uses the standard entity_type attribute, but stores and retrieves the name as a single-element string set. You can combine both of these overrides to use a non-standard value codec with a non-standard attribute name.

§Features

derive: Re-exports the derive macros provided by the modyne-derive crate.
once_cell: Uses the lazy initialization primitives provided by the once_cell crate.

§Minimum supported Rust version (MSRV)

The minimum supported Rust version for this crate is 1.70.0. The MSRV can be reduced to 1.68.0 by enabling the once_cell feature to use that crate’s lazy initialization primitives if desired.

Modules§

expr: Expression builders
keys: Types representing DynamoDB keys in a single-table design
model: Models for interacting with DynamoDB
types: Types useful as attributes in DynamoDB items

Macros§

once_projection_expression: Generate a static projection expression that is computed exactly once in the lifetime of the program
projections: Utility macro for defining an ProjectionSet used when querying items into an Aggregate
read_projection: Utility macro for reading an entity from a DynamoDB item

Structs§

EntityTypeName: The name for a DynamoDB entity type
EntityTypeNameRef: The name for a DynamoDB entity type
Error: An error that occurred while interacting with DynamoDB

Enums§

AttributeValue

Represents the data for an attribute.

Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

For more information, see Data Types in the Amazon DynamoDB Developer Guide.

MalformedEntityTypeError

The entity type attribute was found, but was malformed and could not be extracted

Traits§

Aggregate: An aggregate of multiple entity types, often used when querying multiple items from a single partition key.
Entity: An entity in a DynamoDB table
EntityDef: The name and attribute definition for an Entity
EntityExt: Extension trait for Entity types
Projection: A projection of an entity that may not contain all of the entity’s attributes
ProjectionExt: Extension trait for Projection types
ProjectionSet: A description of the set of entity types that constitute an Aggregate
QueryInput: A value that can be used to query an aggregate
QueryInputExt: Extensions to an aggregate query
ScanInput: A value that can be used to query an aggregate
ScanInputExt: Extensions to an aggregate scan
Table: A description of a DynamoDB table
TestTableExt: Extension trait for Table to provide convenience methods for testing operations

Type Aliases§

Item: An alias for a DynamoDB item

Derive Macros§

EntityDef: Derive macro for the EntityDef trait
Projection: Derive macro for the Projection trait

Crate modyne

Crate modyne Copy item path

§Tables

§Primary keys and indexes

§Entities

§Projections

§Aggregates and queries

§Non-standard tables

§Using a non-standard entity type attribute name

§Using a non-standard entity type attribute value

§Features

§Minimum supported Rust version (MSRV)

Modules§

Macros§

Structs§

Enums§

Traits§

Type Aliases§

Derive Macros§

Crate modyne