Crate snowdon

source ·
Expand description

This crate is an implementation of snowflake IDs - an ID format introduced by Twitter as IDs for tweets.

Snowflakes consist of a timestamp (with millisecond precision) and a sequence number to allow generators to create multiple snowflakes in the same millisecond. The original version used by Twitter also includes a machine ID. This allows distributed systems to create a unique snowflake without requiring coordination between the instances.

While the original implementation of snowflake IDs includes a fixed layout and epoch, this implementation is designed to work with various kinds of snowflake-like IDs. Our only requirements are that the snowflake contains a timestamp and a sequence number in that order (other constant parts like the machine ID can still interleave). Nevertheless, the original layout introduced by Twitter is implemented as ClassicLayout.

The original Twitter snowflake implementation can be found here. You can read more about snowflakes in Twitter’s blog post about them.

Serde compatibility

Snowflake and SnowflakeComparator implement Serde’s Serialize and Deserialize traits if the crate feature serde is enabled. To enable Serde compatibility, simply opt in by changing your dependency line in Cargo.toml to:

[dependencies]
# ...
snowdon = { version = "^0.2", features = ["serde"] }

Note that Generator doesn’t implement (de-)serialization even if the feature is enabled. A generator’s state highly depends on the machine it’s running on, and usually, there’s no reason to serialize it or otherwise store and load it somehow.

Example

The example below implements a custom layout that uses 42 bits for the timestamp, 10 bits for a machine ID, and 12 bits for the sequence number. Note that this example doesn’t include the machine ID implementation. There isn’t a single “valid” implementation of machine IDs, as they heavily depend on your application and the way you deploy your application. However, common ways to derive a machine ID include using the machine’s private IP address and provisioning a machine ID using some coordinated service. To avoid defaulting to an implementation that doesn’t work for most users, we don’t provide an implementation for this function at all.

use snowdon::{Epoch, Generator, Layout, Snowflake, SnowflakeComparator};
use std::time::SystemTime;

fn machine_id() -> u64 {
    // This is where you would implement your machine ID
}

// We make our structure hidden, as this is an implementation detail that most
// users of our custom ID won't need
#[derive(Debug)]
#[doc(hidden)]
pub struct MySnowflakeParams;

impl Layout for MySnowflakeParams {
    fn construct_snowflake(timestamp: u64, sequence_number: u64) -> u64 {
        assert!(
            !Self::exceeds_timestamp(timestamp)
                && !Self::exceeds_sequence_number(sequence_number)
        );
        (timestamp << 22) | (machine_id() << 12) | sequence_number
    }
    fn timestamp(input: u64) -> u64 {
        input >> 22
    }
    fn exceeds_timestamp(input: u64) -> bool {
        input >= (1 << 42)
    }
    fn sequence_number(input: u64) -> u64 {
        input & ((1 << 12) - 1)
    }
    fn exceeds_sequence_number(input: u64) -> bool {
        input >= (1 << 12)
    }
    fn is_valid_snowflake(input: u64) -> bool {
        // Our snowflake format doesn't have any constant parts that we could
        // validate here
        true
    }
}
impl Epoch for MySnowflakeParams {
    fn millis_since_unix() -> u64 {
        // Our epoch for this example is the first millisecond of 2015
        1420070400000
    }
}

// Define our snowflake and generator types
pub type MySnowflake = Snowflake<MySnowflakeParams, MySnowflakeParams>;
pub type MySnowflakeGenerator = Generator<MySnowflakeParams, MySnowflakeParams>;

// Use our types in our API
pub struct UserProfile {
    name: String,
    age: u8,
    employee_id: MySnowflake,
}

pub fn save_user_profile(profile: &UserProfile) {
    fn save_to_db(_id: MySnowflake, _profile: &UserProfile) {}
    // Snowflakes implement Copy, so we don't need to worry about moving the
    // value out of our profile here
    save_to_db(profile.employee_id, profile);
}

// You can compare snowflakes with each other and even with arbitrary timestamps
// (using `SnowflakeComparator`)
let generator = MySnowflakeGenerator::default();
let first_snowflake = generator.generate().unwrap();
let second_snowflake = generator.generate().unwrap();
assert!(first_snowflake < second_snowflake);

let timestamp = first_snowflake.timestamp().unwrap();
let comparator =
    SnowflakeComparator::from_system_time(SystemTime::UNIX_EPOCH).unwrap();
assert!(comparator < first_snowflake);
let comparator = SnowflakeComparator::from_system_time(timestamp).unwrap();
assert_eq!(first_snowflake, comparator);
// The comparator only represents a timestamp, so we can't expect it to be
// greater than `second_snowflake` here

Structs

Enums

Traits

Type Definitions

  • Result
    The primary result type of Snowdon.