Expand description
Snowcloud
a small library for implementing custom ids based on timestamps, static ids, and sequence counters. the module provides 2 types of generators, a thread safe and non thread safe version. they allow for different types of waiting for ids if you want specific behavior. the snowflakes generated by a generator are composed of 3 sections and are described below. small example of how to create a generator and create a snowflake is shown below.
// 43 bit timestamp, 8 bit primary id, 12 bit sequence
type MyCloud = snowcloud::SingleThread<43, 8, 12>;
// 2023/03/23 9:00:00 in milliseconds, timestamps will start from this
// date
const START_TIME: u64 = 1679587200000;
// primary id could be a machine/node id for example
const PRIMARY_ID: i64 = 1;
let mut cloud = MyCloud::new(PRIMARY_ID, START_TIME)
.expect("failed to create MyCloud");
let flake = cloud.next_id()
.expect("failed to create snowflake");
println!("{}", flake.id());Behavior
MultiThread is a thread safe implementation for sharing between threads
on a system. it uses an Arc Mutex
to handle sharing the sequence count and prev_time. the only time it will
block is when acquiring the mutex and will not wait if a valid snowflake
cannot be acquired. if a generator is unable to create a snowflake because
the max sequence number has been reached an error will be returned
providing an estimated duration to the next millisecond. how you want to
wait can be decided by the user.
SingleThread is similar in most aspects to MultiThread expect next_id
is a mutating call and sequence count with prev_time are not stored in an
Arc Mutext. THIS IS NOT THREAD SAFE.
Traits
to help with using a generator in other situations, traits are provided and implemented for the base types (more can be added later if necessary/desired).
IdGeneratordescribes the basic layout of an id generator. requiring an Error, Id, and Output type to be specified along with the next_id method.IdGeneratorMutis similar toIdGeneratorexcept the next_id call allows for mutating the objectNextAvailIddescribes an object that is capable of returing aduraitonto the next available millisecond.
current use case would be for allowing different types of waiting for the
next available id. see blocking_next_id
for example implementation.
Snowflake Bits
the format of the snowflake is as follows:
01111111111111111111111111111111111111111111 - 11111111 - 111111111111
| | | | | |
64 21 20 13 12 1
timestamp | |
primary id |
sequence
the bit values for each segment can be specified by the user if you need/want certain ranges. there is currently no check to ensure that the values specified equal 63 bits (at least none that I can thing of).
Timestamp
the timestamp is in millisconds and is based from a specific start date
that you can specify. the start date must be in the future of
UNIX_EPOCH and cannot be a date in
the future, now >= start_time >= UNIX_EPOCH. internally, a snowcloud will
use SystemTime to get the timestamp and the
convert to the necessary values.
// the current example date is 2023/03/23 9:00:00.
const VALID_START_DATE: u64 = 1679587200000;
// if a date that is after the current system time is provided the
// snowcloud will return an error. 2077/10/23 9:00:00
const INVALID_START_DATE: u64 = 3402205200000;below is a table with various bit values and how many years you can get out of a timestamp. you will probably get diminishing returns with lower bit values if this is to be used over a long duration of time.
| bits | max value | years |
|---|---|---|
| 43 | 8796093022207 | 278 |
| 42 | 4398046511103 | 139 |
| 41 | 2199023255551 | 69 |
| 40 | 1099511627775 | 34 |
| 39 | 549755813887 | 17 |
| 38 | 274877906943 | 8 |
| 37 | 137438953471 | 4 |
| 36 | 68719476735 | 2 |
| 35 | 34359738367 | 1 |
Primary Id
the primary id is static to the instance of a generator. if using a generator across multiple machines then it could be the id of the machine. this would allow for those machines to create unique snowflakes that will not conflict with each other as long as the primary id is different between them.
Sequence
the sequence number is used to indicate how many records can be generated within a single millisecond. when the max sequence value is reached in a generator, the user will have to wait for the next available millisecond to get a new snowflake. the higher the bit value the more records that can be created each millisecond. the sequence number will always start with 1 when creating a generator.
De/Serialize
snowflakes support serde Serialize and
Deserialize to i64 with
an addtional option to de/serailize to a string with
i64_string_id
Modules
- base traits for implementing helper code
- methods for waiting on the next available id from a snowcloud
Structs
- thread safe snowcloud
- non-thread safe generator
- id generated from a Snowcloud
Enums
- possible errors for Snowclouds/Snowflakes