sequence-generator-rust

64-bit IDs sequence generator based on the concepts outlined in Twitter Server's ID (formerly snowflake). Build on Rust

Installation & build
Description
Usage
Library
Support
Contributing

Installation

  git clone https://github.com/drconopoima/sequence-generator-rust.git
  cd sequence-generator-rust
  cargo build --release

The binary was generated under target/release/sequence-generator-rust

Description

You can generate sequential IDs based on timestamp, sequence number and node/worker ID (based on Twitter snowflake):

By default this package format defines:

the right-most 9 bits are used to store worker and/or host information (up to 512)
subsequently, 11 bits are used to store a sequence number (up to 2048)
the left-most, 44 bits are used to store a custom epoch with precision of 10 samples every millisecond (10^-1). That's enough to store 55 years from a custom epoch
There are no bits left unused.
Custom epoch is set to the beginning of current decade (2020-01-01)

Usage

Generate a single sequence number as follows, with a worker-id set up from .env file (default 0):

$ cargo run \
  0: 731587959438966784

Generate many sequence values (-n|--number), provide a custom worker id (--node-id), and measure the time taken (-d|--debug):

cargo run --release -- -n 8 --node-id 505 --debug

0: 731586108621586937
1: 731586108621587449
2: 731586108621587961
3: 731586108621588473
4: 731586108621588985
5: 731586108621589497
6: 731586108621590009
7: 731586108621590521
It took 661 nanoseconds

Each one of the parameters for the sequence are customizable.

By default the original Twitter snowflake format defines:

1 bit left unused (sign)
41 bits are used to store a custom epoch with millisecond precision (10^3 microseconds for 69 years from a custom epoch)
10 bits are used to store worker and datacenter information (up to 1024)
12 bits are used to store a sequence number (up to 4096)
Uses a custom epoch of 1288834974657 or Nov 04 2010 01:42:54.

You can perfectly and easily recreate Twitter's snowflakes by passing the following command arguments.

$ cargo run --release -- -n 8 -d --unused-bits 1 --node-id-bits 10 --sequence-bits 12 --micros-ten-power 3 --custom-epoch '2010-11-04T01:42:54Z'  --node-id 128
0: 137870923482005632
1: 137870923482006656
2: 137870923482007680
3: 137870923482008704
4: 137870923482009728
5: 137870923482010752
6: 137870923482011776
7: 137870923482012800
It took 571 nanoseconds

The specific structure of the integers at the binary level includes:

The left-most bits (customizable, by default none) might be unused and set to 0.
The second group of bits store the timestamp in a custom exponential by microseconds (by default 44 bits and sampling every 100 mcs, equivalent to argument --micros-ten-power 2). You cannot customize number of bits of the timestamp directly, but by indirectly setting different values for other bit groups.
The third group of bits store the sequence (by default 11 bits)
The right-most group of bits store the host/worker ID (by default 9 bits)

You can also customize by dotenv file. Copy the file .env-example into .env

cp .env-example .env

And change the example values to your liking.

The precedence of parameters assigned through the command-line launch arguments is the highest, whichever are not assigned can be retrieved by use of a .env file, and if still unassigned parameters remains, then default values described above are used.

The only supported custom epoch format is RFC-3339/ISO-8601 both as CLI argument and from the dotenv file.

Check a detailed analysis for a generated value in the auxiliar bit structure analysis

Benchmarking

See auxiliar benchmarking notes

Library

use std::time::UNIX_EPOCH;
use ::sequence_generator::*;

let custom_epoch = UNIX_EPOCH;  // SystemTime object representing custom epoch time. Use checked_add(Duration) for different time
let node_id_bits = 10;          // 10-bit node/worker ID
let sequence_bits = 12;         // 12-bit sequence
let unused_bits = 1;            // unused (sign) bits at the start of the ID. 1 or 0 generally
let micros_ten_power = 3;       // Operate in milliseconds (10^3 microseconds)
let node_id = 500;              // Current worker/node ID
let cooldown_ns = 1500;         // initial time in nanoseconds for exponential backoff wait after sequence is exhausted

// Generate SequenceProperties
let properties = sequence_generator::SequenceProperties::new(
        custom_epoch,
        node_id_bits,
        node_id,
        sequence_bits,
        micros_ten_power,
        unused_bits,
        cooldown_ns,
    );

// Generate an ID
let id = sequence_generator::generate_id(&properties).unwrap();
// Decode ID
// Timestamp
let timestamp_micros = sequence_generator::decode_id_unix_epoch_micros(id, &properties);
// Sequence
let sequence = sequence_generator::decode_sequence_id(id, &properties);
// Node ID
let id_node = sequence_generator::decode_node_id(id, &properties);

Support

Please open an issue for support.

Changelog

See changelog

Contributing

Please contribute using Github Flow. Create a branch, add commits, and open a pull request.

sequence-generator-rust 0.3.1