pub struct DedupKey(/* private fields */);Expand description
An idempotent-append deduplication key: a 64-byte SHA3-512 digest that is
a deterministic function of (namespace, payload).
An ingest pipeline keys in-flight and recently-committed records by their
DedupKey; a re-submission (network retry, at-least-once delivery, a client
replaying a request) produces the same key and can be dropped before it
double-appends. The key is post-quantum (SHA3-512), domain-separated, and
namespace-scoped, so keys never collide across namespaces or across the two
derivation modes.
Two derivations are offered:
DedupKey::from_recordkeys on the content itself (the canonical leaf bytes). Identical content in the same namespace is the same record.DedupKey::from_tokenkeys on a client-supplied idempotency token (e.g. a request UUID). Use this when the same logical submission may carry non-identical bytes (timestamps, nonces) yet must append at most once.
The key is opaque; compare keys for equality or use DedupKey::as_bytes /
DedupKey::to_hex as a storage index.
Implementations§
Source§impl DedupKey
impl DedupKey
Sourcepub fn from_record(namespace: &Namespace, payload: &[u8]) -> Self
pub fn from_record(namespace: &Namespace, payload: &[u8]) -> Self
Derive a content-addressed dedup key from a namespace and the canonical
record bytes: SHA3-512_ctx("…/content/v1", lp(ns) || lp(payload)).
Sourcepub fn from_token(namespace: &Namespace, token: &[u8]) -> Self
pub fn from_token(namespace: &Namespace, token: &[u8]) -> Self
Derive a dedup key from a namespace and a client-supplied idempotency
token: SHA3-512_ctx("…/token/v1", lp(ns) || lp(token)).