Struct MinHashStreaming 

pub struct MinHashStreaming<T: Tokenizer, const H: usize> { /* private fields */ }

Available on crate feature minhash only.

Buffered streaming sketcher.

Wraps a MinHashFingerprinter and accumulates UTF-8 bytes across update calls. finalize runs the offline algorithm on the accumulated buffer.

Implementations

impl<T: Tokenizer, const H: usize> MinHashStreaming<T, H>

pub fn new(inner: MinHashFingerprinter<T, H>) -> Self

Construct a streamer wrapping inner.

Buffer cap defaults to DEFAULT_MAX_BUFFER_BYTES (16 MiB).

Arguments

• inner — the offline MinHashFingerprinter whose canonicalizer + tokenizer + hash configuration the streamer inherits.

Example

use txtfp::{
    Canonicalizer, MinHashFingerprinter, MinHashStreaming,
    ShingleTokenizer, WordTokenizer,
};

let s = MinHashStreaming::<_, 64>::new(MinHashFingerprinter::new(
    Canonicalizer::default(),
    ShingleTokenizer { k: 3, inner: WordTokenizer },
));
assert_eq!(s.buffered_bytes(), 0);

pub fn with_max_bytes(self, max_bytes: usize) -> Self

Override the buffer cap.

Useful for tests or constrained environments where 16 MiB is too generous. Setting the cap below the document size causes the next update call to return crate::Error::InvalidInput.

Arguments

• max_bytes — maximum total bytes the streamer is willing to accumulate.

pub fn buffered_bytes(&self) -> usize

Total bytes accumulated so far (excluding the unfinished multi-byte UTF-8 carry).

Returns

The size of the validated UTF-8 prefix. The streamer may also hold a few additional bytes in a transient carry buffer when an update arrives mid-codepoint; those are not counted here.

Trait Implementations

impl<T: Tokenizer, const H: usize> StreamingFingerprinter for MinHashStreaming<T, H>

type Output = MinHashSig<H>

Available on crate features minhash or simhash or lsh or tlsh only.

The fingerprint produced at end-of-stream.

fn update(&mut self, chunk: &[u8]) -> Result<()>

Available on crate features minhash or simhash or lsh or tlsh only.

Append chunk to the internal buffer.

fn finalize(self) -> Result<Self::Output>

Available on crate features minhash or simhash or lsh or tlsh only.

Finalize the running state and produce the fingerprint. Consumes the streamer.

fn reset(&mut self)

Available on crate features minhash or simhash or lsh or tlsh only.

Drop the buffer so the same streamer can be reused without reallocating.