Module engine 

The Fast-mode clone-detection engine.

Input is the lexed token stream and unit boundaries of every file in scope; output is a set of clone groups with per-group noise signals and run statistics. Two passes run over the input:

• a raw pass that finds Type-1 (verbatim) clones anywhere — winnowed k-gram fingerprints seed candidates, each seed is verified token-by-token and extended to a maximal run bounded by function boundaries;
• a fragment pass that finds Type-2 (consistently renamed) clones — candidate fragments are normalized scope-locally and matched whole, so a renamed statement run transplanted into an unrelated host function still matches its origin.

The engine never executes the code it reads, uses no randomness, and sorts every output deterministically: the same input produces the same report, token by token. Candidate-explosion controls (posting caps, a global pair budget, rarest-first pairing) act before the quadratic pairing step, and everything they drop is counted in EngineStats rather than vanishing.

Re-exports

pub use normalize::LiteralNorm;

Modules

fingerprint

Deterministic content hashing: token hashes, rolling k-grams, winnowing.

normalize

Scope-local token normalization for Type-2 clone matching.

Structs

CloneGroup

A set of instances sharing identical matched content.

ClonePair

A verified match between two instances of the same content.

EngineConfig

Engine tuning. The defaults are the evaluated configuration.

EngineReport

The engine’s output: clone groups plus run statistics.

EngineStats

Counters describing what a detection run saw and dropped.

InputFile

One lexed file, as the engine consumes it.

Instance

One occurrence of matched content.

Enums

SuppressReason

Why a group was marked suppressed.

Constants

ENGINE_VERSION

Version of Fast-mode detection and cross-pass consolidation rules.

Functions

content_entropy_bits

Shannon entropy, in bits, of a token slice’s normalized-token distribution.

detect

Detect clones across files.

detect_with_arm_paths

Detect clones while excluding pairs separated by C-family preprocessor arms.

entropy_ratio

Entropy as a share of the largest value a token sequence of this length could have.

group_pairs

Group clone pairs into clone groups by matched content.