Skip to main content

Crate cpd_similarity

Crate cpd_similarity 

Source
Expand description

Structural similarity of functions (--similarity).

Every function and method is parsed and its syntax tree normalized: the names of the functions and methods it calls stay, and so do its operators, while local names, field names and literals become markers and comments, punctuation and parentheses around one expression drop out. Each subtree of the normalized tree is a fingerprint ([prints]), and two units score the Jaccard index of their fingerprint sets: shared over all. A renamed copy scores 1, a copy with an edited statement loses the fingerprints of that statement and of the subtrees around it.

A name has to stand for one name on the other side (issue #1217): the fingerprint of a subtree tells how its names repeat, so x = x + x * x loses the fingerprints d = a + b * c has, and a copy that swaps two names somewhere loses those of the subtrees that hold both places. With --ignore-identifiers=blind any name matches any other, as in 5.4.

The units of each language ([syntax], [clojure]):

  • Python: functions and methods; a function in a function is part of it, a method of a class declared in a function is a unit of its own;
  • JavaScript and TypeScript: functions and methods with a body outside every function, and the variables and fields a function is assigned to;
  • Java: methods with a body, not those of classes declared in a method;
  • Kotlin, Scala, C#, C, C++, PHP, Ruby and Swift: functions and methods with a body outside every function, constructors left out;
  • Go: functions and methods with a body;
  • Rust: functions with a body, not those in a function or a mod tests;
  • Clojure: every top-level form except ns;

and the same in the code blocks of Markdown files and the scripts of Vue, Svelte and Astro components. Units pair within one language, JavaScript and TypeScript being one, and C and C++ another. Test files are left out (reads).

The search is exact ([join]): every pair whose score reaches the threshold is found.

Modules§

test_files
Which files hold tests, by the naming conventions of the languages jscpd compares: --similarity leaves them out, and --compare and --semantic measure tests and code apart.

Structs§

Coverage
What the clones found so far report, indexed by source. A pair of functions is covered when one clone spans both, or when two clones copy one fragment into each of them: detection pairs every copy of a fragment with its first copy only, so with three copies the pair of the second and the third is implied and is not reported again as similar.
Form
A unit: a function, a method, a Clojure form.
FormSource
The units of one source, as the search reads them.
SimilarPairs
What a search finds: the pairs to report as clones and, when the search is asked for them, every pair.
SimilarityIndex
The units of many sources, to pair among themselves or to search for a source outside them, such as a snippet. Owns its sources so a long-lived holder (the MCP server) can build it once per scan.

Enums§

Names
How --similarity compares the names that become :symbol.

Constants§

DEFAULT_MIN_NODES
The fewest normalized nodes a unit has to have to be compared (--min-nodes): smaller ones match too easily.
DEFAULT_THRESHOLD
The threshold --similarity compares at when it is given without one.

Functions§

declared_name
Shared with --semantic, which names C and C++ functions the same way. The name a C or C++ declarator wraps: f in *f(void), (&f)(int) or shop::Cart::f(). None for a declarator that names nothing, such as a lambda’s parameter list.
discount_token_lines
Leave out of the lines of each --similarity pair in clones the lines that the clones of the token passes there already hold in its files, so a line counts once. Statistics and the console read what remains (see CpdClone::matched_lines). Run it on the clones a report shows, after --kind drops any.
embedded_forms
The units of the code source, a file of host_format, embeds, each with the format of its block: the code blocks of a Markdown file, the scripts and frontmatter of a component. Positions are the host’s, so a unit reports where it sits in the .md or .vue file. ignored are byte ranges of the host; names as in forms.
embeds
Formats whose files embed code with units in it: Markdown with its code blocks, and Vue, Svelte and Astro components with their scripts.
find_similar
The pairs of units in sources whose score reaches threshold, of at least min_nodes nodes and min_lines lines each (see SimilarityIndex::pairs).
forms
The units of source, a file of format, without the code in its jscpd:ignore blocks and in the byte ranges ignored (the matches of --ignore-pattern): a unit that lies in such code is none, and such code adds nothing to the units around it. Their names compare by names.
jaccard
The Jaccard index of two sorted sets: shared over all.
reads
Whether --similarity reads the file at path, of format: a file of a format with units or one that embeds them, not a test file. Of the Python formats it reads .py, of the Clojure ones the source files, not EDN data, and no TypeScript declaration file.
reads_with_tests
reads, test files included: the boundary rules of a config file count a copy in a test as a copy.
supported_formats
The formats whose files hold units, for messages and docs.
supports
Whether files of format hold units.
token_range
The detection tokens lying inside the byte range start..end of a source, as a half-open index range into its spans; None when there are none.