Expand description
Structural similarity of functions (--similarity).
Every function and method is parsed and its syntax tree normalized: the
names of the functions and methods it calls stay, and so do its
operators, while local names, field names and literals become markers and
comments, punctuation and parentheses around one expression drop out.
Each subtree of the normalized tree is a fingerprint ([prints]), and
two units score the Jaccard index of their fingerprint sets: shared over
all. A renamed copy scores 1, a copy with an edited statement loses the
fingerprints of that statement and of the subtrees around it.
A name has to stand for one name on the other side (issue #1217): the
fingerprint of a subtree tells how its names repeat, so x = x + x * x
loses the fingerprints d = a + b * c has, and a copy that swaps two
names somewhere loses those of the subtrees that hold both places. With
--ignore-identifiers=blind any name matches any other, as in 5.4.
The units of each language ([syntax], [clojure]):
- Python: functions and methods; a function in a function is part of it, a method of a class declared in a function is a unit of its own;
- JavaScript and TypeScript: functions and methods with a body outside every function, and the variables and fields a function is assigned to;
- Java: methods with a body, not those of classes declared in a method;
- Kotlin, Scala, C#, C, C++, PHP, Ruby and Swift: functions and methods with a body outside every function, constructors left out;
- Go: functions and methods with a body;
- Rust: functions with a body, not those in a function or a
mod tests; - Clojure: every top-level form except
ns;
and the same in the code blocks of Markdown files and the scripts of Vue,
Svelte and Astro components. Units pair within one language, JavaScript
and TypeScript being one, and C and C++ another. Test files are left out
(reads).
The search is exact ([join]): every pair whose score reaches the
threshold is found.
Modules§
- test_
files - Which files hold tests, by the naming conventions of the languages
jscpd compares:
--similarityleaves them out, and--compareand--semanticmeasure tests and code apart.
Structs§
- Coverage
- What the clones found so far report, indexed by source. A pair of
functions is covered when one clone spans both, or when two clones copy
one fragment into each of them: detection pairs every copy of a fragment
with its first copy only, so with three copies the pair of the second and
the third is implied and is not reported again as
similar. - Form
- A unit: a function, a method, a Clojure form.
- Form
Source - The units of one source, as the search reads them.
- Similar
Pairs - What a search finds: the pairs to report as clones and, when the search is asked for them, every pair.
- Similarity
Index - The units of many sources, to pair among themselves or to search for a source outside them, such as a snippet. Owns its sources so a long-lived holder (the MCP server) can build it once per scan.
Enums§
- Names
- How
--similaritycompares the names that become:symbol.
Constants§
- DEFAULT_
MIN_ NODES - The fewest normalized nodes a unit has to have to be compared
(
--min-nodes): smaller ones match too easily. - DEFAULT_
THRESHOLD - The threshold
--similaritycompares at when it is given without one.
Functions§
- declared_
name - Shared with
--semantic, which names C and C++ functions the same way. The name a C or C++ declarator wraps:fin*f(void),(&f)(int)orshop::Cart::f(). None for a declarator that names nothing, such as a lambda’s parameter list. - discount_
token_ lines - Leave out of the lines of each
--similaritypair inclonesthe lines that the clones of the token passes there already hold in its files, so a line counts once. Statistics and the console read what remains (seeCpdClone::matched_lines). Run it on the clones a report shows, after--kinddrops any. - embedded_
forms - The units of the code
source, a file ofhost_format, embeds, each with the format of its block: the code blocks of a Markdown file, the scripts and frontmatter of a component. Positions are the host’s, so a unit reports where it sits in the.mdor.vuefile.ignoredare byte ranges of the host;namesas informs. - embeds
- Formats whose files embed code with units in it: Markdown with its code blocks, and Vue, Svelte and Astro components with their scripts.
- find_
similar - The pairs of units in
sourceswhose score reachesthreshold, of at leastmin_nodesnodes andmin_lineslines each (seeSimilarityIndex::pairs). - forms
- The units of
source, a file offormat, without the code in itsjscpd:ignoreblocks and in the byte rangesignored(the matches of--ignore-pattern): a unit that lies in such code is none, and such code adds nothing to the units around it. Their names compare bynames. - jaccard
- The Jaccard index of two sorted sets: shared over all.
- reads
- Whether
--similarityreads the file atpath, offormat: a file of a format with units or one that embeds them, not a test file. Of the Python formats it reads.py, of the Clojure ones the source files, not EDN data, and no TypeScript declaration file. - reads_
with_ tests reads, test files included: the boundary rules of a config file count a copy in a test as a copy.- supported_
formats - The formats whose files hold units, for messages and docs.
- supports
- Whether files of
formathold units. - token_
range - The detection tokens lying inside the byte range
start..endof a source, as a half-open index range into itsspans;Nonewhen there are none.