Skip to main content

Crate cpd_semantic

Crate cpd_semantic 

Source
Expand description

Semantic clones for jscpd (--semantic, experimental): functions that do the same job written differently — renamed, restructured, or in another language, like a rule a Rust backend enforces and a Svelte frontend repeats.

  • extract and units find the functions of a source and the text a model sees for each;
  • embed turns those texts into vectors: a model run in-process (Provider::Local) or an OpenAI-compatible API (Provider::Http), behind an on-disk cache; embed::catalog holds the thresholds of the models jscpd has calibrated;
  • search pairs the functions whose vectors point the same way;
  • SemanticPass runs all of it as a clone pass of the finder (cpd_finder::pass);
  • compare pairs the functions of two codebases instead (--compare): a project and its port, or the iOS and the Android version of an app.

Everything --semantic needs lives here, with every dependency it brings — the model runtime, the model’s tokenizer, the tree-sitter grammars, the HTTP client — so the core, the tokenizer and the finder compile without them.

Re-exports§

pub use embed::API_KEY_ENV;
pub use embed::DEFAULT_HTTP_MODEL;
pub use embed::DEFAULT_LOCAL_MODEL;
pub use embed::DEFAULT_URL;
pub use embed::Provider;
pub use embed::SemanticOptions;
pub use embed::download;
pub use embed::embedder;
pub use embed::missing_model;
pub use embed::model_list;
pub use search::Embedder;
pub use search::SemanticScope;
pub use search::Thresholds;

Modules§

compare
Two codebases compared function by function (--compare): a project and its port to another language, or two implementations of one app, such as the iOS and the Android one. Every function of one side is paired with the functions of the other side that do the same job; what stays unpaired on a side is what the other side lacks.
embed
extract
Functions for --semantic.
search
Semantic clones (Type-4, --semantic, experimental).
test_code
Which functions are tests, so that --compare measures tests and code apart and pairs a test only with a test.
units
Functions to embed for semantic clones (--semantic).

Structs§

SemanticPass
Finds semantic clones among the functions of the files the finder shows it: reads each file’s functions while the finder holds it, then embeds and pairs them once the token passes are done.
UnitReader
Collects the functions of the files the finder shows it, one UnitSource per detection source. SemanticPass reads files through it; --compare uses it alone, with no clone detection.