Expand description
comex strips the comments out of source code, or strips everything but the comments, without being told what language it is looking at.
let src = "int x = 1; // the answer\n/* gone */\nint y = 2;\n";
assert_eq!(comex::strip_comments(src), "int x = 1;\nint y = 2;\n");
assert_eq!(comex::extract_comments(src), "the answer\ngone\n");§How it works
Languages are grouped by comment syntax, not by name: // + /* */ covers C,
C++, C#, Java, JavaScript, TypeScript, Go, Rust, Swift, Kotlin and PHP; # covers
Python, Ruby, shell, Perl, YAML, TOML and Make. detect scores those families
against structural evidence in the text — line-leading markers, balanced block
delimiters, shebangs, preprocessor lines, how lines end — and picks a winner, or
Dialect::Unknown when the evidence is too thin. scan then makes one linear
pass with a hand-written state machine: no regex, no backtracking, and a lookup
table so uninteresting bytes cost one probe each.
String literals are tracked, which is the whole difficulty: "https://example.com"
contains // and "#hashtag" contains #, and neither is a comment. Raw strings,
character literals, Rust lifetimes, doubled-quote escapes and Lua long brackets are
all handled, and an unterminated quote is confined to its own line so that one odd
apostrophe cannot swallow a file.
§Safety bias
Ambiguity resolves toward leaving the input alone. Deleting a real line of code is
much worse than missing a comment, so weak evidence yields Dialect::Unknown and
an untouched file. When you do know the language, skip detection entirely:
use comex::{Dialect, Options};
let out = comex::strip_comments_with("x = 1 # hi\n", &Dialect::Hash.syntax(), &Options::default());
assert_eq!(out, "x = 1\n");Structs§
- Block
Rule - A comment delimited by an opening and closing marker, possibly spanning lines.
- Detection
- The outcome of
detect. - Line
Rule - A comment that runs from a marker to the end of the line.
- Options
- Output settings for stripping and extraction.
- Scan
Config - Knobs for
scan. - Span
- A classified byte range of the input.
- String
Rule - A string or character literal. Strings are not output, they are protected:
a
#inside"..."must not be mistaken for a comment. - Syntax
- The lexical rules used to scan a file.
Enums§
Constants§
- THRESHOLD
- Minimum winning score. Below this the input is treated as having no comment syntax.
Functions§
- comments
- The comment spans of
src, detecting the syntax from its content. - comments_
with - The comment spans of
srcunder an explicit syntax. - detect
- Identify the comment syntax of
srcfrom its content alone. - extract_
comments - Keep only the comments in
src, detecting the syntax from its content. - extract_
comments_ with - Keep only the comments in
srcusing an explicit syntax. - scan
- Classify
srcundersyn, returning spans in source order and without overlap. - strip_
comments - Remove every comment from
src, detecting the syntax from its content. - strip_
comments_ with - Remove every comment from
srcusing an explicit syntax.