decomment

Remove comments from source text. One function, thirteen languages, no dependencies.

The scanner is a single pass over the bytes with an explicit state for code, line comment, block comment, each string form, and regex literal. A comment marker inside a string, a template, or a regex literal is not a comment, so it is left alone. The output is always a byte subsequence of the input.

[dependencies]
decomment = "0.2.0"

Use

use decomment::{strip, Language, Options};

let source = "let url = \"http://example.com\"; // not the url\n";
let result = strip(source, Options::new(Language::Rust));
assert_eq!(result.output, "let url = \"http://example.com\"; \n");
assert_eq!(result.unterminated, None);

strip cannot fail and cannot panic. Every byte sequence has a defined stripped form, so the return value carries the output and a diagnostic rather than a Result.

Languages

TypeScript is a deliberate alias of JavaScript. A test asserts the two produce identical output on every vector in the suite.

Block comment nesting is a per-language fact rather than a setting.

use decomment::{strip, Language, Options};

let source = "/* a /* b */ c */ x";
assert_eq!(strip(source, Options::new(Language::Rust)).output, " x");
assert_eq!(strip(source, Options::new(Language::C)).output, " c */ x");

Options

Mode picks what to remove. All takes line and block comments, Line and Block take one form each, and First takes the first comment the other settings would remove and copies the rest of the input through unchanged.

Line and Block still run the full scanner, and neither scans the text of the other form. A // inside a block comment is not a line comment.

use decomment::{strip, Language, Mode, Options};

let mut options = Options::new(Language::JavaScript);
options.mode = Mode::Line;
let source = "/* see http://example.com */ x";
assert_eq!(strip(source, options).output, source);

preserve_newlines writes back the line terminators that sat inside removed text, byte for byte, so CRLF stays CRLF.

use decomment::{strip, Language, Options};

let mut options = Options::new(Language::JavaScript);
options.preserve_newlines = true;
assert_eq!(strip("a /* one\r\ntwo */ b", options).output, "a \r\n b");

keep_bang_comments keeps a comment whose body starts with !. The body starts at the byte after the opening marker, so //!, /*!, <!--!, --!, and #! qualify and /// does not. That keeps minifier license banners and keeps a shebang line.

use decomment::{strip, Language, Options};

let mut options = Options::new(Language::Python);
options.keep_bang_comments = true;
let source = "#!/usr/bin/env python3\nx = 1  # note\n";
assert_eq!(strip(source, options).output, "#!/usr/bin/env python3\nx = 1  \n");

Every option composes with every other option. No flag turns another off.

Truncated input

A chunker cuts files at arbitrary boundaries, so a chunk that ends inside a comment is routine. An unterminated block comment is deleted to end of input, which is what a compiler does with one, and the deletion is reported.

use decomment::{strip, Language, Options, UnterminatedKind};

let result = strip("x = 1; /* cut\ny = 2;", Options::new(Language::JavaScript));
assert_eq!(result.output, "x = 1; ");
let open = result.unterminated.unwrap();
assert_eq!(open.kind, UnterminatedKind::BlockComment);
assert_eq!(open.offset, 7);

Unterminated implements core::error::Error, so a caller that treats a truncated chunk as a failure can use ? after ok_or.

Which string forms survive a line terminator is a per-language fact, and it decides what one stray quote costs.

A " or ' string closes at the line terminator in C, C++, C#, CSS, Go, Java, JavaScript, Python, Swift, and TypeScript. Scanning continues in code state, so a stray quote there cannot disable comment removal for the rest of a file, and nothing is reported, because the construct is not open where the scan ends.

Every multi-line form stays open instead, because that is what its grammar says. A stray Rust ", Go backtick, SQL ' or ", triple quote, raw r#", R"d(, #", or verbatim @" runs to end of input. Everything after it is string data, comment removal stops there, and the case is reported as StringLiteral.

use decomment::{strip, Language, Options, UnterminatedKind};

let source = "a \"stray\n// gone\n";
assert_eq!(strip(source, Options::new(Language::C)).output, "a \"stray\n\n");

let source = "a \"stray\n// kept\n";
let result = strip(source, Options::new(Language::Rust));
assert_eq!(result.output, source);
assert_eq!(result.unterminated.unwrap().kind, UnterminatedKind::StringLiteral);

Limits

• Comments inside an embedded HTML <script> or <style> element are left alone. Extract the element and call again with JavaScript or Css.
• Telling a regex literal from a division needs the preceding token. ) and } are treated as ending an expression, so if (x) /re/.test(y) and a regex right after a block close are read as division. Both fail in the direction that keeps code.
• Shell, Ruby, PHP, Perl, YAML, and Lua are not in the table. Heredocs, string interpolation, and embedded sub-languages need more than a byte scanner, and a half lexer for them would damage heredoc bodies.
• Stripping is not idempotent. Removing a block comment can put a / next to a * that was not next to it before.

use decomment::{strip, Language, Options};

let once = strip("//* c */* b", Options::new(Language::Css)).output;
assert_eq!(once, "/* b");
assert_eq!(strip(&once, Options::new(Language::Css)).output, "");

Requirements

#![no_std] with extern crate alloc. An allocator is required, because the output is a String. No cargo features, no dependencies, and one allocation per call.

MSRV is 1.81.0, the release that stabilized core::error::Error. A CI job builds the library and runs the doctests on that exact release, so the floor is tested and not asserted. The full test suite runs on stable.

License

Licensed under either of Apache License, Version 2.0 or the MIT license at your option.

Unless you state otherwise, any contribution you intentionally submit for inclusion in this crate, as defined in the Apache-2.0 license, is dual licensed as above, with no additional terms or conditions.