cargo docgen - a Rust Documentation Test Helper
Rationale
Documentation tests are an essential part of releasing good Rust crates on crates.io. To quote the first edition of the Rust book:
Nothing is better than documentation with examples. Nothing is worse than examples that don't actually work
Every item (module, function, method, etc) should have an example which both compiles and runs as a test.
However, if you mosey to that most excellent site docs.rs and browse some of
the 11,000-odd crates, you will see that many don't even try to provide any documentation, which
is disappointimg and leaves you with the irritating necessity of actually reading the source.
Part of this is just human nature, or at least the nature of programmers who find it difficult
to switch from code to English, but much of it is that good documentation is hard work.
Not only is formatting doc tests tiresome, but running cargo test to run all the tests
can take a fair amount of time even for small projects.
The Guidelines for
documentation are very comprehensive and fairly demanding. cargo docgen aims to make preparing
working tests and embedding them in your source easier.
A Simple Example
Say you wish to publish your great work, the crate life. You wish to document the function
life::answer. Write a little code snippet like so in some subdirectory of the life project
(I personally create a scratch dir and put it in .gitignore)
// answer.rs
let a = answer;
assert_eq!;
And run cargo docgen:
$ cargo docgen answer.rs
****** Copy and paste this output into your code ******
/// ```
/// let a = life::answer();
/// assert_eq!(a, 42);
/// ```
It will run this snippet using cargo run --example and comment the result appropriately.
You can type the doc test in a real editor, run it immediately, and
have something that can be pasted directly into your code. (I don't know about other people, but
I like typing Rust in a code-aware editor, and I do not like waiting
to find out if I have inevitable mistakes.)
This comment is suitable for any code item which is not module-level. If I said cargo docgen -m answer.rs,
the result is formatted for a module-level example:
//! ```
//! let a = life::answer();
//! assert_eq!(a, 42);
//! ```
You can indent the result using --indent. I tend to say -i4 because I like spaces, but -i1t will
indent by one tab, and so forth. (Mixing spaces and tabs is an Abomination.)
Support for the Question Mark Operator
Consider this snippet which I wrote to test lua-patterns.
let mut pat = new_try.unwrap;
assert!;
It's common to see unwrap in little examples, and it is both nasty and misleading, because
in well-written code, it hardly appears. In real life, we use the question mark operator for
error handling. As the Guidelines say: "Like it or not, example code is often copied verbatim
by users. Unwrapping an error should be a conscious decision that the user needs to make."
This is the purpose of the --question flag (-q for short.)
So you should write:
let mut pat = new_try?;
assert!;
And cargo docgen -q -i4 new_try.rs will generate the following code:
/// ```
/// # use std::error::Error;
/// #
/// # fn run() -> Result<(),Box<Error>> {
/// let mut pat = lua_patterns::LuaPattern::new_try("^%s*$")?;
/// assert!(pat.matches(" "));
/// # Ok(())
/// # }
/// #
/// # fn main() {
/// # run().unwrap();
/// # }
/// ```
This is the recommended way to present code where
errors may occur, and it's a lot of boilerplate. The doc test syntax allows for
lines to be hidden using #, so only the actual snippet lines will appear in the rendered
documentation.
(Here we're using the convenient fact that any Error type will convert into a Box<Error>)
Compiling and running this snippet took 1.2s - cargo test for the whole project took 14.7s in clock time!
And it would take far longer, and be more painful, to enter the full commented code directly
into the library source.
Examples which are Not Tests
A doc test (like any other Rust test) consists of a set of assertions. You may use
println! but the test runner will swallow this output. cargo docgen will print out
the output, but will issue a warning.
Some examples should be compiled, but not run. Here we process an example of obviously bad test code from The Book, First Edition:
$ cat loop.rs
loop
$ cargo docgen -n loop.rs
****** Copy and paste this into your code ******
/// ```rust,no_run
/// loop {
/// println!("hello, world");
/// }
/// ```
There is a further variation - when you are creating your tests, you might like to run them in your own environment, but want them not to appear as running tests. (as The Book says, documentation tests should not try to download from the internet. But you would would at least like to try.)
// read-a-file.rs
use *;
use File;
let mut f = open?;
let mut s = Stringnew;
f.read_to_string?;
println!;
We need -q for the question-operator, -n for marking as not-run,
and -r for making it run locally:
$ cargo docgen -nrq read-a-file.rs
****** tests will ignore this output ****
got use std::io::prelude::*;
use std::fs::File;
let mut f = File::open("read-a-file.rs")?;
let mut s = String::new();
f.read_to_string(&mut s)?;
println!("got {}",s);
******
****** Copy and paste this into your code ******
/// ```rust,no_run
/// # use std::error::Error;
/// #
/// # fn run() -> Result<(),Box<Error>> {
/// use std::io::prelude::*;
/// use std::fs::File;
///
/// let mut f = File::open("read-a-file.rs")?;
/// let mut s = String::new();
/// f.read_to_string(&mut s)?;
/// println!("got {}",s);
/// # Ok(())
/// # }
/// #
/// # fn main() {
/// # run().unwrap();
/// # }
/// ```
Getting the Output into your Editor
cargo docgen prints out any actual output as a warning. Only the
snippet to be embedded goes to stdout. How you actually copy to
clipboard is your responsibility - I considered the 'clipboard' crate
but it cannot help us on Linux, where the clipboard only lives as long
as the program that creates it. So on Linux, I would recommend xclip:
$ cargo docgen -nrq read-a-file.rs | xclip -i -selection clipboard
****** tests will ignore this output ****
got use std::io::prelude::*;
use std::fs::File;
let mut f = File::open("read-a-file.rs")?;
let mut s = String::new();
f.read_to_string(&mut s)?;
println!("got {}",s);
******
****** Copy and paste this into your code ******
And thereafter things work as expected - the doc test can now be pasted into your editor.
(There's a command-line utility called clip on Windows which does
exactly this as well; the MacOS equivalent is pbcopy)
Testing and Formatting Markdown
The --module-doc (-M) flag lets you process a whole Markdown file containing little doc test
snippets. Here is a silly example:
This should be any text whatsoever which can be edited safely. Snippets are only run if they change:
use lua_patterns::*; let mut pat = LuaPattern::new_try("^%s*$")?; assert!(pat.matches(" ")); assert!(! pat.matches(" x "));and the text continues.
This shows how by default matches are 'unanchored':
let mut pat = new; assert!; assert!;And another:
for i in 0..4
This is almost the Github-flavoured Markdown that we know and love, with one little change.
If a doc test uses the question-operator, cargo codegen needs to know so it can
generate the necessary boilerplate. Since reliably detecting ? in source is tricky
(it could be in a comment, or in a string) I've opted for an explicit approach, where
in the usual guard after the backticks "rust" becomes "rust?".
Running cargo docgen -M doc.md gives, after running each snippet as a test:
//! This should be any text
//! whatsoever which can be edited safely. Snippets are only
//! run if they change:
//!
//! ```
//! # use std::error::Error;
//! #
//! # fn run() -> Result<(),Box<Error>> {
//! use lua_patterns::*;
//! let mut pat = LuaPattern::new_try("^%s*$")?;
//! assert!(pat.matches(" "));
//! assert!(! pat.matches(" x "));
//! # Ok(())
//! # }
//! #
//! # fn main() {
//! # run().unwrap();
//! # }
//! ```
//! and the text continues.
//!
//! This shows how by default matches are 'unanchored':
//!
//! ```
//! let mut pat = lua_patterns::LuaPattern::new("boo");
//! assert!( pat.matches("boo") );
//! assert!( pat.matches(" boo ") );
//! ```
//!
//! And another:
//! ```
//! for i in 0..4 {
//! println!("gotcha! {}",i);
//! }
//! ```
//!
Furthermore, these code snippets are cached (look in 'doc.md.cache' afterwards) and subsequent runs will only re-run those doc tests which have in fact changed.
Good Rust document tests are hard to type, and I hope this utility makes it easier
for other lazy people to write better, functional documentation for their crates.
I used an early version of this tool to help me generate the documentation
for lua-patterns and it saved me a lot of irritating busy-work.
To install, just use cargo install cargo-docgen.