rsticle - Generates articles from source code

Sometimes you want to show others a “worked example” of how to do something, like using a library or implementing an algorithm. With rsticle, you add special comments into your source file that will be interpreted as narrative text, which then get transformed into a markup document.

So you’ll go from something like this:

//: # A basic Rust Example
//:
//: This file will showcase the following function:
//:
//> ____
pub fn strlen<'a>(s: &'a str) -> usize {
    s.len()
}
//:
//{
#[test]
fn test_strlen() {
    //}
    //: It works as expected:
    //:
    assert_eq!(strlen("Hello world!"), 12);
} //

to this:

# A basic Rust Example

This file will showcase the following function:

    pub fn strlen<'a>(s: &'a str) -> usize {
        s.len()
    }

It works as expected:

    assert_eq!(strlen("Hello world!"), 12);

These are the default comment markers:

• //: introduce narrative comments. These lines will become the document text (by simply stripping the //: part). Everything that doesn’t start with such a comment will be be output verbatim (though perhaps indented; see below).
• // at the end of a line will exclude that line from the output.
• //{ and //} enclose blocks that will be excluded. This is so you don’t have to postfix every line with //, but also because some code formatters will sometimes rearrange trailing comments.
• //> to indent all following verbatim output by this many spaces. So //> ___, //> ..., and //> abc would all result in an indentation by 3 spaces. The default indentation is zero, and a plain //> resets the indentation to zero.

These can be configured, so they’ll work with any programming language that has line comments. There is also nothing special about the use of Markdown in the example above. You can use any markup language at all; rsticle doesn’t actually do a lot besides processing these comment markers.

This repository includes

• A tiny Rust library (no dependencies)
• A rust macro for including examples files with rustdoc
• A command line tool

Installation

To use the library:

cargo add rsticle

To use the macro with rustdoc

cargo add rsticle-rustdoc

To install the command line tool:

cargo install rsticle

The binaries should also be found on the Releases page.

Usage

Macro

Let’s say you have the above code in examples/basic.rs, and want to showcase it it your API-docs.

Add the following to your lib.rs

//! Highly advanced string length calculation
//!
//! Get a load of this:
#![doc = rsticle_rustdoc::include_as_doc!("examples/basic.rs")]

Running cargo doc will now include the processed file in the docs.

Library

Simple way of converting from a string:

let source = r#"\
//: Look at this:
//:
//: ```rust
fn some_func() -> String {
    String::new("Hi!")
}
//: ```rust
"#;

let profile = rsticle::SLASH; // The default profile for "slashy" languages
let doc = rsticle::convert_str(&profile, source).unwrap();

assert_eq!(doc, r#"\
Look at this:

```rust
fn some_func() -> String {
    String::new("Hi!")
}
```rust
"#)

See the API docs for more.

Command line

Let’s say you have the above code in /home/me/rust_code/basic.rs. You’re super proud of it and want to publish it to your blog. Running:

rsticle basic.rs > basic.md

will give you the Markdown file. Upload it to your blog and watch the job offers pour in.

It supports a couple of other file / comment types out of the box:

• Rust/C/Java/… (//)
• Python/Shell/… (#)
• Haskell/Lua/… (--)

It tries to guess the comment type from the file name, so this should work:

rsticle basic.py > basic.rst

If your language is unsupported, you can tell rsticle about its line comments it like so

rsticle basic.bas --profile "REM" > basic.txt

which lets you use REM:, REM{, REM} and REM> comments (plus trailing REM to ignore the line.)

If you need even more customization, you can give a whitespace separated list. So for Brainfuck (where pretty much anything is a comment), you could say:

rsticle basic.bf --profile "blah not! shutup listen moveit" > basic.txt

This shows that the suffixes after the line comment can be anything. If you prefer, you could prepare your Rust docs with more descriptive comments like these:

rsticle basic.rs --profile "//narrate //ignore_line //start_ignoring //stop_ignoring //indent_examples" > basic.md

Why would I use this instead of …

… Literate Programming?

Literate Programming usually starts from the document and produces source code/binaries:

graph LR
    lit[Literate Markdown]
    src[Program Source]
    bin[Program Binary]
    art[Narrative Article]

    lit --> art
    lit --> src --> bin

rsticle starts from the actual source code:

graph LR
    src[Program Source]
    mkp[Markup]
    bin[Program Binary]
    art[Narrative Article]

    src --> mkp --> art
    src --> bin

This is sometimes called “Reverse Literate Programming”. But most tools, no matter the direction, are fairly complicated. This is because they support re-arranging the input in some way.

rsticle is much less ambitious. You write your source code, rsticle goes through it line-by-line, and you have your output.

If you just want to walk someone through a file, and nothing more, then you may find rsticle fits the bill.

… API Documentation?

API documenation is meant to document individual items (functions, data types, …), but not so much for giving examples of workflows or trains of thought.

Most notably, API docs will rearrange the individual doc items to achieve a uniform structure, while narrative documentation goes linearly from basic concepts to more elaborate.

graph LR
    src[Program Source]
    bin[Program Binary]
    doc[Library Documentation]

    src --> doc
    src --> bin

rsticle is certainly meant to augment API docs, however. In fact, the whole reason it exists is because I wanted to include example code in the published documentation.

… Plain Markdown/Asciidoc/etc?

Because your editor likely doesn’t support compiling, checking or formatting source code inside these files.

This won’t be so bad for small, self-contained examples, but anything more elaborate can get cumbersome, especially when the examples build on each other.

In that case you may prefer writing the source code as you would normally, and have the tutorial text right next to what it explains.

… similar crates?

• doctest-file is pretty much the direct inspiration for rsticle; the way you hide lines is exactly the same in both libraries. doctest-file works by including one file and treat that as exactly one doctest. In contrast, rsticle can result in multiple code blocks, interspersed with narrative text, that can build on each other. Code blocks created by rsticle are not treated as tests. (Though they can certainly come from tests, which is often the case and thus achieves the same result as a doctest, namely showing off working code)

  • I should also note that I also have a my own fork of doctest-file, which adds the ability to only snip parts out of existing files, and create multiple doctests from those snips, or assemble doctests from several snips (for instance to share setup code between doctests)

• include-doc is at once more sophisticated and more restrictive. It requires the example code to be Rust, whereas rsticle can use anything that has // comments.

… extracted examples (like mdBook)

Tools like mdBook allow writing documentation and source code separately, and then include (parts of) the source within the document.

This is probably preferred for more elaborate cases, where both the markup and the source code are relatively complex.

But mdBook needs more setup, so it might be a bit heavy for single-file, one-off examples. There is also the danger of drift: The source code being shown could change in some important way, but the surrounding text isn’t updated to reflect that. This is less likely when you have both directly besides each other.