Problemo

This library aims to improve the experience of working with Rust's std Error trait by allowing for deep causation chains, arbitrary attachments, and error accumulation with the goal of making it easy and rewarding to return richly-typed errors for callers to inspect and handle.

This page will give you a tour of Problemo's design and features. The examples will show you how to use it in practice. And don't forget the API documentation.

Specifically, we've identified three features missing from std Error:

1. Causation Chains

Firstly, we want to be able to set any error as the cause of any other error. This allows us to build causation chains that we can inspect and handle as needed.

The std Error trait does have a source() function, but the trait does not provide a way for setting the source. Indeed, it is an optional feature.

A common workaround for this limitation is to create an enum error type where each variant contains the source error. Libraries such as derive_more and thiserror help by setting up the plumbing. Unfortunately this solution requires us to define variants for all the error types that can be our source. This bookkeeping become tedious as new error types must be added when functionality grows, while old, no-longer-used error types linger on and clutter the codebase. And since the same lower-level errors crop up again and again these enums have a lot of duplication. The biggest problem is that further nesting is possible only if our sources are themselves enums, leading to a proliferation of these enums.

Problemo's solution is to introduce a wrapper type for errors, Problem, which is simply a dynamic causation chain relying on the familiar Box<dyn Error> mechanism. Call via() on a Problem to add an error to the top of the chain. This mechanism does not replace the std source() but instead complements it as we provide APIs that make it easy to iterate our chaining as well as recursively traversing source().

Example of traversing the "tree":

fn read_the_file(path: &str) -> Result<String, Problem> {
    Ok(std::fs::read_to_string(path)?)
}

fn main() {
    if let Err(problem) = read_the_file("non-existing.txt") {
        // We can iterate the problem directly (the causation chain "trunk")
        for cause in &problem {
            println!("cause branch:");
            // For each cause we can iterate the "branch" of Error::source() calls
            for error in cause.iter_sources() {
                println!("• {}", error);
            }
        }
    }
}

Note that Problem does not itself implement the std Error trait, though it does implement Debug and Display. This is due to a current limitation in Rust's type system that may have a workaround in the future. Until then we can simply call into_error() when we need a Problem to implement Error.

Tag Errors

Because it's so easy to chain errors with via() an elegant pattern emerges. Instead of managing one big enum we can create simple, reusable "tag" error types (they are just an empty struct), which can be chained on top of any error. This is so common that we provide the tag_error!() macro to make it easy to create them. Example:

use problemo::*;

// The second optional argument is the Display representation
// (defaults to the type name)
tag_error!(OperatingSystemError, "operating system");

fn read_the_file(path: &str) -> Result<String, Problem> {
    std::fs::read_to_string(path).via(OperatingSystemError)
}

fn main() {
    if let Err(problem) = read_the_file("non-existing.txt") {
        if problem.has_error_type::<OperatingSystemError>() {
            println!("Your computer is broken!");
        } else {
            println!("Could not read. Try again?");
        }
    }
}

Problemo comes with commonly used tag error types in its common module.

Handling Errors

If we care about the error(s) that caused the problem then we can traverse the causation chain with cause_with_error_type(), under(), and iter_under():

if let Some(cause) = problem.cause_with_error_type::<OperatingSystemError>() {
    println!("Your computer is broken!");
    for cause in cause.iter_under() {
        println!("  because: {}", cause.error);
    }
}

A consequence of how easy it is to add a via() and then iterate the causation chain is that tag errors can be used to mark barriers between segments of the chain. For example, if we want to print everything that's not low-level, we can just stop there:

for cause in problem.iter_causes_until_error_type::<LowLevelError>() {
    println!("• {}", cause.error);
}

Grouping Error Types

Tag errors are unrelated to each other in the type system. However, sometimes it can be useful to group them together as variants of a basic type. In other words, an enum.

We can use derive_more to easily define such an enum and then use Problemo's has_error() and cause_with_error() instead of has_error_type() and cause_with_error_type(). Example:

use {derive_more::*, problemo::*};

// Note that we need to implement PartialEq for has_error() to work
#[derive(Debug, Display, Error, PartialEq)]
enum OperatingSystemError {
    #[display("I/O")]
    IO,

    #[display("network")]
    Network,
}

// The above is similar to just using:
//   tag_error!(IoError);
//   tag_error!(NetworkError);
// 
// The advantage of grouping them together in an enum is that we can do:
//   has_error_type::<OperatingSystemError>()

fn read_the_file(path: &str) -> Result<String, Problem> {
    // We're using via() twice here just to show that we can
    std::fs::read_to_string(path)
        .via(OperatingSystemError::IO)
        .via(common::LowLevelError)
}

fn main() {
    if let Err(problem) = read_the_file("non-existing.txt") {
        if problem.has_error(&OperatingSystemError::IO) {
            println!("Your computer is broken!");
        }
        ...
    }
}

As it turns out, the advantage of using an enum over individual error types is rather minor. Because Problem handles causation chaining for us the enum doesn't have to fulfil that role. As such, we don't end up reaching for this pattern very often.

Gloss Errors

An enum is one way to express variations of a type. But what if the difference between our variations is merely cosmetic and doesn't matter for actual error handling?

For this simpler use case we provide the gloss_error!() and static_gloss_error!() macros, which are similar to tag_error!() but with the option of allowing us to change their Display. They are essentially string newtypes. Example:

use problemo::*;

// The second optional argument is a prefix for Display
static_gloss_error!(InvalidPathError, "invalid path");

fn read_the_file(path: &str) -> Result<String, Problem> {
    if path.is_empty() {
        return Err(InvalidPathError::as_problem("empty"));
    } else if !path.starts_with('/') {
        return Err(InvalidPathError::as_problem("not absolute"));
    }
    ...
}

Although we could compare the inner strings of gloss errors in order to differentiate them, it's not recommended. String comparison is much less efficient than type matching. If we find ourselves caring about the gloss's value then it's a poor fit for our use case. We should be using tag errors or an enum instead. Or... an attachment (see below).

Problemo comes with a bunch of commonly used gloss error types in its common module. One of them is simply called GlossError. We even provide a convenience function to create it:

use problemo::{*, common::*};

// Disgustingly easy
return Err("I failed".gloss());

// Easy and *fancy*
return Err(format!("{} failed", subject).gloss());

That's both clear and kinda cute, isn't it? But it's also a bit lazy. gloss() will work in a pinch but we still recommend using one of the common gloss error types or defining our own so that callers can better differentiate them. Examples:

// Like this:
return Err(InvalidError::as_problem("wrong type"));

// On a Result:
fn read_file(path: &str) -> Result<String, Problem> {
    std::fs::read_to_string(path).into_gloss::<InvalidError>()
}

It's really not that much more verbose than a plain gloss(), is it?

2. Attachments

Secondly, we want to be able to attach additional, typed values to any error. This allows us to provide contextual information for handling and debugging the error, such as backtraces, exit codes, locations in source files, request IDs, custom representations, formatting rules for different environments, etc.

The std Error trait supports exactly three attachments: the optional source() mentioned above as well as the required Debug and Display textual representations.

An extensible solution would require storing and exposing APIs for additional attachments in our error types via a trait and/or wrapper types.

Problemo's solution is remarkably trivial. Because we already have a wrapper type, Problem, we've simply included attachments as a vector of Box<dyn Any> for every cause in the chain. To add an attachment we just need to call with(). We also provide APIs that make it easy to find specific attachment types. Example:

use problemo::*;

tag_error!(OperatingSystemError, "operating system");
tag_error!(ParseError, "parse");

struct Location {
    row: usize,
    column: usize,
}

fn parse_file(path: &str) -> Result<(), Problem> {
    let content = std::fs::read_to_string(path).via(OperatingSystemError)?;
    ...
    // The as_problem() function is a shortcut constructor for Problem::from(error)
    // You can also use into_problem() for any std Error
    return Err(ParseError::as_problem().with(Location { row, column }));
    ...
}

fn main() {
    if let Err(problem) = parse_file("non-existing.txt") {
        println!("{}", problem);
    
        // Note that this will return the first Location in the causation chain;
        // Use attachments_of_type() if you want all of them 
        if let Some(location) = problem.attachment_of_type::<Location>() {
            println!("  at {}/{}", location.row, location.column);
        }
    }
}

Data vs. Metadata

Attachments are an error-handling super power as they allow us to cleanly separate the error "data" from its contextual "metadata".

Consider that in our parser example above we might have various error types happening at a Location. Without the attachments feature we would probably have create something like a Locatable trait and implement it for all potential error types. And if it's an external error type we would have to also wrap it (because of the orphan rules). It's all quite painful boilerplate. We know because we've done it a lot. Indeed, this pain is the raison d'être for Problemo.

If you've used other error-handling libraries for Rust then you might have seen the term "context" being used not for metadata but for the actual error itself, i.e. the data. We think this seemingly small terminological choice has contributed to considerable misunderstanding and indeed to misuse. Even if you don't want to use Problemo we hope that learning about it nudges you to think more clearly about this distinction.

It's Probably Metadata

A best practice emerges: When we start defining an error struct, for every one of its fields we ask ourselves if it's really contextual metadata. In less philosophical terms: Could this field potentially be relevant to other error types? If so, we make it an attachment type instead of a field. It then becomes a reusable building block for providing context to any error type.

If we follow this rule of thumb we find out that in practice many if not most of our error fields are better defined as attachments. This is why tag errors are so useful: When all the data lives in attachments then we can get away with an empty struct for the actual error.

Indeed, this is so common that we provide the attachment!(), string_attachment!(), and static_string_attachment!() macros to make it easy to define newtypes for single-value attachments. Example:

use problemo::*;

tag_error!(UrlError, "URL");
string_attachment!(UrlAttachment);

fn read_url(url: &str) -> Result<String, Problem> {
    let content = reqwest::blocking::get(url)
        .via(UrlError)
        .with(UrlAttachment::new(url))?;
    ...
}

Loophole by Design

The causation chain is Box<dyn Error> while attachments are Box<dyn Any>. Doesn't that create a loophole through which we can use non-Error types for everything and just "mask" them as tag errors?

Yes, and it's entirely by design. Problemo's headlining goal is "to improve the experience of working with Rust's std Error trait". Keep in mind that a Result has no constraints for its Err and it's entirely viable to use types and traits that are incompatible with Error. Problem itself is not an Error. But we believe that it's a good idea to encourage, support, and indeed require the use Error in the causation chain because it allows for a bare-minimum, std-supported way of handling of errors even when their concrete types are unknown.

To go beyond the bare-minimum we would need to work with known, concrete types. As such it doesn't really matter if we're downcasting from a Box<dyn Error> or a Box<dyn Any>. In practice these two approaches look almost the same:

struct MyError { ... };

impl std::error::Error for MyError {}

fn handle_by_error(problem: &Problem) {
    if let Some(my_error) = problem.error_of_type::<MyError>() {
        ...
    }
}

// Does not implement Error
struct MyCustomError { ... };

fn handle_by_attachment(problem: &Problem) {
    if let Some(my_custom_error) = problem.attachment_of_type::<MyCustomError>() {
        ...
    }
}

So, yes, we can use attachments for our "real" error types while relying on simple tags and glosses for the Error. It's not cheating. It's a way for us to provide a useful std facade.

For a different approach using supertraits see this example and compare it to this.

Debug Context

It can be useful to attach debug context to problems.

Problemo has support for attaching std Location via with_location() and backtraces via with_backtrace(). Both of these can be auto-attached to all problems by setting the environment variables PROBLEMO_LOCATION and/or PROBLEMO_BACKTRACE to anything other than "0".

By default the backtrace is std Backtrace but by enabling the backtrace-external feature it will use (and re-export) the backtrace library instead, which provides access to stack frames.

See the example.

Note that even though Location is in the std::panic module it works even when we're not panicking. However, for it to be useful requires some discipline and care. We would need to make sure to add the #[track_caller] annotation to all functions on the call path that we want to skip, and that could have consequences in the case of debugging an actual panic. Also, closures count as functions but cannot be annotated (see issue), so they might have to be avoided. See this example. All things considered, it might be better to rely on backtraces instead. They are more cluttered but will work without these caveats.

3. Error Accumulation

Thirdly and finally, we want to let functions return more than one error. A classic use case is parsing, in which there might be multiple syntax and grammar errors in the input. Callers (and users) would be interested in all of them. Another example is a function that distributes work among threads, during which each thread could encounter different errors. Again, all of the errors can be important to the caller.

A common solution is to create a custom error type that internally stores multiple errors. Or, even more simply, the Err could just be a vector of errors.

But our requirement goes beyond mere multiplicity. In some cases callers might care only that the function succeeds, in which case it would be more efficient to fail on the first error, a.k.a. "fail fast". We might also sometimes prefer to stream the errors instead of storing them all in memory. For example, consider that our parser might emit thousands of errors on a bad input. It would be more memory-efficient as well as more responsive to print them out as they arrive. In other words, we should be able to accumulate them into the terminal instead of into memory.

Problemo's solution is the ProblemReceiver trait. If we want to store the errors then we can use the Problems type, which implements the trait by "swallowing" the errors into a vector. If we want to fail on the first error then we can use the FailFast type, which implements the trait by simply returning the error. Problems also supports an optional list of "critical" error types: If it encounters one of these it fails fast instead of swallowing. If we need custom behavior then we can implement the trait on our own types. It has just one simple function, give(Problem).

Challenges

Although the ProblemReceiver trait is very simple, using it involves awkwardness and requires discipline. We believe it's worth it for the flexibility and for providing opportunities for optimization.

The first challenge is that this is an inversion-of-control design, meaning that the caller has to provide the ProblemReceiver implementation. Commonly it's just passed as an extra function argument.

A useful advantage of inversion-of-control is that because the caller owns the receiver it can be reused: A function can pass the same receiver reference along to other functions that it calls, and the caller can likewise call multiple functions with one receiver, finally handling all the accumulated errors at once.

The second challenge is that an error-accumulating function's Result might actually be Ok even if there are errors. This is because they could have all been swallowed by the receiver. The first consequence is that such a function needs to be able to return something with Ok. This could be a partial result, which can be useful in itself. In our parser example we would be able to show the user what we succeeded in parsing in spite of the errors. In other cases it could be an empty collection or, if even that's impossible or irrelevant, it could be a None, in which case we would have to make sure to return an Option for Ok. The second consequence is that upon getting an Ok the caller would still need to check for accumulated errors. Problems has a check() function that does just that.

Example:

use problemo::{*, common::*};

/// Error-accumulating functions have special signatures:
///   By our convention we put the receiver as the *last* argument;
///   its type is generic
/// And also special internal implementations:
///   Specifically we have to make sure to give() all errors to the receiver
///   in order to give it an opportunity to swallow them or to fail fast;
///   We provide a few friendly APIs to make this easier
fn read_files<ProblemReceiverT>(
    paths: &[&str],
    problems: &mut ProblemReceiverT,
) -> Result<Vec<String>, Problem>
where
    ProblemReceiverT: ProblemReceiver,
{
    let mut strings = Vec::default();
    for path in paths {
        // give_ok() is like ok() but will give the problem to the receiver;
        // Note that we still use "?" in order to support a fast fail
        if let Some(string) = std::fs::read_to_string(path)
            .via(LowLevelError)
            .give_ok(problems)?
        {
            strings.push(string);
        }
    }

    // If we had swallowed errors then this would be a partial result
    // (i.e. not all files were read)
    Ok(strings)
}

fn main() -> Result<(), Problem> {
    let mut problems = Problems::default();
    let strings = read_files(&["non-existing1.txt", "non-existing2.txt"], &mut problems)?;
    
    // When using Problems the call above will *never* return Err
    // (it swallows all the errors)
    // Thus we thus *must* call check() here if we want to fail
    problems.check()?;

    // By contrast, we can trust that FailFast will *always* return Err on error
    let strings = read_files(&["non-existing3.txt", "non-existing4.txt"], &mut FailFast)?;
    ...
}

Working with std Result

? on a Result will Just Work™ if the error is a std Error type. This is because Problem implements From<Error>. In effect such an ? is the start of a causation chain.

That said, we want to put our best foot forward. Problemo comes with an extension trait for std Result, so we can insert a via() and/or a with() before the ?. At the very least we can add a quick via(common::LowLevelError).

The functions also have lazy versions, such as map_via() and map_with(), that will generate values only when there is an error.

The Locality Caveat

Problemo's first requirement, for a causation chain, implies historical continuity between the links in the chain. As we iterate we are also moving back in time. We are also moving in "space". Thanks, Einstein! For now, Problemo does not support faster-than-light travel, which would allow causes to precede effects. Nevertheless, the notion that our errors "travel" to get to us is a common metaphor. Sometimes this is described as errors "bubbling up" from "lower levels" in the application.

In Rust terms, this expectation implies movability in memory as well as portability across thread boundaries. Specifically, the Box<dyn Error> approach requires 'static, and Problemo adds a requirement for Send and Sync, too.

But not all errors are expected to travel. Specifically, they may contain lifetime-bounded references and/or data that is not Send and Sync. Such errors must be handled "locally" within the lifetime and/or thread in which they are created. As such they cannot be captured into a Problem's causation chain.

That said, we might still want to include a record of such an error in the chain; not the error itself but rather a representation of it. If it's enough to just capture its Display, then into_thread_problem(), into_gloss(), and even gloss() would all do the trick. Otherwise, we can use the standard map_err() function to provide our own conversion, in which we can make use of a new error type and/or add attachments. Here's a simple example:

use {problemo::{*, common::*}, std::sync::*};

fn get_string(string: &Mutex<String>) -> Result<String, Problem> {
    // We are glossing the PoisonError (capturing its Display)
    string
        .lock()
        .into_thread_problem()
        .map(|string| string.clone())
}

Memory Usage

Problemo relies on thin-vec by default, allowing Result<_, Problem> to take as little memory as possible when there is no error. If this is undesirable we can use default-features = false in our Cargo.toml to switch to std collection types and remove the thin-vec dependency.

FAQ

Why should I use Problemo instead of the many alternatives?

Maybe because you appreciate its goals and design? We're not in competition with others and we're not trying to win you over. (Unless there were a trophy involved, then things would get heated!)

Error handling is a very opinionated topic and one size does not fit all. As it stands, you're going to have to do the homework to evaluate each library and decide which is "best".

At the very least we urge you to consider not only the experience of writing your code but also the experience of users of your code. How easy is it for them to inspect and handle the errors you return?

Why doesn't Problem wrap a concrete error type? Wouldn't it be better to have the compiler validate correct usage?

To be clear, Problemo embraces the importance of concrete types for both errors and attachments. Rust's type system is our essential tool for inspecting and handling the causation chain.

But that doesn't mean it's a good idea to require the top of the causation chain to be of a certain type. Our experience in large projects with such a requirement has led us to believe that it's an arbitrary, cumbersome, and ultimately pointless practice, especially for high-level functions that can have long causation chains. More often than not, the details we and our users care about are deeper in the chain.

Simply put, any returned Err signifies failure, period. In practice just having the compiler test that the topmost error is of a "correct" type guarantees very little if we're not actually handling the details of that failure. All we've done is add clutter and unnecessary complexity.

This dilemma isn't unique to Rust. For example, the Java world has had a long-running debate about the usefulness of checked exceptions (example). It all hinges on whether you believe that having the compiler enforce the recognition of the topmost error type encourages programmers to handle that error. Do you? We don't.

Why no pretty formatting of causation chains with attachments?

Formatting is deliberately out of scope precisely because it's important.

We're not going to be able to do it justice because it's very specific to the reason for formatting, the reporting environment, and even personal taste. Are you printing out errors to a console for debugging? Are you displaying an error to a user in a dialog box? Are you logging errors to a textual file or a database for auditing? These are all very different use cases, some of which may require you to specifically hide sensitive information, provide text in multiple human languages, etc.

We might provide add-on libraries to help with this in the future, but we want to keep the core unopinionated in this regard. For ideas on how to implement formatting, check out the function_attachment and supertrait examples.

Why doesn't Problemo come with macros such as bail!()?

It seems that most error-handling libraries have these. They do the job of optimizing the creation and returning of errors so that it would take the smallest number of keystrokes. Problemo instead prefers verbosity, clarity, and debuggability by encouraging the use of explicit function calls.

We use macros only when functions can't work. Generally speaking our code is deliberately straightforward and non-magical and it should be easy for any Rust programmer to read and understand. Error handling is foundational and we believe that it should not be a mysterious black box.

You are of course free to create your own macros for Problemo but we don't want to promote them in our published API.

no_std?

Problemo's stated goal is to improve the use of std Error. That said, it's easy to imagine causation chains of types with fewer constraints. If the need for no_std arises let's work together to make it possible.

I like Problemo but it's missing my favorite feature!

That's not a question! Anyway, we are happy to hear your suggestions. Please be nice about it and do keep in mind that we want to keep Problemo lean, simple, and focused on the essentials. If the feature is something that can be built on top of Problemo, perhaps as a supplementary library, then that will likely be the preferred route.

Spanish?

Actually it's Spanglish. And even more actually it's Esperanto. Mi havas naŭdek naŭ problemojn, sed hundino ne estas unu.

"AI"?

Please, no.

Popular Alternatives

• error-stack, like Problemo, supports chaining and attachments. It does, however, require you to provide a concrete error type for your returns, which it (confusingly in our opinion) calls "the context". It supports returning groups of errors as long as they are of that same "context" type, as well as backtraces and pretty printing.

• rootcause works similarly to error-stack in practice while also supporting type-less wrappers like Problemo. It also features first-class (but limited) support for non-static errors without having to convert them, which is achieved through an innovative use of generic markers. Its scope is broad and it's relatively complex. It includes a customizable error formatter and other powerful features.

• anyhow is simple in its usage but is in fact a sophisticated library. It solves the problem of not being able to set the source() of a std Error by rewriting its dynamic dispatch vtable. On top of this, it lets you add non-errors to the causation chain via an internal wrapper, which (again, confusingly in our opinion) it calls "a context". It only supports one attachment type, a backtrace, which is handled implicitly and automatically.

• SNAFU works similarly to Anyhow in practice but takes a different design approach by introducing its own set of traits as a replacement for std Error while also allowing for compatibility with it. This allows you to build custom, rich error types on top of SNAFU.

• eyre is a fork of Anyhow with support for customizable formatting.

License

Like much of the Rust ecosystem, licensed under your choice of either of

• Apache License, Version 2.0
• MIT license

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.