Struct arbitrary::Unstructured

pub struct Unstructured<'a> { /* private fields */ }

A source of unstructured data.

An Unstructured helps Arbitrary implementations interpret raw data (typically provided by a fuzzer) as a “DNA string” that describes how to construct the Arbitrary type. The goal is that a small change to the “DNA string” (the raw data wrapped by an Unstructured) results in a small change to the generated Arbitrary instance. This helps a fuzzer efficiently explore the Arbitrary’s input space.

Unstructured is deterministic: given the same raw data, the same series of API calls will return the same results (modulo system resource constraints, like running out of memory). However, Unstructured does not guarantee anything beyond that: it makes not guarantee that it will yield bytes from the underlying data in any particular order.

You shouldn’t generally need to use an Unstructured unless you are writing a custom Arbitrary implementation by hand, instead of deriving it. Mostly, you should just be passing it through to nested Arbitrary::arbitrary calls.

Example

Imagine you were writing a color conversion crate. You might want to write fuzz tests that take a random RGB color and assert various properties, run functions and make sure nothing panics, etc.

Below is what translating the fuzzer’s raw input into an Unstructured and using that to generate an arbitrary RGB color might look like:

use arbitrary::{Arbitrary, Unstructured};

/// An RGB color.
#[derive(Arbitrary)]
pub struct Rgb {
    r: u8,
    g: u8,
    b: u8,
}

// Get the raw bytes from the fuzzer.
let raw_data: &[u8] = get_input_from_fuzzer();

// Wrap it in an `Unstructured`.
let mut unstructured = Unstructured::new(raw_data);

// Generate an `Rgb` color and run our checks.
if let Ok(rgb) = Rgb::arbitrary(&mut unstructured) {
    run_my_color_conversion_checks(rgb);
}

Implementations

impl<'a> Unstructured<'a>

pub fn new(data: &'a [u8]) -> Self

Create a new Unstructured from the given raw data.

Example

use arbitrary::Unstructured;

let u = Unstructured::new(&[1, 2, 3, 4]);

pub fn len(&self) -> usize

Get the number of remaining bytes of underlying data that are still available.

Example

use arbitrary::{Arbitrary, Unstructured};

let mut u = Unstructured::new(&[1, 2, 3]);

// Initially have three bytes of data.
assert_eq!(u.len(), 3);

// Generating a `bool` consumes one byte from the underlying data, so
// we are left with two bytes afterwards.
let _ = bool::arbitrary(&mut u);
assert_eq!(u.len(), 2);

pub fn is_empty(&self) -> bool

Is the underlying unstructured data exhausted?

unstructured.is_empty() is the same as unstructured.len() == 0.

Example

use arbitrary::{Arbitrary, Unstructured};

let mut u = Unstructured::new(&[1, 2, 3, 4]);

// Initially, we are not empty.
assert!(!u.is_empty());

// Generating a `u32` consumes all four bytes of the underlying data, so
// we become empty afterwards.
let _ = u32::arbitrary(&mut u);
assert!(u.is_empty());

pub fn arbitrary<A>(&mut self) -> Result<A>where
    A: Arbitrary<'a>,

Generate an arbitrary instance of A.

This is simply a helper method that is equivalent to <A as Arbitrary>::arbitrary(self). This helper is a little bit more concise, and can be used in situations where Rust’s type inference will figure out what A should be.

Example

use arbitrary::{Arbitrary, Unstructured};

#[derive(Arbitrary)]
struct MyType {
    // ...
}

fn do_stuff(value: MyType) {
    // ...
}

let mut u = Unstructured::new(&[1, 2, 3, 4]);

// Rust's type inference can figure out that `value` should be of type
// `MyType` here:
let value = u.arbitrary()?;
do_stuff(value);

pub fn arbitrary_len<ElementType>(&mut self) -> Result<usize>where
    ElementType: Arbitrary<'a>,

Get the number of elements to insert when building up a collection of arbitrary ElementTypes.

This uses the <ElementType as Arbitrary>::size_hint method to smartly choose a length such that we most likely have enough underlying bytes to construct that many arbitrary ElementTypes.

This should only be called within an Arbitrary implementation.

Example

use arbitrary::{Arbitrary, Result, Unstructured};

impl<'a, T> Arbitrary<'a> for MyCollection<T>
where
    T: Arbitrary<'a>,
{
    fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self> {
        // Get the number of `T`s we should insert into our collection.
        let len = u.arbitrary_len::<T>()?;

        // And then create a collection of that length!
        let mut my_collection = MyCollection::with_capacity(len);
        for _ in 0..len {
            let element = T::arbitrary(u)?;
            my_collection.insert(element);
        }

        Ok(my_collection)
    }
}

pub fn int_in_range<T>(&mut self, range: RangeInclusive<T>) -> Result<T>where
    T: Int,

Generate an integer within the given range.

Do not use this to generate the size of a collection. Use arbitrary_len instead.

Panics

Panics if range.start > range.end. That is, the given range must be non-empty.

Example

use arbitrary::{Arbitrary, Unstructured};

let mut u = Unstructured::new(&[1, 2, 3, 4]);

let x: i32 = u.int_in_range(-5_000..=-1_000)
    .expect("constructed `u` with enough bytes to generate an `i32`");

assert!(-5_000 <= x);
assert!(x <= -1_000);

pub fn choose<'b, T>(&mut self, choices: &'b [T]) -> Result<&'b T>

Choose one of the given choices.

This should only be used inside of Arbitrary implementations.

Returns an error if there is not enough underlying data to make a choice or if no choices are provided.

Examples

Selecting from an array of choices:

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]);
let choices = ['a', 'b', 'c', 'd', 'e', 'f', 'g'];

let choice = u.choose(&choices).unwrap();

println!("chose {}", choice);

An error is returned if no choices are provided:

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]);
let choices: [char; 0] = [];

let result = u.choose(&choices);

assert!(result.is_err());

pub fn choose_index(&mut self, len: usize) -> Result<usize>

Choose a value in 0..len.

Returns an error if the len is zero.

Examples

Using Fisher–Yates shuffle shuffle to gerate an arbitrary permutation.

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]);
let mut permutation = ['a', 'b', 'c', 'd', 'e', 'f', 'g'];
let mut to_permute = &mut permutation[..];
while to_permute.len() > 1 {
    let idx = u.choose_index(to_permute.len()).unwrap();
    to_permute.swap(0, idx);
    to_permute = &mut to_permute[1..];
}

println!("permutation: {:?}", permutation);

An error is returned if the length is zero:

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]);
let array: [i32; 0] = [];

let result = u.choose_index(array.len());

assert!(result.is_err());

pub fn ratio<T>(&mut self, numerator: T, denominator: T) -> Result<bool>where
    T: Int,

Generate a boolean according to the given ratio.

Panics

Panics when the numerator and denominator do not meet these constraints:

• 0 < numerator <= denominator

Example

Generate a boolean that is true five sevenths of the time:

use arbitrary::Unstructured;

let mut u = Unstructured::new(&my_data);

if u.ratio(5, 7)? {
    // Take this branch 5/7 of the time.
}

pub fn fill_buffer(&mut self, buffer: &mut [u8]) -> Result<()>

Fill a buffer with bytes from the underlying raw data.

This should only be called within an Arbitrary implementation. This is a very low-level operation. You should generally prefer calling nested Arbitrary implementations like <Vec<u8>>::arbitrary and String::arbitrary over using this method directly.

If this Unstructured does not have enough underlying data to fill the whole buffer, it pads the buffer out with zeros.

Example

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4]);

let mut buf = [0; 2];

assert!(u.fill_buffer(&mut buf).is_ok());
assert_eq!(buf, [1, 2]);

assert!(u.fill_buffer(&mut buf).is_ok());
assert_eq!(buf, [3, 4]);

assert!(u.fill_buffer(&mut buf).is_ok());
assert_eq!(buf, [0, 0]);

pub fn bytes(&mut self, size: usize) -> Result<&'a [u8]>

Provide size bytes from the underlying raw data.

This should only be called within an Arbitrary implementation. This is a very low-level operation. You should generally prefer calling nested Arbitrary implementations like <Vec<u8>>::arbitrary and String::arbitrary over using this method directly.

Example

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3, 4]);

assert!(u.bytes(2).unwrap() == &[1, 2]);
assert!(u.bytes(2).unwrap() == &[3, 4]);

pub fn peek_bytes(&self, size: usize) -> Option<&'a [u8]>

Peek at size number of bytes of the underlying raw input.

Does not consume the bytes, only peeks at them.

Returns None if there are not size bytes left in the underlying raw input.

Example

use arbitrary::Unstructured;

let u = Unstructured::new(&[1, 2, 3]);

assert_eq!(u.peek_bytes(0).unwrap(), []);
assert_eq!(u.peek_bytes(1).unwrap(), [1]);
assert_eq!(u.peek_bytes(2).unwrap(), [1, 2]);
assert_eq!(u.peek_bytes(3).unwrap(), [1, 2, 3]);

assert!(u.peek_bytes(4).is_none());

pub fn take_rest(self) -> &'a [u8]

Consume all of the rest of the remaining underlying bytes.

Returns a slice of all the remaining, unconsumed bytes.

Example

use arbitrary::Unstructured;

let mut u = Unstructured::new(&[1, 2, 3]);

let mut remaining = u.take_rest();

assert_eq!(remaining, [1, 2, 3]);

pub fn arbitrary_iter<'b, ElementType: Arbitrary<'a>>(
    &'b mut self
) -> Result<ArbitraryIter<'a, 'b, ElementType>>

Provide an iterator over elements for constructing a collection

This is useful for implementing Arbitrary::arbitrary on collections since the implementation is simply u.arbitrary_iter()?.collect()

pub fn arbitrary_take_rest_iter<ElementType: Arbitrary<'a>>(
    self
) -> Result<ArbitraryTakeRestIter<'a, ElementType>>

Provide an iterator over elements for constructing a collection from all the remaining bytes.

This is useful for implementing Arbitrary::arbitrary_take_rest on collections since the implementation is simply u.arbitrary_take_rest_iter()?.collect()

pub fn arbitrary_loop(
    &mut self,
    min: Option<u32>,
    max: Option<u32>,
    f: impl FnMut(&mut Self) -> Result<ControlFlow<(), ()>>
) -> Result<()>

Call the given function an arbitrary number of times.

The function is given this Unstructured so that it can continue to generate arbitrary data and structures.

You may optionaly specify minimum and maximum bounds on the number of times the function is called.

You may break out of the loop early by returning Ok(std::ops::ControlFlow::Break). To continue the loop, return Ok(std::ops::ControlFlow::Continue).

Panics

Panics if min > max.

Example

Call a closure that generates an arbitrary type inside a context an arbitrary number of times:

use arbitrary::{Result, Unstructured};
use std::ops::ControlFlow;

enum Type {
    /// A boolean type.
    Bool,

    /// An integer type.
    Int,

    /// A list of the `i`th type in this type's context.
    List(usize),
}

fn arbitrary_types_context(u: &mut Unstructured) -> Result<Vec<Type>> {
    let mut context = vec![];

    u.arbitrary_loop(Some(10), Some(20), |u| {
        let num_choices = if context.is_empty() {
            2
        } else {
            3
        };
        let ty = match u.int_in_range::<u8>(1..=num_choices)? {
            1 => Type::Bool,
            2 => Type::Int,
            3 => Type::List(u.int_in_range(0..=context.len() - 1)?),
            _ => unreachable!(),
        };
        context.push(ty);
        Ok(ControlFlow::Continue(()))
    })?;

    // The number of loop iterations are constrained by the min/max
    // bounds that we provided.
    assert!(context.len() >= 10);
    assert!(context.len() <= 20);

    Ok(context)
}