Crate apollo_smith

Expand description

`apollo-smith`

A test case generator for GraphQL language.

§About

The goal of apollo-smith is to generate valid GraphQL documents by sampling from all available possibilities of GraphQL grammar.

We’ve written apollo-smith to use in fuzzing, but you may wish to use it for anything that requires GraphQL document generation.

apollo-smith is inspired by bytecodealliance’s wasm-smith crate, and the article written by Nick Fitzgerald on writing test case generators in Rust.

This is still a work in progress, for outstanding issues, checkout out the apollo-smith label in our issue tracker.

§Rust versions

apollo-smith is tested on the latest stable version of Rust. Older version may or may not be compatible.

§Using `apollo-smith` with `cargo fuzz`

Define a new target with cargo fuzz,

$ cargo fuzz add my_apollo_smith_fuzz_target

and add apollo-smith to your Cargo.toml:

## fuzz/Cargo.toml

[dependencies]
apollo-smith = "0.16.0"

It can then be used in a fuzz_target along with the arbitrary crate,

// fuzz/fuzz_targets/my_apollo_smith_fuzz_target.rs

#![no_main]

use libfuzzer_sys::fuzz_target;
use arbitrary::Unstructured;
use apollo_smith::DocumentBuilder;

fuzz_target!(|input: &[u8]| {
    let mut u = Unstructured::new(input);
    let document = DocumentBuilder::new(&mut u).build()?;
    let document_str = String::from(document);


});

and fuzzed with the following command:

$ cargo +nightly fuzz run my_apollo_smith_fuzz_target

§Using `apollo-smith` with `apollo-parser`

You can use apollo-parser to generate valid operations in apollo-smith.

use std::fs;

use apollo_parser::Parser;
use apollo_smith::{Document, DocumentBuilder};

use libfuzzer_sys::arbitrary::{Result, Unstructured};

/// This generate an arbitrary valid GraphQL operation
pub fn generate_valid_operation(input: &[u8]) -> Result<String> {
    let parser = Parser::new(&fs::read_to_string("supergraph.graphql").expect("cannot read file"));

    let tree = parser.parse();
    if tree.errors().next().is_some() {
        panic!("cannot parse the graphql file");
    }

    let mut u = Unstructured::new(input);

    // Convert `apollo_parser::Document` into `apollo_smith::Document`.
    let apollo_smith_doc = Document::try_from(tree.document()).unwrap();

    // Create a `DocumentBuilder` given an existing document to match a schema.
    let mut gql_doc = DocumentBuilder::with_document(&mut u, apollo_smith_doc)?;
    let operation_def = gql_doc.operation_definition()?.unwrap();

    Ok(operation_def.into())
}

§Generating responses using `apollo-smith` with `apollo-compiler`

If you have a GraphQL operation in the form of an ExecutableDocument and its accompanying Schema, you can generate a response matching the shape of the operation with apollo_smith::ResponseBuilder.

ResponseBuilder is generic over its randomness source via the RandomProvider trait. This allows it to be used with arbitrary::Unstructured for fuzz testing, with RandProvider for standard random generation, or with any custom implementation.

§Using `Unstructured` (for fuzz testing)

use apollo_compiler::validation::Valid;
use apollo_compiler::ExecutableDocument;
use apollo_compiler::Schema;
use apollo_smith::{ResponseBuilder, ResponseError};
use arbitrary::Unstructured;
use rand::RngExt as _;
use serde_json_bytes::Value;

pub fn generate_valid_response(
    doc: &Valid<ExecutableDocument>,
    schema: &Valid<Schema>,
) -> Result<Value, ResponseError> {
    let mut buf = [0u8; 2048];
    rand::rng().fill(&mut buf);
    let mut u = Unstructured::new(&buf);

    ResponseBuilder::new(&mut u, doc, schema).build()
}

§Using `RandProvider`

Use RandProvider to wrap any rand::Rng:

use apollo_smith::{RandProvider, ResponseBuilder};

let mut rng = RandProvider(rand::rng());
let response = ResponseBuilder::new(&mut rng, &doc, &schema)
    .with_min_list_size(1)
    .with_max_list_size(5)
    .with_null_ratio(1, 4)
    .build()?;

§Configuring generation per type

Use with_generator to register a Generator for any named GraphQL type — scalars, objects, interfaces, or unions. The same trait powers both leaf and composite generation; the fields argument is empty for scalar types and contains the requested selection (flattened across fragments and grouped by response key) for composite types.

To swap one of the built-in scalar defaults:

use apollo_smith::{ResponseBuilder, StringGenerator};
use apollo_compiler::Name;

let response = ResponseBuilder::new(&mut rng, &doc, &schema)
    .with_generator(
        Name::new_unchecked("ID".into()),
        StringGenerator { min_len: 8, max_len: 8 },
    )
    .build()?;

Or with completely custom generation logic:

use apollo_smith::{Generator, Generators, RandomProvider, ResponseBuilder, ResponseError};
use apollo_compiler::executable::Field;
use apollo_compiler::{Name, Node};
use indexmap::IndexMap;
use serde_json_bytes::Value;

struct IncrementingGenerator {
    id: i32,
}

impl<R: RandomProvider> Generator<R> for IncrementingGenerator {
    fn generate(
        &mut self,
        _rng: &mut R,
        _generators: &mut Generators<R>,
        _fields: &IndexMap<String, Vec<Node<Field>>>,
    ) -> Result<Value, ResponseError> {
        self.id += 1;
        Ok(Value::Number(self.id.into()))
    }
}

let response = ResponseBuilder::new(&mut rng, &doc, &schema)
    .with_generator(
        Name::new_unchecked("ID".into()),
        IncrementingGenerator { id: 0 },
    )
    .build();

For composite types, the generator’s return value is used as-is — the builder does not recurse into it. The generator receives the requested fields already flattened across fragments and grouped by response key, so it can return only what the caller asked for:

use apollo_smith::{Generator, Generators, RandomProvider, ResponseBuilder, ResponseError};
use apollo_compiler::executable::Field;
use apollo_compiler::{Name, Node};
use indexmap::IndexMap;
use serde_json_bytes::{Map, Value};

/// Generator for the federation `_Service` type that returns the real schema SDL.
struct ServiceGenerator {
    sdl: String,
}

impl<R: RandomProvider> Generator<R> for ServiceGenerator {
    fn generate(
        &mut self,
        _rng: &mut R,
        _generators: &mut Generators<R>,
        fields: &IndexMap<String, Vec<Node<Field>>>,
    ) -> Result<Value, ResponseError> {
        let mut obj = Map::new();
        for (response_key, group) in fields {
            // The first field in the group is representative — multiple entries only
            // appear when the same response key shows up in several fragments.
            if group[0].name == "sdl" {
                obj.insert(response_key.clone(), Value::String(self.sdl.clone().into()));
            }
        }
        Ok(Value::Object(obj))
    }
}

let response = ResponseBuilder::new(&mut rng, &doc, &schema)
    .with_generator(
        Name::new_unchecked("_Service"),
        ServiceGenerator { sdl },
    )
    .build()?;

§Accessing the default generators

The generators argument passed to every Generator::generate call is the same registry the builder is using. It starts from Generators::default(), which pre-registers a generator for each of the five standard GraphQL scalars:

Type	Generator	Default range
`Boolean`	`BooleanGenerator`	`true` or `false`
`Int`	`IntGenerator`	`0..=100`
`Float`	`FloatGenerator`	`-1.0..=1.0`
`String`	`StringGenerator`	1–10 alphanumeric characters
`ID`	`IdGenerator`	`0..=100`, serialized as a string

Each per-type struct is public, so a custom composite generator can:

delegate a single field back to the registry via generators.generate_scalar(type_name, rng) — uses whatever is registered for that scalar (default or user-supplied), falling back to a StringGenerator if nothing is registered;
dispatch to any registered type by name with generators.try_generate(type_name, rng, fields) — returns None if no generator is registered for that type;
construct one of the built-in per-type generators directly (e.g. IntGenerator { min: -10, max: 10 }.generate(rng, generators, &fields)) when you want a tuned instance without registering it.

The example below uses generate_scalar so each field is filled by whichever generator is registered for its scalar type — including any overrides the caller has installed via with_generator:

impl<R: RandomProvider> Generator<R> for PartialUserGenerator {
    fn generate(
        &mut self,
        rng: &mut R,
        generators: &mut Generators<R>,
        fields: &IndexMap<String, Vec<Node<Field>>>,
    ) -> Result<Value, ResponseError> {
        let mut obj = Map::new();
        for (response_key, group) in fields {
            let field = &group[0];
            let value = if field.name == "id" {
                Value::String(self.next_id().into())
            } else {
                generators.generate_scalar(field.ty().inner_named_type(), rng)?
            };
            obj.insert(response_key.clone(), value);
        }
        Ok(Value::Object(obj))
    }
}

§Limitations

Recursive object type not yet supported (example : myType { inner: myType })

§License

Licensed under either of

Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)

at your option.

Re-exports§

pub use generators::BooleanGenerator;
pub use generators::FloatGenerator;
pub use generators::Generator;
pub use generators::Generators;
pub use generators::IdGenerator;
pub use generators::IntGenerator;
pub use generators::StringGenerator;
pub use random::RandProvider;
pub use random::RandomProvider;
pub use random::ResponseError;

Modules§

generators: Generator trait and registry used by ResponseBuilder to produce values for GraphQL types, plus the built-in generators for the standard GraphQL scalars.
random

Structs§

DirectiveDef: The __DirectiveDef type represents a Directive definition.
Document: The __Document type represents a GraphQL document.A GraphQL Document describes a complete file or request string operated on by a GraphQL service or client. A document contains multiple definitions, either executable or representative of a GraphQL type system.
DocumentBuilder: DocumentBuilder is a struct to build an arbitrary valid GraphQL document
EnumTypeDef: Enums are special scalars that can only have a defined set of values.
FragmentDef: The __fragmentDef type represents a fragment definition
InputObjectTypeDef: Input objects are composite types used as inputs into queries defined as a list of named input values..
InterfaceTypeDef: InterfaceTypeDef is an abstract type where there are common fields declared.
ObjectTypeDef: Object types represent concrete instantiations of sets of fields.
OperationDef: The __operationDef type represents an operation definition
ResponseBuilder: Builds a GraphQL response which matches the shape of a given executable GraphQL document.
ScalarTypeDef: Represents scalar types such as Int, String, and Boolean. Scalars cannot have fields.
SchemaDef: A GraphQL service’s collective type system capabilities are referred to as that service’s “schema”.
UnionTypeDef: UnionDefs are an abstract type where no common fields are declared.
Unstructured: A source of unstructured data.

Enums§

FromError
Value: Represents any valid JSON value.

Type Aliases§

Result: A Result with the error type fixed as arbitrary::Error.