apollo_configuration/

lib.rs

//! A typed, friendly configuration system for Apollo products.
//!
//! # The `#[configuration]` attribute
//!
//! apollo-configuration's `#[configuration]` attribute provides:
//! - Parsing of YAML files with friendly error reporting
//! - JSON schema generation
//! - Validation of user input according to that JSON schema
//! - Defaulting of field values
//! - Expansion of environment variables
//!
//! ```rust
//! use apollo_configuration::configuration;
//! use apollo_configuration::parse_yaml;
//!
//! #[configuration]
//! struct CsrfConfig {
//!     #[config(default = true)]
//!     enabled: bool,
//!     #[config(default = default_required_headers())]
//!     required_headers: Vec<String>,
//! }
//!
//! fn default_required_headers() -> Vec<String> {
//!     vec![
//!         "x-apollo-operation-name".to_string(),
//!         "apollo-require-preflight".to_string(),
//!     ]
//! }
//! ```
//!
//! ## JSON Schema Generation
//! Use [`export_json_schema`] to generate a JSON schema for a [`Configuration`] struct.
//!
//! ## YAML Parsing
//! Use [`parse_yaml`] to parse and validate YAML text into a [`Configuration`] struct.
//!
//! ## Validation
//! Configuration structs are validated using JSON Schema, and each field is validated according to
//! its own [validation rules](#field-validation). It's common for configuration structures to
//! require "cross-cutting" validation: for example, disallowing two features that cannot be used
//! together. To do things like this, you can add an additional custom validation function using
//! the `#[configuration(validate)]` attribute.
//!
//! ```rust
//! use apollo_configuration::configuration;
//! use apollo_configuration::ErrorCollector;
//!
//! #[configuration]
//! struct HomepageConfig {
//!     enabled: bool,
//!     graph_ref: Option<String>,
//! }
//!
//! #[configuration]
//! struct ApolloSandboxConfig {
//!     enabled: bool,
//! }
//!
//! #[configuration(validate = validate_config)]
//! struct Config {
//!     homepage: HomepageConfig,
//!     sandbox: ApolloSandboxConfig,
//! }
//!
//! fn validate_config(config: &Config, mut errors: ErrorCollector<'_>) {
//!     if config.homepage.enabled && config.sandbox.enabled {
//!         errors.nest("sandbox")
//!             .report_simple("sandbox can only be enabled if homepage is disabled");
//!     }
//! }
//! ```
//!
//! # The `#[config]` attribute
//! The `#[config]` attribute can be used on the structure's fields to change how fields are
//! parsed, validated, and displayed.
//!
//! ## Optional/Required fields
//! All fields are assumed to be optional by default. The type's [`Default`] impl is used when a
//! field is absent during deserialization. If a field is required, mark it with
//! `#[config(required)]`. If a configuration structure contains any required fields, it will not
//! implement [`Default`].
//!
//! If a field's type does not have a [`Default`] impl, you can provide a default value:
//! ```rust
//! # use std::num::NonZeroUsize;
//! # use apollo_configuration::configuration;
//! #
//! #[configuration]
//! struct CacheConfig {
//!     #[config(default = NonZeroUsize::new(500).unwrap())]
//!     cache_size: NonZeroUsize,
//! }
//! ```
//!
//! ## Field validation
//! apollo-configuration supports schema-level validation and fully custom validation.
//!
//! Schema-level validation works by using one of the below sub-attributes inside the `#[config]`
//! attribute:
//!
//! - `range(min = 0, max = 1024)` - Restrict the value of an integer field to the given range.
//! - `length(min = 0, max = 256)` - Restrict the length of a string or array field to the given
//!   range.
//! - `length(exact = 10)` - Require a string or array field to have the given length exactly.
//! - `pattern(r"^\d+$")` - Require a string field to match the given regular expression.
//!
//! Schema-level validation is also reflected in the generated JSON schema.
//!
//! Custom validation works by specifying a validation function in the `#[config(validate)]`
//! attribute:
//!
//! ```rust
//! use apollo_configuration::configuration;
//! use apollo_configuration::ErrorCollector;
//!
//! #[configuration]
//! struct Config {
//!     #[config(validate = validate_div_by_3)]
//!     magic_number: usize,
//! }
//!
//! fn validate_div_by_3(magic_number: &usize, mut errors: ErrorCollector<'_>) {
//!     if !magic_number.is_multiple_of(3) {
//!         errors.report_simple("magic number must be divisible by 3");
//!     }
//! }
//! ```
//!
//! Custom validation is not reflected in the JSON schema. Generally, schema-level validation
//! should be preferred. However, custom validation supports rich diagnostics using
//! [`ErrorCollector::report`], so it can still be a good choice if schema-level validation gives poor
//! errors.
//!
//! Custom validation can be disabled using the `#[config(skip_validate)]` attribute. This
//! can be necessary when using external types that do not implement `Validate`:
//!
//! ```rust
//! use apollo_configuration::configuration;
//!
//! #[configuration]
//! struct Config {
//!     #[config(skip_validate, default = tinystr::TinyAsciiStr::try_from_utf8(b"unnamed").unwrap())]
//!     #[schemars(with = "String")]
//!     name: tinystr::TinyAsciiStr<16>,
//! }
//! ```
//!
//! # Expansion
//!
//! Your users can write `${env.VAR_NAME}` syntax inside YAML values to insert the value of an
//! environment variable into the configuration. This can be used to include more dynamic values
//! or secrets without having to write them into the configuration file.
//!
//! It is an error to reference an environment variable that is not defined. The bash-style
//! defaulting syntax can be used to handle optional environment variables:
//! `${env.VAR_NAME:-default}`.
//!
//! If your configuration structure contains secrets, API keys, passwords, or sensitive URLs,
//! remember to use a redaction helper such as [`apollo-redaction`].
//!
//! [`apollo-redaction`]: https://docs.rs/apollo-redaction
//!
//! # Types for use as configuration values
//!
//! Primitive Rust types and several ecosystem types are supported as configuration values.
//! The [`types`] module provides wrappers for common external types that can be
//! used in configuration.
//!
//! Available newtypes:
//! - [`types::Duration`] - human-readable format (`30s`, `5m`, `1h30m`, `100ms`)
//! - [`types::IpAddr`], [`types::Ipv4Addr`], [`types::Ipv6Addr`]
//! - [`types::SocketAddr`], [`types::SocketAddrV4`], [`types::SocketAddrV6`]
//! - [`types::Url`]
//! - [`types::Method`], [`types::Uri`], [`types::HeaderName`], [`types::HeaderValue`]
//! - [`types::MetadataKey`], [`types::MetadataValue`] - gRPC metadata types
//! - [`types::Regex`]
//!
//! To enable these types add the appropriate feature to the apollo-configuration dependency:
//! `duration`, `grpc`, `http`, `regex`, `url`
//!
//! All newtypes support wrapping with [`apollo_redaction::Redacted`] for sensitive values
//! like API keys or passwords:
//!
//! ```rust
//! use apollo_configuration::configuration;
//! use apollo_redaction::Redacted;
//!
//! #[configuration]
//! struct ApiConfig {
//!     #[config(required)]
//!     api_key: Redacted<String>,
//! }
//!
//! let config: ApiConfig = apollo_configuration::parse_yaml(
//!     "api_key: secret-key-123",
//!     &Default::default(),
//! ).unwrap();
//!
//! // The value is accessible but protected from accidental logging
//! assert_eq!(config.api_key.unredact(), "secret-key-123");
//! assert_eq!(config.api_key.to_string(), "[REDACTED]");
//! ```
//!
//! See the [`apollo-redaction`] crate for details.
//!
//! ## Other types
//!
//! To use a type as a configuration value, it must implement these traits:
//! - [`Debug`]
//! - [`schemars::JsonSchema`]
//! - [`serde::Deserialize`]
//!
//! Where it makes sense, types used in configuration values should also implement [`Clone`],
//! [`Default`], and [`serde::Serialize`].
//!
//! # Enum configuration
//!
//! Enums can use the `#[configuration]` attribute. This is useful for configuration options
//! where the user selects between different modes or backends.
//!
//! ```rust
//! use apollo_configuration::configuration;
//!
//! #[configuration]
//! enum Endpoint {
//!     Http { url: String, timeout: u64 },
//!     Grpc { host: String, port: u16 },
//! }
//! ```
//!
//! ## Default variant
//!
//! Mark a variant with `#[config(default)]` to make the enum implement [`Default`]. When parsing
//! from YAML, omitted enum fields will use this default:
//!
//! ```rust
//! use apollo_configuration::configuration;
//! use apollo_configuration::parse_yaml;
//!
//! #[configuration]
//! #[derive(PartialEq, Eq)]
//! enum LogLevel {
//!     Debug,
//!     #[config(default)]
//!     Info,
//!     Warn,
//!     Error,
//! }
//!
//! #[configuration]
//! #[derive(PartialEq, Eq)]
//! struct Config {
//!     log_level: LogLevel,
//! }
//!
//! let config: Config = parse_yaml(r#"log_level: debug"#, &Default::default()).unwrap();
//! assert_eq!(config.log_level, LogLevel::Debug);
//!
//! let config: Config = parse_yaml(r#"{}"#, &Default::default()).unwrap();
//! assert_eq!(config.log_level, LogLevel::Info);
//! ```
//!
//! ## Unit variant fields
//!
//! Unit variant enums can be parsed directly from string values in YAML. The `#[configuration]`
//! attribute applies `#[serde(rename_all = "snake_case")]` automatically:
//!
//! ```rust
//! use apollo_configuration::configuration;
//!
//! #[configuration]
//! #[derive(PartialEq, Eq)]
//! enum EnabledMode {
//!     Measure,
//!     #[config(default)]
//!     Enforce,
//! }
//!
//! #[configuration]
//! struct DemandControlConfig {
//!     enabled: bool,
//!     mode: EnabledMode,
//! }
//!
//! let config: DemandControlConfig = apollo_configuration::parse_yaml(r#"
//!     enabled: true
//!     mode: measure
//! "#, &Default::default()).unwrap();
//! assert_eq!(config.mode, EnabledMode::Measure);
//! ```
//!
//! ## Struct variant fields
//!
//! Struct variant fields support the same `#[config(...)]` attributes as struct fields:
//!
//! ```rust
//! use apollo_configuration::configuration;
//!
//! #[configuration]
//! enum Endpoint {
//!     #[config(default)]
//!     Http {
//!         #[config(default = "http://localhost".to_string())]
//!         url: String,
//!         #[config(default = 30)]
//!         timeout: u64,
//!     },
//!     Grpc {
//!         #[config(required)]
//!         host: String,
//!         #[config(default = 9090)]
//!         port: u16,
//!     },
//! }
//! ```
//!
//! ## Tuple variant fields
//!
//! Tuple variants wrap a single inner type. The `#[config(...)]` attributes apply to the inner
//! type:
//!
//! ```rust
//! use apollo_configuration::configuration;
//!
//! #[configuration]
//! struct HttpConfig {
//!     #[config(default = "http://localhost".to_string())]
//!     url: String,
//! }
//!
//! #[configuration]
//! enum Endpoint {
//!     #[config(default)]
//!     Http(HttpConfig),
//!     Custom(String),
//! }
//! ```
//!
use std::sync::Arc;

use jsonschema::error::ValidationErrorKind;
use jsonschema::paths::LazyLocation;
use jsonschema::paths::Location;
use miette::Diagnostic;
use miette::SourceSpan;
use saphyr::LoadableYamlNode as _;
use saphyr::MarkedYamlOwned;
use schemars::JsonSchema;
use schemars::SchemaGenerator;
use serde::Deserialize;

mod errors;
pub mod expansion;
mod overridable;
#[doc(hidden)]
pub mod private;
pub mod types;
mod validate;

pub use apollo_configuration_macros::configuration;
pub use errors::*;
pub use overridable::Overridable;
pub use validate::ErrorCollector;
pub use validate::Validate;
pub use validate::validate;

/// Marker trait for configuration structs. See the [`#[configuration]`][configuration] proc macro for
/// documentation.
pub trait Configuration: JsonSchema + for<'de> Deserialize<'de> + Validate {}

impl<T> Configuration for Box<T> where T: Configuration {}
// XXX(@goto-bus-stop): make sure we can do this?
// impl <T> Configuration for Arc<T> where T: Configuration {}

/// Return a JSON schema for the given configuration structure.
///
/// # Panics
/// Panics if the generated schema does not match the meta-schema.
/// This can happen only if [`schemars::JsonSchema`] was implemented incorrectly for a type.
pub fn export_json_schema<T: Configuration>() -> serde_json::Value {
    let schema = SchemaGenerator::default().root_schema_for::<T>().to_value();

    jsonschema::meta::validate(&schema).expect("generated schema is invalid");

    schema
}

fn to_miette_span(yaml_span: saphyr_parser::Span) -> SourceSpan {
    let start = yaml_span.start.index();
    let end = yaml_span.end.index();
    SourceSpan::new(start.into(), end - start)
}

fn empty_span() -> SourceSpan {
    SourceSpan::new(0.into(), 0)
}

// We have to do multiple things with our YAML values.
// 1. Validate according to JSON Schema: this requires a dynamic `serde_json::Value`.
// 2. Report errors with source location. This requires a parse tree with location data
// 2. Parse into a Rust struct with source locations. This requires serde + a parse tree with
//    location data
fn yaml_to_json<'a, 'b>(
    location_data: &YamlLocationData,
    location: LazyLocation<'a, 'b>,
    yaml: serde_yaml::Value,
) -> Result<serde_json::Value, YamlToJsonError> {
    let source_code = location_data.source_code.clone();
    let make_span = || {
        location_data
            .resolve_instance_span(&(&location).into())
            .unwrap_or_else(empty_span)
    };

    Ok(match yaml {
        serde_yaml::Value::Null => serde_json::Value::Null,
        serde_yaml::Value::Bool(value) => serde_json::Value::Bool(value),
        serde_yaml::Value::Number(value) => {
            serde_json::Value::Number(if let Some(inner) = value.as_i64() {
                inner.into()
            } else if let Some(inner) = value.as_f64() {
                serde_json::Number::from_f64(inner).ok_or_else(|| {
                    YamlToJsonError::InvalidNumberError {
                        source_code: source_code.clone(),
                        label: make_span(),
                    }
                })?
            } else {
                return Err(YamlToJsonError::InvalidNumberError {
                    source_code: source_code.clone(),
                    label: make_span(),
                });
            })
        }
        serde_yaml::Value::String(value) => serde_json::Value::String(value),
        serde_yaml::Value::Tagged(tagged) => yaml_to_json(location_data, location, tagged.value)?,
        serde_yaml::Value::Mapping(value) => serde_json::Value::Object(
            value
                .into_iter()
                .map(|(key, value)| {
                    let key = key
                        .as_str()
                        .ok_or_else(|| YamlToJsonError::InvalidMappingError {
                            source_code: source_code.clone(),
                            label: make_span(),
                        })?;
                    Ok::<_, YamlToJsonError>((
                        key.to_string(),
                        yaml_to_json(location_data, location.push(key), value)?,
                    ))
                })
                .collect::<Result<_, _>>()?,
        ),
        serde_yaml::Value::Sequence(value) => serde_json::Value::Array(
            value
                .into_iter()
                .enumerate()
                .map(|(index, value)| yaml_to_json(location_data, location.push(index), value))
                .collect::<Result<_, _>>()?,
        ),
    })
}

#[derive(Debug, PartialEq)]
struct YamlLocationData {
    source_code: Arc<str>,
    marked_yaml: MarkedYamlOwned,
}

impl YamlLocationData {
    fn parse(text: &str) -> Result<Self, errors::ConfigError> {
        let source_code = Arc::from(text);
        let mut loaded = MarkedYamlOwned::load_from_str(&source_code)?;

        match loaded.len() {
            0 => Ok(Self::empty()),
            _ => Ok(Self {
                source_code,
                marked_yaml: loaded.remove(0),
            }),
        }
    }

    fn empty() -> Self {
        Self {
            source_code: Arc::from(""),
            marked_yaml: MarkedYamlOwned::from_bare_yaml(saphyr::Yaml::BadValue),
        }
    }

    /// Create a [`miette::SourceSpan`] for the YAML element pointed to by the given JSON pointer.
    fn resolve_instance_span(&self, location: &Location) -> Option<miette::SourceSpan> {
        Some(to_miette_span(self.resolve_element(location)?.span))
    }

    /// Navigate to the YAML element at the given location.
    fn resolve_element(&self, location: &Location) -> Option<&MarkedYamlOwned> {
        let mut element = &self.marked_yaml;
        for segment in location.iter() {
            match segment {
                jsonschema::paths::LocationSegment::Property(name) => {
                    element = element.data.as_mapping_get(&name)?;
                }
                jsonschema::paths::LocationSegment::Index(index) => {
                    element = element.data.as_sequence_get(index)?;
                }
            }
        }
        Some(element)
    }

    /// Create a [`miette::SourceSpan`] for a mapping key at the given location.
    ///
    /// This is useful for errors about unexpected properties, where we want to highlight
    /// the key name rather than the value.
    fn resolve_key_span(&self, location: &Location, key: &str) -> Option<miette::SourceSpan> {
        let element = self.resolve_element(location)?;
        let mapping = element.data.as_mapping()?;
        for (k, _v) in mapping {
            if k.data.as_str() == Some(key) {
                return Some(to_miette_span(k.span));
            }
        }
        None
    }

    fn resolve_error_path(&self, path: &serde_path_to_error::Path) -> Option<miette::SourceSpan> {
        let mut element = &self.marked_yaml;
        for segment in path.iter() {
            match segment {
                serde_path_to_error::Segment::Unknown => return None,
                serde_path_to_error::Segment::Seq { index } => {
                    element = element.data.as_sequence_get(*index)?;
                }
                serde_path_to_error::Segment::Map { key } => {
                    element = element.data.as_mapping_get(key)?;
                }
                serde_path_to_error::Segment::Enum { variant } => {
                    element = element.data.as_mapping_get(variant)?;
                }
            }
        }

        Some(to_miette_span(element.span))
    }
}

/// Options for parsing configuration from YAML.
#[derive(Default)]
pub struct ParseYamlOptions {
    variables: Option<Box<dyn expansion::VariableProvider>>,
}

impl ParseYamlOptions {
    /// Configure the variable provider for expansion.
    ///
    /// ## Default
    /// By default, the [`EnvVariables`] provider is used, using environment variables available to
    /// the process.
    ///
    /// To disable variable expansion outright, provide the unit type:
    /// ```rust,ignore
    /// let options = options.variables(());
    /// ```
    ///
    /// [`EnvVariables`]: expansion::EnvVariables
    pub fn variables(mut self, variables: impl expansion::VariableProvider + 'static) -> Self {
        self.variables = Some(Box::new(variables));
        self
    }

    pub fn parse<T: Configuration>(&self, text: &str) -> Result<T, errors::ConfigError> {
        parse_yaml(text, self)
    }
}

/// Resolve the source span for a validation error.
///
/// For most errors, this returns the span of the value that failed validation.
/// For `AdditionalProperties` errors, this returns the span of the unexpected key name,
/// which provides better error highlighting.
fn resolve_schema_error_span(
    location_data: &YamlLocationData,
    err: &jsonschema::ValidationError<'_>,
) -> SourceSpan {
    let fallback = || SourceSpan::new(0.into(), 0);

    // For AdditionalProperties errors, highlight the unexpected key name
    if let ValidationErrorKind::AdditionalProperties { unexpected } = err.kind()
        && let Some(first_unexpected) = unexpected.first()
        && let Some(span) = location_data.resolve_key_span(err.instance_path(), first_unexpected)
    {
        return span;
    }

    // TODO(@goto-bus-stop): We can further special case;
    // in particular for `anyOf` constraints, we can pick the most suitable error to
    // display based on its inner errors (eg. the inner error that is NOT `additionalProperties`)

    // Default: highlight the value at the instance path
    location_data
        .resolve_instance_span(err.instance_path())
        .unwrap_or_else(fallback)
}

pub fn parse_yaml<T: Configuration>(
    text: &str,
    options: &ParseYamlOptions,
) -> Result<T, errors::ConfigError> {
    let location_data = YamlLocationData::parse(text)?;

    let instance = serde_yaml::from_str(text)?;
    let mut instance: serde_json::Value = if matches!(instance, serde_yaml::Value::Null) {
        serde_json::Value::Object(Default::default())
    } else {
        yaml_to_json(&location_data, LazyLocation::new(), instance)?
    };

    if let Some(variables) = options.variables.as_deref() {
        expansion::expand(&mut instance, variables, Some(&location_data))?;
    }

    let schema = export_json_schema::<T>();
    let errors = jsonschema::validator_for(&schema)
        .expect("schema is known to be valid")
        .iter_errors(&instance)
        .map(|err| {
            Box::new(errors::ValidationError {
                label: resolve_schema_error_span(&location_data, &err),
                message: err.to_string(),
            }) as Box<dyn Diagnostic + Send + Sync + 'static>
        })
        .collect::<Vec<_>>();
    if !errors.is_empty() {
        return Err(ConfigError::ValidationError(ValidationErrors {
            source_code: location_data.source_code.clone(),
            errors,
        }));
    }

    // XXX(@goto-bus-stop): it would be nice to accumulate errors here, for example with https://docs.rs/serdev
    let instance: T = match serde_path_to_error::deserialize(&instance) {
        Ok(instance) => instance,
        Err(err) => {
            return Err(ConfigError::InvalidValue {
                source_code: location_data.source_code.clone(),
                label: location_data.resolve_error_path(err.path()),
                error: err.into_inner(),
            });
        }
    };

    let mut errors = vec![];
    instance.validate(ErrorCollector::new(&location_data, &mut errors));
    if !errors.is_empty() {
        return Err(ConfigError::ValidationError(ValidationErrors {
            source_code: location_data.source_code.clone(),
            errors,
        }));
    }

    Ok(instance)
}

#[cfg(test)]
mod tests {
    use crate::YamlLocationData;
    use jsonschema::paths::Location;

    #[test]
    fn json_schema_instance_path() {
        let yaml = YamlLocationData::parse(
            r#"
            core:
            - a: { c: "/core/0/a/c" }
            - b: { c: 2 }
        "#,
        )
        .expect("valid YAML");

        assert_eq!(
            yaml.resolve_instance_span(&Location::new().join("core").join(1)),
            Some(miette::SourceSpan::new(71.into(), 20)),
        );
        assert_eq!(
            yaml.resolve_instance_span(&Location::new().join("core").join(1).join("b")),
            Some(miette::SourceSpan::new(74.into(), 7)),
        );
        assert_eq!(
            yaml.resolve_instance_span(&Location::new().join("core").join(0).join("a").join("c")),
            Some(miette::SourceSpan::new(41.into(), 14)),
        );
        assert_eq!(
            yaml.resolve_instance_span(&Location::new().join("core").join(2)),
            None,
        );
    }

    #[test]
    fn empty_yaml_string_empty_yaml_location_data() {
        assert_eq!(
            YamlLocationData::parse("").unwrap(),
            YamlLocationData::empty()
        );
    }
}