# data-source
Build-time pipeline that downloads CloudFormation resource provider schemas (with patches pre-applied), derives
per-region resource-type data from the downloaded schemas, syncs rule-source extensions and additional specs (when a
cfn-lint root is provided), processes schemas, and generates all validation artifacts consumed by engine crates at
compile time. Everything compiles into the binary - no runtime fetching.
## Commands
```bash
# Generate from existing upstream data
cargo run -p cloudformation-validate-data-source --features maintenance --example generate
# Refresh all upstream sources, then generate every output (cfn-lint root is required)
cargo run -p cloudformation-validate-data-source --features maintenance --example sync -- --cfn-lint-root <DIR>
```
The `generate` and `sync` examples require the `maintenance` feature, which enables dependencies used only by the
data maintenance pipeline. `sync` is the complete workflow: it refreshes every upstream source, records source
versions, and generates all outputs. `generate` reruns code generation from the existing upstream data without network
access.
`--cfn-lint-root` is required by `sync`, which fails before starting work when it is absent.
A successful sync records both strict, source-qualified values together only after all source processing succeeds.
## Directory Structure
```
data-source/
├── handwritten/ # Manually authored data, checked in
├── upstream/ # Raw data synced from external sources
│ ├── schemas/ # Downloaded CFN + SAM schemas (per resource type)
│ ├── providers/ # Per-region type→hash maps (from the enhanced archive)
│ └── extensions/ # Rule-source extension files (only with --cfn-lint-root)
└── generated/ # All processed/codegen output (never edit manually)
├── patched_schemas/ # Schemas with patches+extensions applied
├── data/ # Extracted metadata consumed by all engines
├── cel-rules/ # CEL rule descriptors
└── schema-validator/ # Compiled schemas for schema-validator
```