Crate dre_core 

DRE core: turns a project directory into a resolved, validated project, and runs it.

Re-exports

pub use diag::Diagnostic;

pub use diag::Diagnostics;

pub use diag::Severity;

Modules

catalog

Live relation context for integrations. Descriptions are read from the database; DRE does not create or maintain upstream comments.

codes

Every diagnostic and error code DRE reports, in one registry.

coltypes

How dre validate --live compares a declared source column’s data_type with what the database returns (an Arrow type, as the source plugin delivers it). The comparison is loose: a declared SQL type matches a family of Arrow types (bigint ≈ Int64, varchar(20) ≈ Utf8), since plugins map database types to Arrow in their own ways. Documented in docs/sources.md.

config

Reading DRE’s YAML files into typed config.

constraints

Plugin version constraints: intersecting semver requirements and detecting contradictions.

dates

Calendar values for templates: run.date, run.now, date(), datetime() and the periods built from them. Everything is worked out at render time, so compiled SQL holds literals and a rerun with the same DRE_RUN_DATE renders the same text.

diag

Validation diagnostics: every problem found in one pass, with file and line where known.

engine

The run engine’s parts around a Binding’s run (crate::run):

error

DRE’s error type at module boundaries: a registered Code (whose Kind decides the exit code and whether trying again can help), the message, and an optional hint at the fix.

fs

Writing DRE’s durable state: write_atomic for every file a later run, or another tool, reads back (run_results.json, the drift snapshot, the manifest, dre.lock, plugin and package metadata). A forced stop mid-write leaves the previous or the next complete file, never broken JSON. Report outputs aren’t state and are written as they are.

lock

dre.lock: the exact plugin versions and package commits a project resolved, committed alongside it.

lookups

Lookups: small tables kept as files under lookups/ (mapping tables, code lists), used in SQL through ref('name') without ever being written to the source database by the user.

manager

The plugin manager: a static JSON registry index of plugin packages, checksum-verified downloads, side-by-side versioned installs, and dre.lock.

manifest

The project manifest, <target>/manifest.json: everything a project declares, as the engine resolves it, for orchestrators, CI and other tools. It’s a deliberate projection of the loaded Project, not a dump: built offline (no connection, no profiles, no plugins), the same bytes for the same project and the same inputs on every OS, and free of secrets and machine-specific values.

message

The built-in message format: a Binding’s query results rendered through Jinja into a short headline, a title plus text in portable Markdown (dre_protocol::markdown).

numbers

Number filters for every template: number, percent, signed, currency and compact, formatted for the run’s locale:.

occurrences

Schedule occurrences: when each schedules.yml entry fires within a window, and the exact command that runs each firing (dre schedule ls). Pure computation over the loaded project: no profiles, plugins, network or writes. Storing, diffing and dispatching occurrences is the caller’s business.

options

Plugin options: a format’s output: keys and a destination entry’s keys. Core owns only the keys every output shares (format, destination, template, extension, profile, path); everything else belongs to the plugin, which checks it (the protocol’s validate). A new format or destination therefore needs no change to core.

packages

Macro packages: reusable macros from another project (e.g. dre_utils), declared under packages: in the project’s root dependencies.yml or packages.yml:

parse

The parse pass: each Binding rendered without a database, as dbt’s parse finds ref() and source(), to learn every query’s sources and the connection it runs on. Its results feed selection (-s source:), the manifest, dre ls and dre validate, none of which connect; a run repeats it with its own inputs and then holds rendering to what it found.

plugins

Finding installed plugin packages.

preflight

Jinja pre-flight: syntax checks and static scans of var(), env_var() and run.* call sites. Nothing is rendered with live values here.

profiles

profiles.yml: connection definitions, kept outside the project.

project

Loading a DRE project directory into a resolved, validated project.

query

A single read-only query on a configured connection, for external integrations.

render

Runtime Jinja rendering: one environment per connection a Binding uses (its queries’ SQL) plus one for output paths and template values. Provides run.*, var(), env_var(), run_query(), columns(), ref(), source(), target.name, connection.*, destination.*, profile(), the calendar functions in crate::dates and every macro in macros/.

rows

Typed, streaming JSON row export for integrations. Decimals retain their exact text; columns carry Arrow types so consumers need not infer types from values or CSV.

run

The run path: turn a resolved project into delivered files.

run_results

run_results.json: what one run of one Binding did. A public, versioned contract (docs/run-results.schema.json is generated from these types): adding an optional field keeps SCHEMA_VERSION; removing, renaming or re-typing one, or changing what it means, bumps it (a minor DRE release before 1.0, a major one after).

runs

A Binding’s run folders, so overlapping and interrupted runs are safe:

schedule

Schedule timings: their shape, and the checks shared by dre validate and dre schedule ls. Core never fires schedules; occurrences works out when they would.

secrets

Secret masking. The value of every environment variable named DRE_SECRET_* is replaced by ***** in everything DRE writes for people to read: the console, logs/dre.log, JSON events, run_results.json and the compiled SQL in target/compiled/. The SQL actually sent to the database is unchanged. mask_secrets: false in dre_project.yml turns masking off.

selector

Selector resolution, shared by dre validate (schedules.yml) and dre run.

settings

DRE’s settings and where each value came from.

sqlsplit

Statement splitting: lexing ;-separated SQL, not parsing it.

target

The target path: the folder DRE writes its generated files to (compiled SQL, run outputs, schema snapshots, run_results.json, the manifest). target/ in the project root unless set, highest first, by --target-path, DRE_TARGET_PATH or target_path: in dre_project.yml. It’s always a local (or mounted) path, never inside the project’s sources.

values

Arrow values as Jinja values, for run_query() rows and message results.

variables

Optional variable declarations, separate from vars so existing map values keep their meaning.

yaml

YAML loading with file/line locations for diagnostics.

Functions

dre_home

DRE’s home directory: ~/.dre.

slash

A path with / separators on every platform, so messages and goldens read the same everywhere.

version

The version string dre --version prints. DRE has no released version yet.