Crate dre_core 

DRE core: turns a project directory into a resolved, validated project, and runs it.

Re-exports

pub use diag::Diagnostic;

pub use diag::Diagnostics;

pub use diag::Severity;

Modules

coltypes

How dre validate --live compares a declared source column’s data_type with what the database returns (an Arrow type, as the source plugin delivers it). The comparison is loose: a declared SQL type matches a family of Arrow types (bigint ≈ Int64, varchar(20) ≈ Utf8), since plugins map database types to Arrow in their own ways. Documented in docs/sources.md.

constraints

Plugin version constraints: intersecting semver requirements and detecting contradictions.

dates

Calendar values for templates: run.date, run.now, date(), datetime() and the periods built from them. Everything is worked out at render time, so compiled SQL holds literals and a rerun with the same DRE_RUN_DATE renders the same text.

diag

Validation diagnostics: every problem found in one pass, with file and line where known.

lock

dre.lock: the exact plugin versions and package commits a project resolved, committed alongside it.

lookups

Lookups: small tables kept as files under lookups/ (mapping tables, code lists), used in SQL through ref('name') without ever being written to the source database by the user.

manager

The plugin manager: a static JSON registry index of plugin packages, checksum-verified downloads, side-by-side versioned installs, and dre.lock.

manifest

The project manifest, <target>/manifest.json: everything a project declares, as the engine resolves it, for orchestrators, CI and other tools. It’s a deliberate projection of the loaded Project, not a dump: built offline (no connection, no profiles, no plugins), the same bytes for the same project and the same inputs on every OS, and free of secrets and machine-specific values.

occurrences

Schedule occurrences: when each schedules.yml entry fires within a window, and the exact command that runs each firing (dre schedule ls). Pure computation over the loaded project: no profiles, plugins, network or writes. Storing, diffing and dispatching occurrences is the caller’s business.

options

Plugin options: a format’s output: keys and a destination entry’s keys. Core owns only the keys every output shares (format, destination, template, extension, profile, path); everything else belongs to the plugin, which checks it (the protocol’s validate). A new format or destination therefore needs no change to core.

packages

Macro packages: reusable macros from another project (e.g. dre_utils), declared under packages: in the project’s root dependencies.yml or packages.yml:

parse

The parse pass: each Binding rendered without a database, as dbt’s parse finds ref() and source(), to learn every query’s sources and the connection it runs on. Its results feed selection (-s source:), the manifest, dre ls and dre validate, none of which connect; a run repeats it with its own inputs and then holds rendering to what it found.

plugins

Finding installed plugin packages.

preflight

Jinja pre-flight: syntax checks and static scans of var(), env_var() and run.* call sites. Nothing is rendered with live values here.

profiles

profiles.yml: connection definitions, kept outside the project.

project

Loading a DRE project directory into a resolved, validated project.

render

Runtime Jinja rendering: one environment per connection a Binding uses (its queries’ SQL) plus one for output paths and template values. Provides run.*, var(), env_var(), run_query(), columns(), ref(), source(), target.name, connection.*, destination.*, profile(), the calendar functions in crate::dates and every macro in macros/.

run

The run path: turn a resolved project into delivered files.

schedule

Schedule timings: their shape, and the checks shared by dre validate and dre schedule ls. Core never fires schedules; occurrences works out when they would.

secrets

Secret masking. The value of every environment variable named DRE_SECRET_* is replaced by ***** in everything DRE writes for people to read: the console, logs/dre.log, JSON events, run_results.json and the compiled SQL in target/compiled/. The SQL actually sent to the database is unchanged. mask_secrets: false in dre_project.yml turns masking off.

selector

Selector resolution, shared by dre validate (schedules.yml) and dre run.

sqlsplit

Statement splitting: lexing ;-separated SQL, not parsing it.

target

The target path: the folder DRE writes its generated files to (compiled SQL, run outputs, schema snapshots, run_results.json, the manifest). target/ in the project root unless set, highest first, by --target-path, DRE_TARGET_PATH or target_path: in dre_project.yml. It’s always a local (or mounted) path, never inside the project’s sources.

values

Arrow values as Jinja values, for run_query() rows.

yaml

YAML loading with file/line locations for diagnostics.

Functions

dre_home

DRE’s home directory: ~/.dre.

slash

A path with / separators on every platform, so messages and goldens read the same everywhere.

version

The version string dre --version prints. DRE has no released version yet.