Struct ExportConfig 

pub struct ExportConfig {

    pub name: String,
    pub query: Option<String>,
    pub query_file: Option<String>,
    pub table: Option<String>,
    pub mode: ExportMode,
    pub cdc: Option<CdcExportConfig>,
    pub cursor_column: Option<String>,
    pub cursor_fallback_column: Option<String>,
    pub incremental_cursor_mode: IncrementalCursorMode,
    pub chunk_column: Option<String>,
    pub chunk_dense: bool,
    pub chunk_size: usize,
    pub chunk_size_memory_mb: Option<u64>,
    pub chunk_count: Option<usize>,
    pub chunk_by_days: Option<u32>,
    pub chunk_by_key: Option<String>,
    pub parallel: usize,
    pub time_column: Option<String>,
    pub time_column_type: TimeColumnType,
    pub days_window: Option<u32>,
    pub partition_by: Option<String>,
    pub partition_granularity: PartitionGranularity,
    pub format: FormatType,
    pub compression: CompressionType,
    pub compression_level: Option<u32>,
    pub compression_profile: Option<CompressionProfile>,
    pub skip_empty: bool,
    pub destination: DestinationConfig,
    pub verify: VerifyMode,
    pub meta_columns: MetaColumns,
    pub quality: Option<QualityConfig>,
    pub max_file_size: Option<String>,
    pub chunk_checkpoint: bool,
    pub chunk_max_attempts: Option<u32>,
    pub tuning: Option<TuningConfig>,
    pub source_group: Option<String>,
    pub reconcile_required: bool,
    pub columns: HashMap<String, String>,
    pub target: Option<String>,
    pub on_schema_drift: SchemaDriftPolicy,
    pub shape_drift_warn_factor: Option<f64>,
    pub parquet: Option<ParquetConfig>,
}

Fields

name: String

query: Option<String>

query_file: Option<String>

table: Option<String>

Shortcut for query: "SELECT * FROM <schema>.<table>".

Accepts table or schema.table with ASCII-only identifiers ([A-Za-z_][A-Za-z0-9_]*). Generates an unquoted single-table query so the Postgres NUMERIC catalog-hint resolver recognises it and auto-types numeric(p,s) columns without manual overrides.

Mutually exclusive with query and query_file.

mode: ExportMode

cdc: Option<CdcExportConfig>

Change-data-capture settings, required when mode: cdc. Reuses the export’s table, destination, and format; carries only the CDC-specific knobs (resume checkpoint, per-engine stream params).

cursor_column: Option<String>

cursor_fallback_column: Option<String>

Secondary column for IncrementalCursorMode::Coalesce only (see ADR-0007).

incremental_cursor_mode: IncrementalCursorMode

How primary (and optional fallback) columns drive incremental progression.

chunk_column: Option<String>

chunk_dense: bool

chunk_size: usize

chunk_size_memory_mb: Option<u64>

Target memory budget per chunk in MB. When set, chunk_size is derived from this budget at plan-build time using a pg_class row-size estimate (pg_relation_size / reltuples), clamped to [10_000, 5_000_000] rows.

Mutually exclusive with an explicit non-default chunk_size:. Only applies to mode: chunked on a Postgres source using the table: shortcut (the row-size probe needs a known relation).

exports:
  - name: page_views
    table: public.page_views
    mode: chunked
    chunk_size_memory_mb: 256

chunk_count: Option<usize>

Divide the column range into exactly this many equal chunks. Mutually exclusive with chunk_dense and chunk_by_days. When set, chunk_size is computed dynamically from min/max.

chunk_by_days: Option<u32>

chunk_by_key: Option<String>

Keyset (seek) pagination on this single index-backed unique key — the source-safe shape for tables without a single-integer PK (OPT-4). The column MUST be backed by a usable index (PK or unique); the planner refuses a non-indexed key rather than emit a full-scan + filesort query.

parallel: usize

time_column: Option<String>

time_column_type: TimeColumnType

days_window: Option<u32>

partition_by: Option<String>

Date/time output partitioning: split this export’s rows into one destination sub-prefix per calendar bucket of this DATE or TIMESTAMP column, bucketed by partition_granularity (day / month / year), in a Hive-style col=value/ layout (created_at=2023-01-01/, created_at=2023-01/, created_at=2023/). Requires a {partition} token in destination.path / destination.prefix.

This is not arbitrary value partitioning: the column’s min/max is read and parsed as a date to generate contiguous calendar buckets, so a non-temporal column (e.g. partition_by: status) fails at run time with “could not parse partition min ‘’ from column ‘’ as a date”. To split by a categorical column, write one export per value with a WHERE filter instead.

Orthogonal to mode: each partition runs the export’s own mode, so mode: chunked chunks within a day. Rows whose partition column is NULL land in col=__HIVE_DEFAULT_PARTITION__/ (Hive default partition) so no row is silently dropped. Not compatible with mode: time_window.

exports:
  - name: events
    table: events
    partition_by: created_at        # must be a DATE or TIMESTAMP column
    partition_granularity: day
    destination:
      type: s3
      bucket: my-bucket
      prefix: "events/{partition}/"   # → events/created_at=2023-01-01/

partition_granularity: PartitionGranularity

Calendar bucket width for partition_by: day (default), month, or year. Determines how the partition column’s date/timestamp range is split into contiguous Hive buckets (col=2023-01-01/ / col=2023-01/ / col=2023/). Has no effect unless partition_by is set.

format: FormatType

compression: CompressionType

compression_level: Option<u32>

compression_profile: Option<CompressionProfile>

skip_empty: bool

destination: DestinationConfig

verify: VerifyMode

Integrity depth required of --validate for this export’s parts. size (default) accepts size-only verification; content requires every part’s content MD5 to be checked against the store’s listing (no download) and fails validation for any part that could only be size-verified — e.g. a part too large to upload as a single PUT (raise max_file_size down so it fits), or a backend that exposes no checksum.

meta_columns: MetaColumns

quality: Option<QualityConfig>

max_file_size: Option<String>

Rotate to a new part when the current file reaches this size. Accepts B/KB/MB/GB (case-insensitive) or a bare byte count; a fractional value is allowed (1.5GB). Units are binary (IEC-style): KB = 1024 bytes, MB = 1024 KB, GB = 1024 MB. Example: 256MB.

chunk_checkpoint: bool

chunk_max_attempts: Option<u32>

tuning: Option<TuningConfig>

source_group: Option<String>

Optional logical group for shared source capacity (replica, host). Advisory prioritization only.

reconcile_required: bool

Hint (Epic C / ADR-0006) that this export should always be treated as reconcile-heavy by planning, independent of the --reconcile CLI flag. Advisory only.

columns: HashMap<String, String>

Per-column type overrides (roadmap §8). Keys are column names; values are short type strings such as decimal(18,2), timestamp_tz, json.

exports:
  - name: payments
    columns:
      amount: decimal(18,2)
      fee: decimal(18,6)
      created_at: timestamp_tz

Overrides take priority over autodetection and are validated at plan time — an invalid type string fails before the export runs.

target: Option<String>

Downstream warehouse this export targets (bigquery / bq, duckdb). When set, rivet check --type-report resolves each column against it (native type, honest autoload type, recovery hint) without needing --target on the CLI — the CLI flag still wins when both are present. The Parquet interchange stays target-neutral (ADR-0014 T2); target: only drives guidance and the future load-schema artifact.

exports:
  - name: payments
    target: bigquery

on_schema_drift: SchemaDriftPolicy

Policy applied when structural schema drift is detected (column added, removed, or retyped). Defaults to warn: log a warning and continue.

shape_drift_warn_factor: Option<f64>

Growth-factor threshold for data shape drift warnings (Epic 8). When a string/binary column’s max observed byte length in the current run exceeds stored_max * shape_drift_warn_factor, Rivet logs a warning. None uses the default of 2.0. Set to 0.0 to disable shape tracking.

parquet: Option<ParquetConfig>

Parquet row group tuning. Only meaningful when format: parquet. When absent, the parquet library default (1,048,576 rows/group) is used.

Implementations

impl ExportConfig

pub fn effective_compression(&self) -> (CompressionType, Option<u32>)

Resolve the effective (CompressionType, level) for this export. compression_profile takes precedence over compression + compression_level.

L24: when a profile is set and a conflicting explicit codec/level was written, warn once that the profile wins rather than silently dropping the explicit choice. An explicit codec is only detectable when it differs from the #[serde(default)] (Zstd) — a literal compression: zstd alongside a profile is indistinguishable from an omitted field and stays silent.

pub fn max_file_size_bytes(&self) -> Option<u64>

pub fn resolve_query( &self, config_dir: &Path, params: Option<&HashMap<String, String>>, ) -> Result<String>

Trait Implementations

impl Clone for ExportConfig

fn clone(&self) -> ExportConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ExportConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ExportConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl JsonSchema for ExportConfig

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword.