Struct SourceTuning 

pub struct SourceTuning {

    pub batch_size: usize,
    pub batch_size_memory_mb: Option<usize>,
    pub throttle_ms: u64,
    pub statement_timeout_s: u64,
    pub max_retries: u32,
    pub retry_backoff_ms: u64,
    pub lock_timeout_s: u64,
    pub memory_threshold_mb: usize,
    pub max_batch_memory_mb: Option<usize>,
    pub on_batch_memory_exceeded: BatchMemoryPolicy,
    pub adaptive: bool,
    pub min_parallel: Option<usize>,
    pub max_value_mb: Option<usize>,
    /* private fields */
}

Fields

batch_size: usize

batch_size_memory_mb: Option<usize>

throttle_ms: u64

statement_timeout_s: u64

max_retries: u32

retry_backoff_ms: u64

lock_timeout_s: u64

memory_threshold_mb: usize

RSS limit in MB before chunk processing throttles. 0 = no limit (disabled).

max_batch_memory_mb: Option<usize>

Hard cap on a single Arrow batch in MB. None = no cap.

on_batch_memory_exceeded: BatchMemoryPolicy

adaptive: bool

When true, Rivet samples DB pressure metrics every super::ADAPTIVE_SAMPLE_INTERVAL batches and shrinks/restores the fetch size in response. Default: false.

min_parallel: Option<usize>

Floor for the OPT-2 concurrency governor: the lowest worker/connection count it will back parallelism down to under source pressure. None ⇒ 1. The ceiling is the export’s configured parallel. Only consulted when adaptive is on and parallel > 1.

max_value_mb: Option<usize>

Hard ceiling on a single cell/value in MB (OPT-1 memory hardening): a variable-length value (text/JSON/blob) larger than this aborts the run with RIVET_VALUE_TOO_LARGE instead of risking OOM, since the average-based batch cap can’t bound one giant cell. Some(0) / None disable the guard. Default: 256 MiB.

Implementations

impl SourceTuning

pub fn from_config(config: Option<&TuningConfig>) -> Self

Build tuning with the legacy Balanced fallback. Public for downstream callers and tests; production resolution in [crate::plan::build] uses Self::from_config_with_default_profile so that source.environment: can pick the right default.

pub fn from_config_with_default_profile( config: Option<&TuningConfig>, fallback_profile: TuningProfile, ) -> Self

Like Self::from_config but lets the caller override the fallback profile used when config.profile is unset.

pub fn profile_name(&self) -> &'static str

pub fn effective_batch_size(&self, schema: Option<&SchemaRef>) -> usize

If batch_size_memory_mb is set, compute and return an adjusted batch_size from the schema; otherwise return the configured batch_size.

pub fn batch_memory_bytes(batch: &RecordBatch) -> usize

Return the actual Arrow memory footprint of a batch in bytes.

Sums get_array_memory_size() across all columns — includes buffers for validity bitmaps, offsets, and value data. Does not include Arrow struct overhead (~few hundred bytes) which is negligible at batch scale.

pub fn resource_summary(&self) -> ResourceSummary

Produce a ResourceSummary from the resolved tuning settings.

The summary requires no database connection. It reports two batch-memory bounds based on narrow-table (~200 B/row) and wide-table (~10 KB/row) heuristics. A wide_table_risk flag is set when the upper bound exceeds 128 MB per batch.

Trait Implementations

impl Clone for SourceTuning

fn clone(&self) -> SourceTuning

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SourceTuning

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SourceTuning

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for SourceTuning

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for SourceTuning

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.