pub struct MaterializeConfig {
pub mode: MaterializeMode,
pub ref_strategy: RefStrategy,
pub memtable_max_rows: usize,
pub max_row_group_rows: usize,
pub max_row_group_bytes: usize,
pub iceberg: IcebergConfig,
pub wap: bool,
}Expand description
Optional materialization / ref() registration policy (materialize: in yml).
All fields are optional; omitting the whole block keeps lake-as-truth Parquet re-read and stream write mode.
Fields§
§mode: MaterializeModestream (default) | collect
ref_strategy: RefStrategyparquet (default) | memtable
memtable_max_rows: usizeUsed only when ref_strategy: memtable. Defaults to DEFAULT_MEMTABLE_MAX_ROWS.
max_row_group_rows: usizeParquet WriterProperties max rows per row group (stream + collect writers).
max_row_group_bytes: usizeSoft flush threshold for Parquet in_progress_size (bytes).
iceberg: IcebergConfigIceberg catalog vs filesystem layout.
wap: boolWrite-Audit-Publish: stage under .wap/{run_id}/, audit, then atomic publish.
Default false (stream still uses partial→rename atomicity without WAP dirs).
Implementations§
Source§impl MaterializeConfig
impl MaterializeConfig
Sourcepub fn effective_mode(&self) -> MaterializeMode
pub fn effective_mode(&self) -> MaterializeMode
Resolve write mode: env RBT_MATERIALIZE_MODE=stream|collect overrides yml.
Sourcepub fn choose_ref_backend(&self, row_count: usize) -> RefBackend
pub fn choose_ref_backend(&self, row_count: usize) -> RefBackend
Decide MemTable vs lake file for a model that produced row_count rows.
Trait Implementations§
Source§impl Clone for MaterializeConfig
impl Clone for MaterializeConfig
Source§fn clone(&self) -> MaterializeConfig
fn clone(&self) -> MaterializeConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for MaterializeConfig
impl Debug for MaterializeConfig
Source§impl Default for MaterializeConfig
impl Default for MaterializeConfig
Source§impl<'de> Deserialize<'de> for MaterializeConfig
impl<'de> Deserialize<'de> for MaterializeConfig
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
impl Eq for MaterializeConfig
Source§impl PartialEq for MaterializeConfig
impl PartialEq for MaterializeConfig
Source§impl Serialize for MaterializeConfig
impl Serialize for MaterializeConfig
impl StructuralPartialEq for MaterializeConfig
Auto Trait Implementations§
impl Freeze for MaterializeConfig
impl RefUnwindSafe for MaterializeConfig
impl Send for MaterializeConfig
impl Sync for MaterializeConfig
impl Unpin for MaterializeConfig
impl UnsafeUnpin for MaterializeConfig
impl UnwindSafe for MaterializeConfig
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> Downcast for T
impl<T> Downcast for T
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more