Skip to main content

CheckpointConfig

Struct CheckpointConfig 

Source
pub struct CheckpointConfig {
    pub interval: Option<Duration>,
    pub max_retained: usize,
    pub alignment_timeout: Duration,
    pub pre_commit_timeout: Duration,
    pub persist_timeout: Duration,
    pub commit_timeout: Duration,
    pub incremental: bool,
    pub state_inline_threshold: usize,
    pub max_checkpoint_bytes: Option<usize>,
    pub max_pending_changelog_entries: usize,
    pub adaptive: Option<AdaptiveCheckpointConfig>,
    pub unaligned: Option<UnalignedCheckpointConfig>,
}
Expand description

Unified checkpoint configuration.

Fields§

§interval: Option<Duration>

Interval between checkpoints. None = manual only.

§max_retained: usize

Maximum number of retained checkpoints.

§alignment_timeout: Duration

Maximum time to wait for barrier alignment at fan-in nodes.

§pre_commit_timeout: Duration

Maximum time to wait for all sinks to pre-commit.

A stuck sink will not block checkpointing indefinitely. Defaults to 30 seconds.

§persist_timeout: Duration

Maximum time to wait for manifest persist (filesystem I/O).

A hung or degraded filesystem will not stall the runtime indefinitely. Defaults to 120 seconds.

§commit_timeout: Duration

Maximum time to wait for all sinks to commit (phase 2).

Defaults to 60 seconds.

§incremental: bool

Whether to use incremental checkpoints.

§state_inline_threshold: usize

Maximum operator state size (bytes) to inline in the JSON manifest.

States larger than this threshold are written to a state.bin sidecar file and referenced by offset/length in the manifest. This avoids inflating the manifest with base64-encoded blobs (~33% overhead).

Default: 1 MB (1_048_576). Set to usize::MAX to inline everything.

§max_checkpoint_bytes: Option<usize>

Maximum total checkpoint size in bytes (manifest + sidecar).

If the packed operator state exceeds this limit, the checkpoint is rejected with [LDB-6014] and not persisted. This prevents unbounded state from creating multi-GB sidecar files.

None means no limit (default). A warning is logged at 80% of the cap.

§max_pending_changelog_entries: usize

Maximum pending changelog entries per drainer before forced clear.

On checkpoint failure, clear_pending() is normally skipped. If failures repeat, pending entries grow unboundedly → OOM. This cap is a safety valve: if any drainer exceeds it after a failure, all drainers are cleared and an [LDB-6010] warning is logged.

Default: 10,000,000 entries.

§adaptive: Option<AdaptiveCheckpointConfig>

Adaptive checkpoint interval configuration.

When Some, the coordinator dynamically adjusts the checkpoint interval based on observed checkpoint durations using an exponential moving average. When None (default), the static interval is used unchanged.

§unaligned: Option<UnalignedCheckpointConfig>

Unaligned checkpoint configuration.

When Some, the coordinator can fall back to unaligned checkpoints when barrier alignment takes too long under backpressure. When None (default), only aligned checkpoints are used.

Trait Implementations§

Source§

impl Clone for CheckpointConfig

Source§

fn clone(&self) -> CheckpointConfig

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for CheckpointConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for CheckpointConfig

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> ArchivePointee for T

Source§

type ArchivedMetadata = ()

The archived version of the pointer metadata for this type.
Source§

fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata

Converts some archived metadata to the pointer metadata for itself.
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> LayoutRaw for T

Source§

fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>

Returns the layout of the type.
Source§

impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
where T: SharedNiching<N1, N2>, N1: Niching<T>, N2: Niching<T>,

Source§

unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool

Returns whether the given value has been niched. Read more
Source§

fn resolve_niched(out: Place<NichedOption<T, N1>>)

Writes data to out indicating that a T is niched.
Source§

impl<T> Pointee for T

Source§

type Metadata = ()

The metadata type for pointers and references to this type.
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> Scope for T

Source§

fn with<F, R>(self, f: F) -> R
where Self: Sized, F: FnOnce(Self) -> R,

Scoped with ownership.
Source§

fn with_ref<F, R>(&self, f: F) -> R
where F: FnOnce(&Self) -> R,

Scoped with reference.
Source§

fn with_mut<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut Self) -> R,

Scoped with mutable reference.
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

Source§

impl<T> Value for T
where T: Send + Sync + 'static,