Struct Inserter

Source
pub struct Inserter<T> { /* private fields */ }
Available on crate feature inserter only.
Expand description

Performs multiple consecutive INSERTs.

By default, it doesn’t end the current active INSERT automatically. Use with_max_bytes, with_max_rows and with_period to set limits. Alternatively, call force_commit to forcibly end an active INSERT.

Rows are being sent progressively to spread network load.

All rows written by Inserter::write() between Inserter::commit() calls are sent in one INSERT statement.

Implementations§

Source§

impl<T> Inserter<T>
where T: Row,

Source

pub fn with_timeouts( self, send_timeout: Option<Duration>, end_timeout: Option<Duration>, ) -> Self

See Insert::with_timeouts().

Note that Inserter::commit() can call Insert::end() inside, so end_timeout is also applied to commit() method.

Source

pub fn with_max_bytes(self, threshold: u64) -> Self

The maximum number of uncompressed bytes in one INSERT statement.

This is the soft limit, which can be exceeded if rows between Inserter::commit() calls are larger than set value.

Note: ClickHouse inserts batches atomically only if all rows fit in the same partition and their number is less max_insert_block_size.

Unlimited (u64::MAX) by default.

Source

pub fn with_max_rows(self, threshold: u64) -> Self

The maximum number of rows in one INSERT statement.

In order to reduce overhead of merging small parts by ClickHouse, use larger values (e.g. 100_000 or even larger). Consider also/instead Inserter::with_max_bytes() if rows can be large.

This is the soft limit, which can be exceeded if multiple rows are written between Inserter::commit() calls.

Note: ClickHouse inserts batches atomically only if all rows fit in the same partition and their number is less max_insert_block_size.

Unlimited (u64::MAX) by default.

Source

pub fn with_period(self, period: Option<Duration>) -> Self

The time between INSERTs.

Note that Inserter doesn’t spawn tasks or threads to check the elapsed time, all checks are performend only on Inserter::commit(). However, it’s possible to use Inserter::time_left() and set a timer up to call Inserter::commit() to check passed time again.

Usually, it’s reasonable to use 1-10s period, but it depends on desired delay for reading the data from the table. Larger values = less overhead for merging parts by CH. Smaller values = less delay for readers.

Extra ticks are skipped if the previous INSERT is still in progress:

Expected ticks: |     1     |     2     |     3     |     4     |     5     |     6     |
Actual ticks:   | work -----|          delay          | work ---| work -----| work -----|

Unlimited (None) by default.

Source

pub fn with_period_bias(self, bias: f64) -> Self

Adds a bias to the period, so actual period is in the following range:

  [period * (1 - bias), period * (1 + bias)]

The bias parameter is clamped to the range [0, 1].

It helps to avoid producing a lot of INSERTs at the same time by multiple inserters.

Source

pub fn with_option( self, name: impl Into<String>, value: impl Into<String>, ) -> Self

Similar to Client::with_option, but for the INSERT statements generated by this Inserter only.

Source

pub fn set_timeouts( &mut self, send_timeout: Option<Duration>, end_timeout: Option<Duration>, )

Source

pub fn set_max_bytes(&mut self, threshold: u64)

Source

pub fn set_max_rows(&mut self, threshold: u64)

Source

pub fn set_period(&mut self, period: Option<Duration>)

Source

pub fn set_period_bias(&mut self, bias: f64)

Source

pub fn time_left(&mut self) -> Option<Duration>

How much time we have until the next tick.

None if the period isn’t configured.

Source

pub fn pending(&self) -> &Quantities

Returns statistics about data not yet inserted into ClickHouse.

Source

pub fn write(&mut self, row: &T) -> Result<()>
where T: Serialize,

Serializes the provided row into an internal buffer.

To check the limits and send the data to ClickHouse, call Inserter::commit().

§Panics

If called after the previous call that returned an error.

Source

pub async fn commit(&mut self) -> Result<Quantities>

Checks limits and ends the current INSERT if they are reached.

Source

pub async fn force_commit(&mut self) -> Result<Quantities>

Ends the current INSERT unconditionally.

Source

pub async fn end(self) -> Result<Quantities>

Ends the current INSERT and whole Inserter unconditionally.

If it isn’t called, the current INSERT is aborted.

Auto Trait Implementations§

§

impl<T> Freeze for Inserter<T>

§

impl<T> !RefUnwindSafe for Inserter<T>

§

impl<T> Send for Inserter<T>

§

impl<T> Sync for Inserter<T>

§

impl<T> Unpin for Inserter<T>

§

impl<T> !UnwindSafe for Inserter<T>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

impl<T> ErasedDestructor for T
where T: 'static,

Source§

impl<T> MaybeSendSync for T