Struct questdb::ingress::Buffer

pub struct Buffer { /* private fields */ }

A reusable buffer to prepare ILP messages.

Example

use questdb::ingress::{Buffer, TimestampMicros, TimestampNanos};

let mut buffer = Buffer::new();

// first row
buffer
    .table("table1")?
    .symbol("bar", "baz")?
    .column_bool("a", false)?
    .column_i64("b", 42)?
    .column_f64("c", 3.14)?
    .column_str("d", "hello")?
    .column_ts("e", TimestampMicros::now())?
    .at(TimestampNanos::now())?;

// second row
buffer
    .table("table2")?
    .symbol("foo", "bar")?
    .at(TimestampNanos::now())?;

The buffer can then be sent with the Sender’s flush method.

Sequential Coupling

The Buffer API is sequentially coupled:

• A row always starts with table.
• A row must contain at least one symbol or column ( column_bool, column_i64, column_f64, column_str, column_ts).
• Symbols must always appear before columns.
• A row must be terminated with either at or at_now.

This diagram might help:

Buffer method calls, Serialized ILP types and QuestDB types

Buffer Method	Serialized as ILP type (Click on link to see possible casts)
symbol	SYMBOL
column_bool	BOOLEAN
column_i64	INTEGER
column_f64	FLOAT
column_str	STRING
column_ts	TIMESTAMP

QuestDB supports both STRING columns and SYMBOL column types.

To understand the difference refer to the QuestDB documentation, but in short symbols are interned strings that are most suitable for identifiers that you expect to be repeated throughout the column.

Inserting NULL values

To insert a NULL value, skip the symbol or column for that row.

Recovering from validation errors

If you want to recover from potential validation errors, you can use the set_marker method to track a last known good state, append as many rows or parts of rows as you like and then call clear_marker on success.

If there was an error in one of the table names or other, you can use the rewind_to_marker method to go back to the marked last known good state.

Implementations

impl Buffer

pub fn new() -> Self

Construct an instance with a max_name_len of 127, which is the same as the QuestDB default.

pub fn with_max_name_len(max_name_len: usize) -> Self

Construct with a custom maximum length for table and column names.

This should match the cairo.max.file.name.length setting of the QuestDB instance you’re connecting to.

If the server does not configure it the default is 127 and you might as well call new.

pub fn reserve(&mut self, additional: usize)

Pre-allocate to ensure the buffer has enough capacity for at least the specified additional byte count. This may be rounded up. This does not allocate if such additional capacity is already satisfied. See: capacity.

pub fn len(&self) -> usize

Number of bytes accumulated in the buffer.

pub fn is_empty(&self) -> bool

pub fn capacity(&self) -> usize

Number of bytes that can be written to the buffer before it needs to resize.

pub fn as_str(&self) -> &str

Inspect the contents of the buffer.

pub fn set_marker(&mut self) -> Result<()>

Mark a rewind point. This allows undoing accumulated changes to the buffer for one or more rows by calling rewind_to_marker. Any previous marker will be discarded. Once the marker is no longer needed, call clear_marker.

pub fn rewind_to_marker(&mut self) -> Result<()>

Undo all changes since the last set_marker call.

As a side-effect, this also clears the marker.

pub fn clear_marker(&mut self)

Discard any marker as may have been set by set_marker.

Idempodent.

pub fn clear(&mut self)

Reset the buffer and clear contents whilst retaining capacity.

pub fn table<'a, N>(&mut self, name: N) -> Result<&mut Self>where N: TryInto<TableName<'a>>, Error: From<N::Error>,

Begin recording a row for a given table.

buffer.table("table_name")?;

or

use questdb::ingress::TableName;

let table_name = TableName::new("table_name")?;
buffer.table(table_name)?;

pub fn symbol<'a, N, S>(&mut self, name: N, value: S) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, S: AsRef<str>, Error: From<N::Error>,

Record a symbol for a given column.

buffer.symbol("col_name", "value")?;

or

let value: String = "value".to_owned();
buffer.symbol("col_name", value)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.symbol(col_name, "value")?;

pub fn column_bool<'a, N>(&mut self, name: N, value: bool) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record a boolean value for a column.

buffer.column_bool("col_name", true)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_bool(col_name, true)?;

pub fn column_i64<'a, N>(&mut self, name: N, value: i64) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record an integer value for a column.

buffer.column_i64("col_name", 42)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_i64(col_name, 42);

pub fn column_f64<'a, N>(&mut self, name: N, value: f64) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record a floating point value for a column.

buffer.column_f64("col_name", 3.14)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_f64(col_name, 3.14)?;

pub fn column_str<'a, N, S>(&mut self, name: N, value: S) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, S: AsRef<str>, Error: From<N::Error>,

Record a string value for a column.

buffer.column_str("col_name", "value")?;

or

let value: String = "value".to_owned();
buffer.column_str("col_name", value)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_str(col_name, "value")?;

pub fn column_ts<'a, N, T>(&mut self, name: N, value: T) -> Result<&mut Self>where N: TryInto<ColumnName<'a>>, T: TryInto<Timestamp>, Error: From<N::Error> + From<T::Error>,

Record a timestamp for a column.

use questdb::ingress::TimestampMicros;
buffer.column_ts("col_name", TimestampMicros::now())?;

or

use questdb::ingress::TimestampMicros;

buffer.column_ts("col_name", TimestampMicros::new(1659548204354448))?;

or

use questdb::ingress::TimestampMicros;
use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_ts(col_name, TimestampMicros::now())?;

or you can also pass in a TimestampNanos.

Note that both TimestampMicros and TimestampNanos can be constructed easily from either chrono::DateTime and std::time::SystemTime.

This last option requires the chrono_timestamp feature.

pub fn at<T>(&mut self, timestamp: T) -> Result<()>where T: TryInto<Timestamp>, Error: From<T::Error>,

Terminate the row with a specified timestamp.

use questdb::ingress::TimestampNanos;
buffer.at(TimestampNanos::now())?;

or

use questdb::ingress::TimestampNanos;

buffer.at(TimestampNanos::new(1659548315647406592))?;

You can also pass in a TimestampMicros.

Note that both TimestampMicros and TimestampNanos can be constructed easily from either chrono::DateTime and std::time::SystemTime.

pub fn at_now(&mut self) -> Result<()>

Terminate the row with a server-specified timestamp.

This is NOT equivalent to calling at with the current time. There’s a trade-off: Letting the server assign the timestamp can be faster since it a reliable way to avoid out-of-order operations in the database for maximum ingestion throughput.

On the other hand, it removes the ability to deduplicate rows.

In almost all cases, you should prefer at over this method.

buffer.at_now()?;

The QuestDB instance will set the timestamp once it receives the row. If you’re flushing infrequently, the timestamp assigned by the server may drift significantly from when the data was recorded in the buffer.

Trait Implementations

impl Clone for Buffer

fn clone(&self) -> Buffer

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Buffer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Buffer

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq<Buffer> for Buffer

fn eq(&self, other: &Buffer) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Buffer