Struct Buffer

pub struct Buffer { /* private fields */ }

A reusable buffer to prepare a batch of ILP messages.

Example

use questdb::ingress::{Buffer, TimestampMicros, TimestampNanos};

let mut buffer = Buffer::new();

// first row
buffer
    .table("table1")?
    .symbol("bar", "baz")?
    .column_bool("a", false)?
    .column_i64("b", 42)?
    .column_f64("c", 3.14)?
    .column_str("d", "hello")?
    .column_ts("e", TimestampMicros::now())?
    .at(TimestampNanos::now())?;

// second row
buffer
    .table("table2")?
    .symbol("foo", "bar")?
    .at(TimestampNanos::now())?;

Send the buffer to QuestDB using sender.flush(&mut buffer).

Sequential Coupling

The Buffer API is sequentially coupled:

• A row always starts with table.
• A row must contain at least one symbol or column ( column_bool, column_i64, column_f64, column_str, column_ts).
• Symbols must appear before columns.
• A row must be terminated with either at or at_now.

This diagram visualizes the sequence:

Buffer method calls, Serialized ILP types and QuestDB types

Buffer Method	Serialized as ILP type (Click on link to see possible casts)
symbol	SYMBOL
column_bool	BOOLEAN
column_i64	INTEGER
column_f64	FLOAT
column_str	STRING
column_ts	TIMESTAMP

QuestDB supports both STRING and SYMBOL column types.

To understand the difference, refer to the QuestDB documentation. In a nutshell, symbols are interned strings, most suitable for identifiers that are repeated many times throughout the column. They offer an advantage in storage space and query performance.

Inserting NULL values

To insert a NULL value, skip the symbol or column for that row.

Recovering from validation errors

If you want to recover from potential validation errors, call buffer.set_marker() to track the last known good state, append as many rows or parts of rows as you like, and then call buffer.clear_marker() on success.

If there was an error in one of the rows, use buffer.rewind_to_marker() to go back to the marked last known good state.

Implementations

impl Buffer

pub fn new() -> Self

Construct a Buffer with a max_name_len of 127, which is the same as the QuestDB server default.

pub fn with_max_name_len(max_name_len: usize) -> Self

Construct a Buffer with a custom maximum length for table and column names.

This should match the cairo.max.file.name.length setting of the QuestDB instance you’re connecting to.

If the server does not configure it, the default is 127 and you can simply call new.

pub fn reserve(&mut self, additional: usize)

Pre-allocate to ensure the buffer has enough capacity for at least the specified additional byte count. This may be rounded up. This does not allocate if such additional capacity is already satisfied. See: capacity.

pub fn len(&self) -> usize

The number of bytes accumulated in the buffer.

pub fn row_count(&self) -> usize

The number of rows accumulated in the buffer.

pub fn transactional(&self) -> bool

Tells whether the buffer is transactional. It is transactional iff it contains data for at most one table. Additionally, you must send the buffer over HTTP to get transactional behavior.

pub fn is_empty(&self) -> bool

pub fn capacity(&self) -> usize

The total number of bytes the buffer can hold before it needs to resize.

pub fn as_str(&self) -> &str

A string representation of the buffer’s contents. Useful for debugging.

pub fn set_marker(&mut self) -> Result<()>

Mark a rewind point. This allows undoing accumulated changes to the buffer for one or more rows by calling rewind_to_marker. Any previous marker will be discarded. Once the marker is no longer needed, call clear_marker.

pub fn rewind_to_marker(&mut self) -> Result<()>

Undo all changes since the last set_marker call.

As a side-effect, this also clears the marker.

pub fn clear_marker(&mut self)

Discard any marker as may have been set by set_marker.

Idempotent.

pub fn clear(&mut self)

Reset the buffer and clear contents whilst retaining capacity.

pub fn table<'a, N>(&mut self, name: N) -> Result<&mut Self>

where N: TryInto<TableName<'a>>, Error: From<N::Error>,

Begin recording a new row for the given table.

buffer.table("table_name")?;

or

use questdb::ingress::TableName;

let table_name = TableName::new("table_name")?;
buffer.table(table_name)?;

pub fn symbol<'a, N, S>(&mut self, name: N, value: S) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, S: AsRef<str>, Error: From<N::Error>,

Record a symbol for the given column. Make sure you record all symbol columns before any other column type.

buffer.symbol("col_name", "value")?;

or

let value: String = "value".to_owned();
buffer.symbol("col_name", value)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.symbol(col_name, "value")?;

pub fn column_bool<'a, N>(&mut self, name: N, value: bool) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record a boolean value for the given column.

buffer.column_bool("col_name", true)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_bool(col_name, true)?;

pub fn column_i64<'a, N>(&mut self, name: N, value: i64) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record an integer value for the given column.

buffer.column_i64("col_name", 42)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_i64(col_name, 42);

pub fn column_f64<'a, N>(&mut self, name: N, value: f64) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, Error: From<N::Error>,

Record a floating point value for the given column.

buffer.column_f64("col_name", 3.14)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_f64(col_name, 3.14)?;

pub fn column_str<'a, N, S>(&mut self, name: N, value: S) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, S: AsRef<str>, Error: From<N::Error>,

Record a string value for the given column.

buffer.column_str("col_name", "value")?;

or

let value: String = "value".to_owned();
buffer.column_str("col_name", value)?;

or

use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_str(col_name, "value")?;

pub fn column_ts<'a, N, T>(&mut self, name: N, value: T) -> Result<&mut Self>

where N: TryInto<ColumnName<'a>>, T: TryInto<Timestamp>, Error: From<N::Error> + From<T::Error>,

Record a timestamp value for the given column.

use questdb::ingress::TimestampMicros;
buffer.column_ts("col_name", TimestampMicros::now())?;

or

use questdb::ingress::TimestampMicros;

buffer.column_ts("col_name", TimestampMicros::new(1659548204354448))?;

or

use questdb::ingress::TimestampMicros;
use questdb::ingress::ColumnName;

let col_name = ColumnName::new("col_name")?;
buffer.column_ts(col_name, TimestampMicros::now())?;

or you can also pass in a TimestampNanos.

Note that both TimestampMicros and TimestampNanos can be constructed easily from either chrono::DateTime and std::time::SystemTime.

This last option requires the chrono_timestamp feature.

pub fn at<T>(&mut self, timestamp: T) -> Result<()>

where T: TryInto<Timestamp>, Error: From<T::Error>,

Complete the current row with the designated timestamp. After this call, you can start recording the next row by calling Buffer::table again, or you can send the accumulated batch by calling Sender::flush or one of its variants.

use questdb::ingress::TimestampNanos;
buffer.at(TimestampNanos::now())?;

or

use questdb::ingress::TimestampNanos;

buffer.at(TimestampNanos::new(1659548315647406592))?;

You can also pass in a TimestampMicros.

Note that both TimestampMicros and TimestampNanos can be constructed easily from either chrono::DateTime and std::time::SystemTime.

pub fn at_now(&mut self) -> Result<()>

Complete the current row without providing a timestamp. The QuestDB instance will insert its own timestamp.

Letting the server assign the timestamp can be faster since it reliably avoids out-of-order operations in the database for maximum ingestion throughput. However, it removes the ability to deduplicate rows.

This is NOT equivalent to calling Buffer::at with the current time: the QuestDB server will set the timestamp only after receiving the row. If you’re flushing infrequently, the server-assigned timestamp may be significantly behind the time the data was recorded in the buffer.

In almost all cases, you should prefer the Buffer::at function.

After this call, you can start recording the next row by calling Buffer::table again, or you can send the accumulated batch by calling Sender::flush or one of its variants.

buffer.at_now()?;

Trait Implementations

impl Clone for Buffer

fn clone(&self) -> Buffer

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Buffer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Buffer

fn default() -> Self

Returns the “default value” for a type.