Crate kdb_codec 

kdb_codec - Kdb+ IPC Codec Library

This library provides a codec for handling the kdb+ IPC (Inter-Process Communication) wire protocol. It focuses on encoding and decoding kdb+ data types for communication with q/kdb+ processes.

Features

• IPC Codec: Tokio-based codec implementation for kdb+ IPC protocol
• QStream: High-level async client for connecting to q/kdb+ processes
• Compression Control: Explicit control over compression behavior (Auto, Always, Never)
• Header Validation: Configurable validation strictness for incoming messages
• Type Safety: Strong typing for kdb+ data types
• Multiple Connection Methods: TCP, TLS, and Unix Domain Socket support

Security Constants

The library provides default limits to prevent attacks, which can be customized via the KdbCodec builder or configuration methods:

• MAX_LIST_SIZE: 100 million elements (default for max_list_size)
• MAX_RECURSION_DEPTH: 100 levels (default for max_recursion_depth)
• MAX_MESSAGE_SIZE: 256 MB (default for max_message_size)
• MAX_DECOMPRESSED_SIZE: 512 MB (default for max_decompressed_size)

These defaults are based on kdb+ database limits documented at: https://www.timestored.com/kdb-guides/kdb-database-limits

These can be adjusted per codec instance using:

use kdb_codec::*;

let codec = KdbCodec::builder()
    .max_list_size(5_000_000)  // Custom limit
    .max_recursion_depth(50)    // Custom depth
    .build();

Usage

Basic Example

use kdb_codec::*;
use tokio::net::TcpStream;
use tokio_util::codec::Framed;
use futures::{SinkExt, StreamExt};

#[tokio::main]
async fn main() -> Result<()> {
    let stream = TcpStream::connect("127.0.0.1:5000").await?;
    let mut framed = Framed::new(stream, KdbCodec::new(true));
     
    let query = K::new_symbol("1+1".to_string());
    framed.feed(KdbMessage::new(qmsg_type::synchronous, query)).await?;
    framed.flush().await?;
     
    if let Some(Ok(response)) = framed.next().await {
        println!("Result: {}", response.payload);
    }
    Ok(())
}

Compression Control

use kdb_codec::*;

// Auto mode (default): compress large messages on remote connections only
let codec = KdbCodec::new(false);

// Using builder pattern (recommended)
let codec = KdbCodec::builder()
    .is_local(false)
    .compression_mode(CompressionMode::Always)
    .validation_mode(ValidationMode::Strict)
    .build();

// Using builder to disable compression
let codec = KdbCodec::builder()
    .is_local(false)
    .compression_mode(CompressionMode::Never)
    .validation_mode(ValidationMode::Strict)
    .build();

Header Validation

use kdb_codec::*;

// Strict mode (default): reject invalid headers
let codec = KdbCodec::builder()
    .is_local(false)
    .compression_mode(CompressionMode::Auto)
    .validation_mode(ValidationMode::Strict)
    .build();

// Using builder pattern for lenient validation
let codec = KdbCodec::builder()
    .validation_mode(ValidationMode::Lenient)
    .build();

Type Mapping

Note: This table reflects the types currently supported by this library’s IPC encoder/decoder. Not every q/kdb+ datatype is supported yet (e.g., enums, foreign objects, and some function/derived types).

All types are expressed as K struct. The table below shows the input types of each q type:

q	Rust
bool	bool
GUID	[u8; 16]
byte	u8
short	i16
int	i32
long	i64
real	f32
float	f64
char	char
symbol	String
timestamp	chrono::DateTime<Utc>
month	chrono::NaiveDate
date	chrono::NaiveDate
datetime	chrono::DateTime<Utc>
timespan	chrono::Duration
minute	chrono::Duration
second	chrono::Duration
time	chrono::Duration
list	Vec<Item>
compound list	Vec<K>
table	Vec<K>
dictionary	Vec<K>
null	()

Environmental Variables

• KDBPLUS_ACCOUNT_FILE: Credential file for acceptors (format: username:sha1_password)
• KDBPLUS_TLS_KEY_FILE and KDBPLUS_TLS_KEY_FILE_SECRET: TLS certificate files
• QUDSPATH: Optional path for Unix domain socket abstract namespace

Re-exports

pub use error::Error;

pub use codec::*;

Modules

codec

KDB+ Codec

error

This module provides error implementation.

qattribute

This module provides a list of q attributes. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qattribute:: prefix, e.g., qattribute::UNIQUE.

qinf

This module provides a list of q infinite values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.

qinf_base

This module provides a list of q null values. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qinf_base::J.

qmsg_type

Message types for kdb+ IPC protocol

qninf

This module provides a list of q negative infinite values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.

qninf_base

This module provides a list of q null values. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qninf_base::I.

qnull

This module provides a list of q null values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.

qnull_base

This module provides a list of underlying null values of q objects. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull_base::F.

qtype

This module provides a list of q types. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qtype:: prefix, e.g., qtype::BOOL_LIST.

Macros

k

Main macro for creating K objects with simplified syntax.

Structs

K

Struct representing q object.

QStream

Stream to communicate with q/kdb+.

QStreamBuilder

Use builder syntax to set the inputs and finish with build().

Enums

ConnectionMethod

Connection method to q/kdb+.

Constants

KDB_DAY_OFFSET

KDB_MONTH_OFFSET

KDB_TIMESTAMP_OFFSET

MAX_DECOMPRESSED_SIZE

Maximum allowed decompressed message size in bytes (512 MB)

MAX_LIST_SIZE

Maximum allowed list size during deserialization (100 million elements)

MAX_MESSAGE_SIZE

Maximum allowed message size in bytes (256 MB)

MAX_RECURSION_DEPTH

Maximum recursion depth for nested structures (100 levels)

ONE_DAY_MILLIS

ONE_DAY_NANOS

kdb+ offset constants

Traits

Query

Feature of query object.

Functions

handshake

Send a credential and receive a common capacity.

q_date_to_date

Convert q month (elapsed time in days since 2000.01.01) into Date<Utc>.

q_datetime_to_datetime

Convert q datetime (elapsed time in days with glanularity of milliseconds since 2000.01.01T00:00:00) into DateTime<Utc>.

q_minute_to_duration

Convert q minute into Duration.

q_month_to_date

Convert q month (elapsed time in months since 2000.01.01) into Date<Utc>.

q_second_to_duration

Convert q second into Duration.

q_time_to_duration

Convert q time into Duration.

q_timespan_to_duration

Convert q timespan into Duration.

q_timestamp_to_datetime

Convert q timestamp (elapsed time in nanoseconds since 2000.01.01D00:00:00) into DateTime<Utc>.

Type Aliases

C

q type denoting char.

E

q type denoting real.

F

q type denoting float and datetime.

G

q type denoting bool and byte.

H

q type denoting short.

I

q type denoting int and its compatible types (month, date, minute, second and time) of q.

J

q type denoting long and its compatible types (timestamp and timespan) of q.

Result

S

q type denoting symbol and string.

U

q type denoting GUID.