Module ipc

Expand description

This ipc module provides an interface to interact with q/kdb+ via IPC. The expected usage is to send a (text) query to q/kdb+ process from Rust client and receive its response. Query to kdb+ is supported in two ways:

text query
functional query which is represented by a compound list of kdb+ (See detail of IPC).

Compression/decompression of messages is also implemented following kdb+ implementation.

As for connect method, usually client interfaces of q/kdb+ do not provide a listener due to its protocol. However, sometimes Rust process is connecting to upstream and q/kdb+ starts afterward or is restarted more frequently. Then providing a listener method is a natural direction and it was achieved here. Following ways are supported to connect to kdb+:

TCP
TLS
Unix domain socket

Furthermore, in order to improve inter-operatability some casting, getter and setter methods are provided.

§Environmentl Variables

This crate uses q-native or crate-specific environmental variables.

KDBPLUS_ACCOUNT_FILE: A file path to a credential file which an acceptor loads in order to manage access from a q client. This file contains a user name and SHA-1 hashed password in each line which are delimited by ':' without any space. For example, a file containing two credentials "mattew:oracle" and "reluctant:slowday" looks like this:
```
 mattew:431364b6450fc47ccdbf6a2205dfdb1baeb79412
 reluctant:d03f5cc1cdb11a77410ee34e26ca1102e67a893c
```
The hashed password can be generated with q using a function .Q.sha1:
```
 q).Q.sha1 "slowday"
 0xd03f5cc1cdb11a77410ee34e26ca1102e67a893c
```
KDBPLUS_TLS_KEY_FILE and KDBPLUS_TLS_KEY_FILE_SECRET: The pkcs12 file and its password which TLS acceptor uses.
QUDSPATH (optional): q-native environmental variable to define an astract namespace. This environmental variable is used by UDS acceptor too. The abstract nameapace will be @${QUDSPATH}/kx.[server process port] if this environmental variable is defined. Otherwise it will be @/tmp/kx.[server process port].

Notes:

Messages will be sent with OS native endian.
When using this crate for a TLS client you need to set two environmental variables KX_SSL_CERT_FILE and KX_SSL_KEY_FILE on q side to make q/kdb+ to work as a TLS server. For details, see the KX website.

§Type Mapping

All types are expressed as K struct which is quite similar to the K struct of api module but its structure is optimized for IPC usage and for convenience to interact with. The table below shows the input types of each q type which is used to construct K object. Note that the input type can be different from the inner type. For example, timestamp has an input type of chrono::DateTime<Utc> but the inner type is i64 denoting an elapsed time in nanoseconds since 2000.01.01D00:00:00.

q	Rust
`bool`	`bool`
`GUID`	`[u8; 16]`
`byte`	`u8`
`short`	`i16`
`int`	`i32`
`long`	`i64`
`real`	`f32`
`float`	`f64`
`char`	`char`
`symbol`	`String`
`timestamp`	`chrono::DateTime<Utc>`
`month`	`chrono::NaiveDate`
`date`	`chrono::NaiveDate`
`datetime`	`chrono::DateTime<Utc>`
`timespan`	`chrono::Duration`
`minute`	`chrono::Duration`
`second`	`chrono::Duration`
`time`	`chrono::Duration`
`list`	`Vec<Item>` (`Item` is a corrsponding type above)
`compound list`	`Vec<K>`
`table`	`Vec<K>`
`dictionary`	`Vec<K>`
`null`	`()`

§Examples

§Client

use kdbplus::qattribute;
use kdbplus::ipc::*;

#[tokio::main(flavor = "multi_thread", worker_threads = 2)]
async fn main() -> Result<()> {
    // Connect to qprocess running on localhost:5000 via UDS
    let mut socket = QStream::connect(ConnectionMethod::UDS, "", 5000_u16, "ideal:person").await?;
    println!("Connection type: {}", socket.get_connection_type());

    // Set remote function with an asynchronous text form message
    socket.send_async_message(&"collatz:{[n] seq:enlist n; while[not n = 1; seq,: n:$[n mod 2; 1 + 3 * n; `long$n % 2]]; seq}").await?;

    // Send a text form emessage synchronously
    let mut result = socket.send_sync_message(&"collatz[12]").await?;
    println!("collatz[12]: {}", result);

    result = socket.send_sync_message(&"collatz[`a]").await?;
    println!("collatz[`a]: {}", result);

    // Send a functional form message synchronously.
    let mut message = K::new_compound_list(vec![
        K::new_symbol(String::from("collatz")),
        K::new_long(100),
    ]);
    result = socket.send_sync_message(&message).await?;
    println!("collatz[100]: {}", result);

    // Modify the message to (`collatz; 20)
    message.pop().unwrap();
    message.push(&K::new_long(20)).unwrap();
    result = socket.send_sync_message(&message).await?;
    println!("collatz[20]: {}", result);

    // Send a functional form message asynchronous query.
    message = K::new_compound_list(vec![
        K::new_string(String::from("show"), qattribute::NONE),
        K::new_symbol(String::from("goodbye")),
    ]);
    socket.send_async_message(&message).await?;

    socket.shutdown().await?;

    Ok(())
}

§Listener

use std::io;
use kdbplus::ipc::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Start listenening over TCP at the port 7000 with authentication enabled.
    let mut socket_tcp = QStream::accept(ConnectionMethod::TCP, "127.0.0.1", 7000).await?;

    // Send a query with the socket.
    let greeting = socket_tcp.send_sync_message(&"string `Hello").await?;
    println!("Greeting: {}", greeting);

    socket_tcp.shutdown().await?;

    Ok(())
}

Then q client can connect to this acceptor with the acceptor’s host, port and the credential configured in KDBPLUS_ACCOUNT_FILE:

q)h:hopen `::7000:reluctant:slowday

Modules§

error: This module provides error implementation.
qinf: This module provides a list of q infinite values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.
qmsg_type: This module provides a list of q message type used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qmsg_type:: prefix, e.g., qmsg_type::asynchronous.
qninf: This module provides a list of q negative infinite values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.
qnull: This module provides a list of q null values set on Rust process and used for IPC. The motivation to contain them in a module is to tie them up as related items rather than scattered values. Hence user should use these indicators with qnull:: prefix, e.g., qnull::FLOAT.

Structs§

K: Struct representing q object.
QStream: Stream to communicate with q/kdb+.

Enums§

ConnectionMethod: Connection method to q/kdb+.

Constants§

KDB_DAY_OFFSET: 2000.01.01 (kdb+ epoch) - 1970.01.01 in day.
KDB_MONTH_OFFSET: 2000.01.01 (kdb+ epoch) - 1970.01.01 in month.
KDB_TIMESTAMP_OFFSET: 2000.01.01 (kdb+ epoch) - 1970.01.01 in nanosecond.
ONE_DAY_MILLIS: 1 day in milli second.
ONE_DAY_NANOS: 1 day in nano second.

Traits§

Query: Feature of query object.

Type Aliases§

C: q type denoting char.
E: q type denoting real.
F: q type denoting float and datetime.
G: q type denoting bool and byte.
H: q type denoting short.
I: q type denoting int and its compatible types (month, date, minute, second and time) of q.
J: q type denoting long and its compatible types (timestamp and timespan) of q.
Result
S: q type denoting symbol and string.
U: q type denoting GUID.

Module ipc

Module ipc Copy item path

§Environmentl Variables

§Type Mapping

§Examples

§Client

§Listener

Modules§

Structs§

Enums§

Constants§

Traits§

Type Aliases§

Module ipc