kts-analyze 0.1.0

Various APIs on top of the arrow and datafusion crates
Documentation

This is work in progress!

Data analysis for know-thy-shell

Some iterators and processors used by know-thy-shell.

It is unclear what is the right api to use at this time, this is just exploration work.

API

Iterators with related wrappers:

  • FlatTable provides a one level iterator on a TableProvider (instead of partitions and record batches)
  • FlatRelation provides a one level iterator on a Relation

Some simple builders in bland_builder.

A simple top K implementation in top_k.

Usage

This create is used by the kts cli.

This is work in progress!