# Datafusion-Bigtable
Bigtable data source for [Apache Arrow Datafusion](https://github.com/apache/arrow-datafusion)
## Run SQL on Bigtable
This crate implements Bigtable data source and Executor for Datafusion. It is built on top of gRPC client [tonic](https://github.com/hyperium/tonic).
## Quick Start
```
let bigtable_datasource = BigtableDataSource::new(
"emulator".to_owned(), // project
"dev".to_owned(), // instance
"weather_balloons".to_owned(), // table
"measurements".to_owned(), // column family
vec!["_row_key".to_owned()], // table_partition_cols
"#".to_owned(), // table_partition_separator
vec![Field::new("pressure", DataType::Utf8, false)], // qualifiers
true, // only_read_latest
).await.unwrap();
let mut ctx = ExecutionContext::new();
ctx.register_table("weather_balloons", Arc::new(bigtable_datasource)).unwrap();
ctx.sql("SELECT \"_row_key\", pressure, \"_timestamp\" FROM weather_balloons where \"_row_key\" = 'us-west2#3698#2021-03-05-1200'").await?.collect().await?;
```
## Roadmap
### Bigtable
- ✅ UTF8 string
- ✅ 64-bit big-endian signed integer
### SQL
- ✅ select by `"_row_key" =`
- ✅ select by `"_row_key" IN`
- ✅ select by `"_row_key" BETWEEN`
- ✅ select by composite row keys `=`
- ✅ select by composite row keys `IN`
- ✅ select by composite row keys `BETWEEN` (only supported by last table_partition_cols)
### General
- ✅ Projection pushdown
- [ ] Predicate push down
+ [Value range](https://cloud.google.com/bigtable/docs/using-filters#value-range)
+ [Value Regex](https://cloud.google.com/bigtable/docs/using-filters#value-regex)
+ [Timestamp range](https://cloud.google.com/bigtable/docs/using-filters#timestamp-range)
- [ ] Multi Thread or Partition aware execution
- [ ] Production ready Bigtable SDK in Rust
Note: datafusion-bigtable provides the physical Executor for Datafusion. Any aggregation, group by, join are implemented and handled by Datafusion.