Skip to main content

Crate fv_value_datafusion

Crate fv_value_datafusion 

Source
Expand description

The fv_value expression dialect as DataFusion expressions.

Translator turns a compiled fv_value::Expr into a datafusion::logical_expr::Expr wherever the dialect’s semantics can be reproduced exactly — so the planner can push it down to a source, vectorize it and reorder it — and leaves the rest to a per-row fallback (EvalUdf). The contract’s conformance vectors, evaluated both natively and through DataFusion, are the test that the two never disagree.

use std::sync::Arc;
use datafusion::arrow::datatypes::{DataType, Field, Schema};
use fv_value_datafusion::Translator;

let schema = Arc::new(Schema::new(vec![
    Field::new("status", DataType::Utf8, true),
    Field::new("speed", DataType::Float64, true),
]));
let expr = fv_value::compile("status == 'ACTIVE' && speed * 1.852 > 100").unwrap();
let t = Translator::new(schema).translate(&expr).unwrap();
assert!(t.exact);
// `t.expr` is a DataFusion `Expr` you can hand to `DataFrame::filter`.

Re-exports§

pub use convert::cell_to_value;
pub use convert::data_type_for;
pub use convert::field_has_value_form;
pub use convert::has_value_form;
pub use convert::values_to_array;
pub use convert::Cells;
pub use convert::Kind;
pub use translate::Options;
pub use translate::Translation;
pub use translate::Translator;
pub use translate::Unsupported;
pub use udf::all_finite;
pub use udf::fallback;
pub use udf::ErrorUdf;
pub use udf::EvalUdf;
pub use udf::FiniteUdf;

Modules§

convert
Arrow ↔ Value conversion for the per-row fallback.
translate
fv_value::Expr → datafusion::logical_expr::Expr, exact or not at all.
udf
The per-row fallback and the two helpers that carry the dialect’s run-time errors into a plan — raising one, and checking a computed number is finite — all as DataFusion scalar UDFs.