Expand description
M lexing and reference resolution.
Tokenizes the Power Query (M) expressions carried by the AST — table
partitions and model-level shared expressions — and extracts the object
references they contain: enough to build the dependency graph, not a full
parse tree. See crate::m::lexer for the tokenizer and
crate::m::refs for the extraction rules.
The design rule is the same conservatism as the DAX side: zero false
negatives. Anything the extractor cannot rule out is treated as a reference
— an unqualified [Name] or a harvested column string names every
column of that name in the model, because M string arguments carry no row
context a lexer could consult. Over-marking costs precision; under-marking
deletes live code. Resolution failures are data, never errors: the graph
layer decides what an unresolvable reference means.
What a reference is worth is the graph layer’s call, and it is not
uniform: Power Query reads the source and produces the columns the model
maps onto, so a column named in M is the column’s supply chain, not a
consumer — unloading it cannot break refresh. A table or shared expression
named in M is different: deleting it deletes the query this expression
reads or joins, which breaks refresh. crate::graph encodes that split.
use ripbi_core::{Column, ModelIndex, Table, TabularDatabase, m};
let db = TabularDatabase {
tables: vec![Table {
name: "Sales".to_string(),
columns: vec![
Column { name: "Amount".to_string(), ..Default::default() },
Column { name: "Region".to_string(), ..Default::default() },
],
..Default::default()
}],
..Default::default()
};
let index = ModelIndex::build(&db);
// The expanded column name resolves to the model column.
let mut refs = m::references(r#"Table.ExpandTableColumn(Source, "Amount")"#);
let binding = m::bind(&db, &index, refs.remove(0));
assert_eq!(binding.targets().len(), 1);
// An unknown name is data, not an error.
let refs = m::references(r#"Table.SelectColumns(Source, {"Gone"})"#);
assert!(m::bind(&db, &index, refs.into_iter().next().unwrap()).is_unresolved());Re-exports§
pub use lexer::Token;pub use lexer::TokenKind;pub use lexer::tokenize;pub use refs::RawRef;pub use refs::references;pub use refs::unescape_name;
Modules§
- lexer
- M tokenizer, following the lexical grammar of the official Power Query specification (https://learn.microsoft.com/en-us/powerquery-m/m-spec-lexical-structure) with microsoft/powerquery-parser (MIT) as the battle-tested reference for the fiddly corners. Reduced to what reference extraction needs — no line-mode bookkeeping, no error positions, no formatter.
- refs
- Reference extraction over the M token stream.