Skip to main content

Crate tp_lexer

Crate tp_lexer 

Source
Expand description

A tiny no_std filter expression parser and evaluator for Linux tracepoint–like filters.

Features:

  • Numeric comparisons: == != < <= > >= (supports bit mask form: field & 0x10 == 0x10).
  • String/byte comparisons: == != ~ (~ performs glob matching supporting * ? [..] and class negation [!a] / [^a]).
  • Unquoted barewords allowed (comm != bash); quote when spaces or special characters are present ("C Program").
  • Logical operators with precedence: parentheses > comparison > && > || (short‑circuit evaluation).
  • Tri‑state runtime: missing field => Unknown; top‑level Unknown is treated as true (fail‑open), Unknown is true‑neutral for && and false‑neutral for ||.
  • Compile‑time validation of field existence and type.
  • Signed decimal / hex integer literals such as -1, 0x10, and -(0x10).
  • u64 / u128 / i128 / isize / usize fields, plus their NonZero* variants, are supported, but evaluation still uses an internal i64; out-of-range values are truncated with Rust’s integer cast semantics before comparison.
  • no_std + alloc friendly (uses BTreeMap in examples; can evaluate directly over a raw byte buffer via BufContext).

Schema & Field Types: A schema is defined via the schema! macro listing (name, FieldType, offset, length) tuples. Integer types implement FieldClassifier exposing a FIELD_TYPE constant. Bytes fields use FieldType::Bytes.

Example (map based):

extern crate alloc;
use alloc::collections::BTreeMap;
use tp_lexer::{schema, FieldType, compile_with_schema, FieldClassifier};

let schema = schema!(
    "sig"  => (u32::FIELD_TYPE, 0, 4),
    "comm" => (FieldType::Bytes, 4, 16),
);
let compiled = compile_with_schema("sig >= 10 && comm != bash", schema).unwrap();
let mut ctx = BTreeMap::from([
    ("sig".to_string(), "12".to_string()),
    ("comm".to_string(), "sh".to_string()),
]);
assert!(compiled.evaluate(&ctx));

Example (raw buffer):

use tp_lexer::{schema, FieldType, compile_with_schema, BufContext, FieldClassifier};
// Layout: [ sig: u32 (4 bytes) | comm: 12 bytes ASCII | padding ]
let schema = schema!(
    "sig"  => (u32::FIELD_TYPE, 0, 4),
    "comm" => (FieldType::Bytes, 4, 12),
);
let mut buf = [0u8; 32];
// sig = 12
buf[0..4].copy_from_slice(&12u32.to_le_bytes());
// comm = "sh" zero padded
buf[4..6].copy_from_slice(b"sh");
let ctx = BufContext::new(&buf, &schema);
let compiled = compile_with_schema("sig >= 10 && comm != bash", schema).unwrap();
assert!(compiled.evaluate(&ctx));

Tri‑state rationale: treating top‑level Unknown as true prevents inadvertent data loss when a new field is not yet populated. If strict behavior is desired you can wrap your filter: (field != value) && defined(field) by introducing an explicit presence flag in your schema.

Limitations / Non‑Goals:

  • No arithmetic besides bit masking.
  • No unary operators or regex; glob only.
  • Evaluation uses an internal i64 representation. u64/u128/i128/isize/usize fields are accepted, but values outside the i64 range are truncated using Rust’s as i64 integer cast semantics before comparison.
  • Evaluation is left‑associative for chains of && / || (standard short‑circuit).

Safety Notes:

  • Buffer offsets and lengths are trusted; ensure schema matches actual event layout.
  • Integer extraction uses user‑provided ToI64 implementations; undefined behavior inside those implementations is out of scope.

See README for a more thorough integration guide.

Macros§

schema
Helper macro to define a schema inline.

Structs§

BufContext
Runtime context for evaluating against a byte buffer and schema.
Compiled
A compiled expression ready to be evaluated against runtime data.
EvalError
An error produced while evaluating a compiled expression.
ParseError
An error produced while lexing or parsing a filter expression.
Schema
Declares the available fields and their types for compile-time validation.
Span
A span in the input expression, used for diagnostics.

Enums§

FieldType
The type of a field in the schema.

Traits§

Context
Runtime context abstraction used during expression evaluation.
FieldClassifier
A trait to classify fields at compile time.
ToI64
A trait to convert byte slices to i64 values.

Functions§

compile_with_schema
Parses and validates a filter expression against the given Schema.