Struct drain_rs::DrainTree

pub struct DrainTree { /* fields omitted */ }

Main drain algorithm implementation Contains the structure of the drain prefix tree along with configuration options

Implementations

impl DrainTree

pub fn new() -> Self

Creates new DrainTree struct with default values

pub fn max_depth(self, max_depth: u16) -> Self

How deep should the tree be allowed to grow The deeper the tree, the more specific the clusters, but also the more space + time used for clustering

pub fn max_children(self, max_children: u16) -> Self

How many children does each inner node allow? Once the number of max_children is reached, the inner node starts putting unmatched tokens into the <*> (wildcard) branch.

pub fn min_similarity(self, min_similarity: f32) -> Self

For a log to be added to a cluster, how similar does it need to be with the current template?

pub fn filter_patterns(self, filter_patterns: Vec<&str>) -> Self

Token filtering and name replacement for tokens If you set this, be sure to call build_patterns so that they can be compiled before use.

Examples:

let mut g = grok::Grok::with_patterns();
let filter_patterns = vec![
        "blk_(|-)[0-9]+",     //blockid
       "%{IPV4:ip_address}", //IP
        "%{NUMBER:number}",   //Num
    ];
let drain_tree = drain_rs::DrainTree::new().filter_patterns(filter_patterns).build_patterns(&mut g);

pub fn log_pattern(self, overall_pattern: &str, drain_field: &str) -> Self

The overall log pattern and which extracted field to cluster most logging formats have a well known format mixed with semi-structured text This allows you to set the well known format and then only cluster on the semi-structured text. If you set this, be sure to call build_patterns so that they can be compiled before use.

Examples:

let mut g = grok::Grok::with_patterns();
let filter_patterns = vec![
        "blk_(|-)[0-9]+",     //blockid
       "%{IPV4:ip_address}", //IP
        "%{NUMBER:number}",   //Num
    ];
let mut drain = drain_rs::DrainTree::new()
        // HDFS log pattern, variable format printout in the content section
        .log_pattern("%{NUMBER:date} %{NUMBER:time} %{NUMBER:proc} %{LOGLEVEL:level} %{DATA:component}: %{GREEDYDATA:content}", "content")
        .build_patterns(&mut g);

pub fn build_patterns(self, grok: &mut Grok) -> Self

Build the patterns that have been supplied in log_pattern and filter_patterns

pub fn log_group(&self, log_line: &str) -> Option<&LogCluster>

Grab the log group for the given log line if it exists. This does NOT modify the underlying tree.

let drain = drain_rs::DrainTree::new();
assert!(drain.log_group("[Sun Dec 04 04:51:08 2005] [notice] jk2_init() Found child 6725 in scoreboard slot 10").is_none());

pub fn add_log_line(&mut self, log_line: &str) -> Option<&LogCluster>

Add a new log line to the overall tree and return the current reference to the created/modified log cluster

Over time, the log clusters could change as new log lines are added.

let mut drain = drain_rs::DrainTree::new();
assert!(drain.add_log_line("[Sun Dec 04 04:51:08 2005] [notice] jk2_init() Found child 6725 in scoreboard slot 10").is_some());

pub fn log_groups(&self) -> Vec<&LogCluster>

Grab all the current log clusters

let mut g = grok::Grok::with_patterns();
let mut drain = drain_rs::DrainTree::new()
        // HDFS log pattern, variable format printout in the content section
        .log_pattern("\\[%{TIMESTAMP_ISO8601:timestamp}\\] - %{WORD:log_level}\\s*. %{GREEDYDATA:log_message}", "log_message")
        .build_patterns(&mut g);
assert!(drain.add_log_line("2015-07-29 19:04:29,071 - WARN  [SendWorker:188978561024:QuorumCnxManager$SendWorker@688] - Send worker leaving thread").is_some());
assert!(drain.add_log_line("2015-08-07 07:27:46,402 - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@542] - Notification: 3 (n.leader), 0x700000197 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x7 (n.peerEPoch), LOOKING (my state)").is_some());
assert_eq!(drain.log_groups().len(), 2);

Trait Implementations

impl Debug for DrainTree

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DrainTree

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DrainTree

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl Display for DrainTree

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for DrainTree

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.