Crate filetrack

Filetrack

Filetrack is a library for persistent reading of logs similar to the mechanisms used in Filebeat and other software alike. It provides a few useful primitives for working with IO and its main intention is to be used for implementation of custom log processors.

• Multireader that lets you work with a list of readers as if you had one single buffer.

let inner_items = vec![Cursor::new(vec![1, 2, 3]), Cursor::new(vec![4, 5])];
// we get result here because Multireader performs seek
// (fallible operation) under the hood to determine sizes
let mut reader = Multireader::new(inner_items)?;
reader.read_to_end(&mut buf)?;
assert_eq!(buf, vec![1, 2, 3, 4, 5]);

• InodeAwareReader that allows working with rotated logs and maintating persistent offset inside them. Scheme of persistence is to be implemented by user.

let mut reader = InodeAwareReader::from_rotated_logs("/var/log/mail.log")?;
reader.seek_persistent(load_state()?)?;
reader.read_exact(& mut buf)?;
save_state(reader.get_persistent_offset())?;

• TrackedReader that allows to read logs or any other content from rotated files with offset persisted across restarts inside a file in case you want a ready-to-use structure.

// running this script will fetch and print new lines on each execution
let mut reader = TrackedReader::new("examples/file.txt", "examples/registry")?;
let mut input = String::new();
loop {
    match reader.read_line(&mut input)? {
        0 => break Ok(()),
        _ => println!("read line: `{}`", input.trim_end()),
    };
}

Modules

• path_utils
  A collection of utility functions for working with paths and filesystem.

Structs

• InodeAwareOffset
  Structure that can be used as persistent offset into rotated logs. See InodeAwareReader for more info.
• InodeAwareReader
  Reader that keeps track of what inode it reads from.
• Multireader
  Structure that provides seeking and reading in a sequence of underlying readables.
• State
  Structure used by TrackedReader for simple file persistence.
• TrackedReader
  Structure that implements Read, ReadBuf and Seek while working with persistent offset in underlying logrotated files. External file is used to persist offset across restarts.

Enums

• StateSerdeError
  Possible errors that could happen while working with persistent state storage.
• TrackedReaderError
  Possible errors that could happen while working with TrackedReader.