Crate filetrack

Filetrack

Filetrack is a library for persistent reading of logs similar to the mechanisms used in Filebeat and other software alike. It provides a few useful primitives for working with IO and its main intention is to be used for implementation of custom log processors.

• Multireader that lets you work with a list of readers as if you had one single buffer

let inner_items = vec![Cursor::new(vec![1, 2, 3]), Cursor::new(vec![4, 5])];
// we get result here because Multireader performs seek
// (fallible operation) under the hood to determine sizes
let mut reader = Multireader::new(inner_items)?;
reader.read_to_end(&mut buf)?;
assert_eq!(buf, vec![1, 2, 3, 4, 5]);

• InodeAwareReader that allows working with rotated logs and maintating persistent offset inside them. Scheme of persistence is to be implemented by user.

let mut reader = InodeAwareReader::from_rotated_logs("/var/log/mail.log")?;
reader.seek_persistent(load_state()?)?;
reader.read_exact(& mut buf)?;
save_state(reader.get_persistent_offset())?;

• TrackedReader that allows to read logs or any other content from rotated files with offset persisted across restarts inside a file in case you want a ready-to-use structure.

// running this program multiple times will output next line on each execution
let mut reader = TrackedReader::new("examples/file.txt", "examples/registry")?;
match reader.read_line(&mut input)? {
    0 => println!("reached end of file"),
    _ => println!("read line: `{}`", input.trim_end()),
};

See documentation for examples and working principles.

Modules

• path_utils

Structs

• InodeAwareOffset
  Structure that can be used as persistent offset into rotated logs. See InodeAwareReader for more info.
• InodeAwareReader
  Reader that keeps track of what inode it reads from
• Multireader
  Structure that provides seeking and reading in a sequence of underlying readables
• State
  Structure used by TrackedReader for simple file persistence
• TrackedReader
  Structure that implements Read, ReadBuf and Seek while working with persistent offset in up to two underlying logrotated files. External file is used to persist offset across restarts.

Enums

• StateSerdeError
  Possible errors that could happen while working with persistent state storage
• TrackedReaderError
  Possible errors that could happen while working with TrackedReader