Struct filetrack::TrackedReader

pub struct TrackedReader { /* private fields */ }

Structure that implements Read, ReadBuf and Seek while working with persistent offset in up to two underlying logrotated files. External file is used to persist offset across restarts.

Usage

Instantiate TrackedReader by passing it a path to logfile intended for reading as well as a path to file used as a registry for persistent offset storage. Created structure can be used where implementation of Read or BufRead is expected. Additionally, limited Seek implementation is provided (see Limitations for more info).

let mut reader = TrackedReader::new("examples/file.txt", "examples/registry")?;
match reader.read_line(&mut input)? {
    0 => println!("reached end of file"),
    _ => println!("read line: `{}`", input.trim_end()),
};

Cleanup

There are two distinct ways to perform cleanup for this structure:

• explicit by calling .close(). This will allow you to handle any errors that may happen in the process
• implicitly by relying on Drop. Note that errors generated while working with the filesystem cannot be handled and will cause a panic in this case.

Working principles

To maintain offset in a file across restarts, separate “registry” file is used for persistence. Inode is stored additionally to offset, which allows to keep reading log file in case it was logrotate’d. During intialization, inode of file to be read is compared to previously known and if it differs, it means that file was rotated and a search for original file is performed by checking a file identified by path appended by .1 (eg. mail.log and mail.log.1) and so on. After that you are given a file-like structure that allows buffered reading and seeking in up to specified number of files files.

Limitations

• You can only expect this to work if logrotation happened not more than the number you specified as search_depth. This means that if you are creating a log processor for example, it should be run frequently enough to keep up with logs that are written and rotated.

• Due to simple scheme of persistence, we cannot seek back into rotated file version after saving state while reading from current log file. This means that if your program must do some conditional seeking in a file, you should perform any pointer rollback before performing final save (done by .close() or Drop). Overall, this library is intended to be used for mostly forward reading of log files.

Implementations

impl TrackedReader

pub fn new( filepath: impl AsRef<Path>, registry: impl AsRef<Path> ) -> Result<Self, TrackedReaderError>

Creates a new TrackedReader possibly loading current offset from a registry file. On a first execution registry file most likely will not exist and in that case it will be created with zero offset.

Arguments

• filepath - a path to log file to be read. TrackedReader will additionally search for logrotated file under {filepath}.1
• registry - path to registry file used to persist offset and other metadata

pub fn with_search_depth( filepath: impl AsRef<Path>, registry: impl AsRef<Path>, search_depth: usize ) -> Result<Self, TrackedReaderError>

Like ::new but allows specifying how many rotated items to check

search_depth of 2 means that apart from log file we will check for log.1 and log.2

pub fn persist(&mut self) -> Result<()>

Explicitly save current state into registry file and return any errors generated

pub fn close(self) -> Result<()>

Explicitly finalize structure, returning any errors that were produced in the process. Alternative to relying on Drop.

pub fn get_persistent_state(&self) -> State

Get current state for possible manual handling

Methods from Deref<Target = InodeAwareReader>

pub fn get_persistent_offset(&self) -> InodeAwareOffset

Get offset that can be used across restarts and log rotations

pub fn seek_persistent(&mut self, offset: InodeAwareOffset) -> Result<()>

Seek by persistent offset

Will return NotFound if inode does not exist

pub fn get_inodes(&self) -> &[u64]

Get slice of inodes for current execution

pub fn get_current_inode(&self) -> u64

Get inode of an item that is currently read

pub fn get_item_index_by_inode(&self, inode: u64) -> Option<usize>

Search for item index by given inode

pub fn compare_offsets( &self, first: InodeAwareOffset, second: InodeAwareOffset ) -> Option<Ordering>

Compare two offsets as if they were pointing into one large buffer. Returns None if any of the offsets do not belong to underlying files

Methods from Deref<Target = Multireader<BufReader<File>>>

pub fn get_global_offset(&self) -> u64

Offset amoung all underlying items

pub fn get_local_offset(&self) -> u64

Offset inside current item

pub fn len(&self) -> usize

Number of underlying items

pub fn get_current_item_index(&self) -> usize

index of an item that is currently read

pub fn get_total_size(&mut self) -> Result<u64>

Get total size of underlying items

Computes total size of underlying items. This method requires mut ref and returns io::Result because we need to seek inside last item to determine its size at the moment of call

pub fn seek_current_item(&mut self, pos: SeekFrom) -> Result<u64>

Seek current underlying reader properly updating any internal state

Returns current local offset after seek

pub fn seek_to_item_start(&mut self, item_index: usize) -> Result<u64>

pub fn seek_by_local_index( &mut self, item_index: usize, pos: SeekFrom ) -> Result<u64>

Seek globally by providing local pos inside item at index item_index

Provided pos must be inside indexed item. Returns current local offset.

pub fn get_current_item_size(&self) -> Option<u64>

Returns item size of item. If it is last, returns None instead

To determine size of last item, use get_last_item_size

pub fn get_bytes_before_current_item(&self) -> u64

Computes global offset from which current item starts

pub fn get_last_item_size(&mut self) -> Result<u64>

Computes last item size

Last file in this reader may still be written into, so this number may soon become invalid

Trait Implementations

impl Deref for TrackedReader

type Target = InodeAwareReader

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl DerefMut for TrackedReader

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

impl Drop for TrackedReader

Executes destructor. If .close() was not called previously, will write state to disk, possibly panicking if any error happens. If panic is not what you want, use .close() and handle errors manually instead.

fn drop(&mut self)

Executes the destructor for this type.