Struct FileSlicer 

pub struct FileSlicer { /* private fields */ }

Implementations

impl FileSlicer

pub fn try_from_path(in_path: PathBuf) -> Result<Self>

Try creating a new FileSlicer from a relative or absolute path to the fixed-length file that is to be sliced.

Errors

This function can return an error for the following reasons:

• Any I/O error was returned when trying to open the path as a file.
• Could not read the metadata of the file at the path.

pub fn from_path(in_path: PathBuf) -> Self

Create a new [FixedLengthFileSlicer] from a relative or absolute path to the fixed-length file that is to be sliced.

Panics

This function can panic for the following reasons:

• Any I/O error was returned when trying to open the path as a file.
• Could not read the metadata of the file at the path.

pub fn bytes_to_read(&self) -> usize

Get the total number of bytes to read.

pub fn remaining_bytes(&self) -> usize

Get the number of remaining bytes to read.

pub fn set_remaining_bytes(&mut self, remaining_bytes: usize)

Set the number of remaining bytes to read.

pub fn bytes_processed(&self) -> usize

Get the total number of processed bytes.

pub fn set_bytes_processed(&mut self, bytes_processed: usize)

Set the total number of processed bytes.

pub fn bytes_overlapped(&self) -> usize

Get the total number of overlapped bytes (due to sliding window).

pub fn set_bytes_overlapped(&mut self, bytes_overlapped: usize)

Set the total number of overlapped bytes.

pub fn try_read_to_buffer(&mut self, buffer: &mut [u8]) -> Result<()>

Try and read from the buffered reader into the provided buffer. This function reads enough bytes to fill the buffer, hence, it is up to the caller to ensure that the buffer has the correct and/or wanted capacity.

Errors

If the buffered reader encounters an EOF before completely filling the buffer.

pub fn try_distribute_buffer_chunks_on_workers( &self, buffer: &[u8], thread_workloads: &mut Vec<(usize, usize)>, ) -> Result<()>

Try and evenly distribute the buffer into uniformly sized chunks for each worker thread. This function expects a Vec of usize tuples, representing the start and end byte indices for each worker threads chunk.

Note

This function is optimized to spend as little time as possible looking for valid chunks, i.e., where there are line breaks, and will not look through the entire buffer. This can have an effect on the CPU cache hit-rate, however, this depends on the size of the buffer.

Errors

This function might return an error for the following reasons:

• If the buffer was empty.
• If there were no line breaks in the buffer.

pub fn read_to_buffer(&mut self, buffer: &mut [u8])

Read from the buffered reader into the provided buffer. This function reads enough bytes to fill the buffer, hence, it is up to the caller to ensure that that buffer has the correct and/or wanted capacity.

Panics

If the buffered reader encounters an EOF before completely filling the buffer.

pub fn try_find_last_line_break(&self, bytes: &[u8]) -> Result<usize>

Try and find the last linebreak character in a byte slice and return the index of the character. The function looks specifically for a line-feed (LF) character, represented as ‘\n’ on Unix systems.

Errors

If either the byte slice to search through was empty, or there existed no linebreak character in the byte slice.

pub fn try_find_line_breaks( &self, bytes: &[u8], buffer: &mut Vec<usize>, add_starting_idx: bool, ) -> Result<()>

Try and find all occurances of linebreak characters in a byte slice and push the index of the byte to a provided buffer. The function looks specifically for a line-feed (LF) character, represented as ‘\n’ on Unix systems.

Errors

If the byte slice to search through was empty.

pub fn try_seek_relative(&mut self, bytes_to_seek: i64) -> Result<()>

Try and seek relative to the current position in the buffered reader.

Errors

Seeking to a negative offset will return an error.

pub fn seek_relative(&mut self, bytes_to_seek: i64)

Seek relative to the current position in the buffered reader.

Panics

Seeking to a negative offset will cause the program to panic.

Trait Implementations

impl Slicer for FileSlicer

fn is_done(&self) -> bool

Get whether or not this Slicer is done reading the input file.