Struct s4lib::readers::blockreader::BlockReader

pub struct BlockReader { /* private fields */ }

A BlockReader reads a file in BlockSz byte-sized Blocks. It interfaces with the filesystem. It handles reading from specialized storage files like compressed files (e.g. .gz, .xz) and archive files (e.g .tar). BlockReader handles the run-time in-memory storage the Blocks of data it reads.

A BlockReader uses the passed FileType to determine how to handle files. This includes reading bytes from files (e.g. .log), compressed files (e.g. .gz, .xz), and archive files (e.g. .tar).

One BlockReader corresponds to one file. For archive files and compressed files, one BlockReader handles only one file within the archive or compressed file.

A BlockReader does not know about chars, only bytes u8.

XXX: not a rust “Reader”; does not implement trait Read.

Implementations

impl BlockReader

Implements the BlockReader.

pub fn new( path: FPath, filetype: FileType, blocksz_: BlockSz ) -> Result<BlockReader>

Create a new BlockReader.

Opens the file at path. Configures settings based on passed filetype.

NOTE: Dissimilar to other *Readers::new, this new function may read some bytes from the file, e.g. gzip header, e.g. tar header, to determine the file type and set other initial values.

pub const fn path(&self) -> &FPath

Return a reference to self.path.

pub const fn filesz(&self) -> FileSz

Return the file size in bytes.

For compressed or archived files, returns the original file size (uncompressed or unarchived) as found in the header.

An exception is .xz files, which are entirely read during function new, and the file size determined from the uncompressed bytes. See Issue #12.

For plain files, returns the file size reported by the filesystem.

Users of this struct should always use this instead of accessing self.filesz or self.filesz_actual directly.

pub const fn blocksz(&self) -> BlockSz

Return a copy of self.blocksz

pub const fn filetype(&self) -> FileType

Return a copy of self.filetype. The FileType enum determines various behaviors around opening and reading from the file.

pub fn metadata(&self) -> Metadata

Return a copy of self.file_metadata.

pub fn mtime(&self) -> SystemTime

Get the best available “Modified Datetime” file attribute available.

For compressed or archived files, use the copied header-embedded “MTIME”. If the embedded “MTIME” is not available then return the encompassing file’s “MTIME”.

For example, if archived file syslog within archive logs.tar does not have a valid “MTIME”, then return the file system attribute “modified time” for logs.tar.

pub const fn block_offset_at_file_offset( fileoffset: FileOffset, blocksz: BlockSz ) -> BlockOffset

Return nearest preceding BlockOffset for given FileOffset.

pub const fn file_offset_at_block_offset( blockoffset: BlockOffset, blocksz: BlockSz ) -> FileOffset

See BlockReader::file_offset_at_block_offset.

pub const fn file_offset_at_block_offset_self( &self, blockoffset: BlockOffset ) -> FileOffset

Return FileOffset (byte offset) at given BlockOffset.

pub const fn file_offset_at_block_offset_index( blockoffset: BlockOffset, blocksz: BlockSz, blockindex: BlockIndex ) -> FileOffset

Return FileOffset (byte offset) at BlockOffset + BlockIndex.

pub const fn fileoffset_last(&self) -> FileOffset

Get the last byte FileOffset (index) of the file (inclusive).

pub const fn block_index_at_file_offset( fileoffset: FileOffset, blocksz: BlockSz ) -> BlockIndex

Return BlockIndex (byte offset into a Block) for the Block that corresponds to the passed FileOffset.

pub const fn block_index_at_file_offset_self( &self, fileoffset: FileOffset ) -> BlockIndex

Return BlockIndex (byte offset into a Block) for the Block that corresponds to the passed FileOffset.

pub const fn count_blocks(filesz: FileSz, blocksz: BlockSz) -> Count

Return Count of Blocks in a file.

Equivalent to the last BlockOffset + 1.

Not a count of Blocks that have been read; the calculated count of Blocks based on the FileSz.

pub fn blocksz_at_blockoffset(&self, blockoffset: &BlockOffset) -> BlockSz

Specific BlockSz (size in bytes) of block at BlockOffset.

This should be self.blocksz for all Blocks except the last, which should be >0 and <=self.blocksz

pub fn last_blockindex_at_blockoffset( &self, blockoffset: &BlockOffset ) -> BlockIndex

Return last valid BlockIndex for block at the BlockOffset.

pub const fn blockoffset_last(&self) -> BlockOffset

The last valid BlockOffset for the file (inclusive).

In other words, expected largest BlockOffset value for the file. Independent of Blocks that have been processed.

pub fn count_blocks_processed(&self) -> Count

Count of blocks read by this BlockReader.

Not always the same as blocks currently stored (those may be dropped during streaming stage).

pub fn count_bytes(&self) -> Count

Count of bytes stored by this BlockReader.

pub fn LRU_cache_enable(&mut self)

Enable internal LRU cache used by read_block.

pub fn LRU_cache_disable(&mut self)

Disable internal LRU cache used by read_block.

pub const fn is_streamed_file(&self) -> bool

Is the current file a “streaming” file? i.e. must the file be read in a linear manner starting from the first byte? Compressed files must be read in this manner.

If true then it is presumed calls to read_block_File will always be in sequential (linear) manner; i.e. only increasing values of the passed blockoffset.

If false then no presumptions are made about the value of blockoffset passed to read_block_File.

pub fn drop_block(&mut self, blockoffset: BlockOffset) -> bool

Proactively drop the Block at BlockOffset. For “streaming stage”.

The caller must know what they are doing!

pub fn read_block(&mut self, blockoffset: BlockOffset) -> ResultS3ReadBlock

Read a Block of data of max size self.blocksz from the file.

A successful read returns Found(BlockP).

When at or past the end of the file and no data was read, returns Done.

All other File and std::io errors are propagated to the caller in Err.

pub fn read_data_to_buffer( &mut self, fileoffset_beg: FileOffset, fileoffset_end: FileOffset, oneblock: bool, buffer: &mut [u8] ) -> ResultReadDataToBuffer

Read data from the file into the passed buffer. Calls private function read_data. When successful, Found contains number of bytes read into the passed buffer. Data is read inclusive of fileoffset_beg and exclusive of fileoffset_end.

Argument oneblock is passed to read_data.

pub fn open_tar(path_tar: &Path) -> Result<TarHandle>

Helper function to open a .tar file.

pub fn summary(&self) -> SummaryBlockReader

Trait Implementations

impl Debug for BlockReader

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.