Struct TgzReader 

pub struct TgzReader<R> { /* private fields */ }

Decompressor for reading .tar.gz files containing embedded chapter information.

Tgz files without chapter information will simply be observed to have a single chapter encompassing their entire contents.

Implementations

impl<R> TgzReader<R>

where R: Read + Seek,

pub fn open(read: R) -> Result<Self>

Parse chapter information from a tgz.

Time complexity: O(c) where c is the number of chapters. In particular, the runtime is not sensitive to the number of entries within a chapter, the compressed size of chapter contents, or the uncompressed size of chapter contents.

pub fn chapters(&self) -> u32

Number of chapters in the tgz.

Time complexity: O(1)

Files without chapter information (i.e. not produced by this crate’s TgzWriter) have 1 chapter.

pub fn next_chapter(&mut self) -> Option<Archive<ChapterReader<'_, R>>>

Sequentially begin reading a single chapter. Returns None if there is no next chapter.

Time complexity: O(1)

Example

This example uses a while let loop to list the contents of every chapter in a tgz.

use chapter_tgz::TgzReader;
use std::fs::File;
use std::io;

fn main() -> io::Result<()> {
    let file = File::open("example.tar.gz")?;
    let mut tgz = TgzReader::open(file)?;
    let mut i = 0;
    while let Some(mut chapter) = tgz.next_chapter() {
        println!("Chapter {i}:");
        for entry in chapter.entries()? {
            let entry = entry?;
            let path = entry.path()?;
            println!("  {}", path.display());
        }
        i += 1;
    }
    Ok(())
}

pub fn jump_to_chapter(&mut self, i: u32) -> Archive<ChapterReader<'_, R>>

Begin reading a single chapter by index.

Time complexity: O(1)

As a side effect, this also affects which chapter next_chapter will pick up next. For example if you call tgz.next_chapter() followed by jump_to_chapter(10) followed by tgz.next_chapter(), the three chapters you get will be chapters 0, 10, 11.

Chapters are indexed starting from 0.

Panics

Requires i < self.chapters().

pub fn independent_read_chapter<'a>( &self, i: u32, ) -> Result<Archive<ChapterReader<'a, R>>>

where R: IndependentRead + 'a,

Begin reading a single chapter in a way that can be parallelized with other reads from the same tgz.

Time complexity: O(1)

This does not affect the next chapter read by next_chapter, which will continue with whichever would have been the next chapter if independent_read_chapter had not been called.

Note the IndependentRead trait bound, which enforces that the underlying reader R given to TgzReader::open can be cloned, and that the original and clone will each have a position that can read and seek independently of the other. This is not the case, for example, for File. While File::try_clone exists, it is documented that “Reads, writes, and seeks will affect both File instances simultaneously.”

Panics

Requires i < self.chapters().

Example

This example demonstrates using independent_read_chapter to decompress all chapters in parallel.

A conveniently runnable form of this example is provided in this crate’s examples/ directory, also with a progress bar that shows overall progress.

use chapter_tgz::TgzReader;
use std::fs;
use std::io::{self, Cursor};
use std::thread;

fn main() -> io::Result<()> {
    let data = fs::read("example.tar.gz")?;
    let tgz = TgzReader::open(Cursor::new(data))?;

    let n = tgz.chapters();
    let mut chapters = Vec::with_capacity(n as usize);
    for i in 0..n {
        let mut chapter = tgz.independent_read_chapter(i)?;
        if i % 2 == 0 {
            // Option 1: We can directly enqueue chapters into the thread pool.
            chapters.push(chapter);
        } else if let Some(first_entry) = chapter.entries()?.next()
            && let Some(file_name) = first_entry?.path()?.file_name()
            && let Some(file_name_str) = file_name.to_str()
            && !file_name_str.starts_with("__")
        {
            // Option 2: We can examine chapter entries to decide whether to
            // process or skip a chapter. Reading the first entry's tar header
            // (path, size, PAX extensions) from a chapter is fast.
            chapters.push(tgz.independent_read_chapter(i)?);
        } else {
            // Option 3: Also fine to pass i and call independent_read_chapter
            // on the other thread.
        }
    }

    thread::scope(|scope| {
        for mut chapter in chapters {
            scope.spawn(move || {
                if let Err(err) = (|| -> io::Result<()> {
                    for _entry in chapter.entries()? {}
                    Ok(())
                })() {
                    eprintln!("Error: {err}");
                }
            });
        }
    });

    Ok(())
}

pub fn compressed_size_of_chapter(&self, i: u32) -> u64

Approximate compressed size of a chapter: the distance in bytes between the start and end of the chapter.

Time complexity: O(1)

This is primarily intended to enable progress reporting through the use of a Read implementation that monitors reads performed against the underlying compressed archive. Outside of this use case, the returned quantity is not guaranteed to be meaningful.

Example

This example demonstrates using compressed_size_of_chapter to size an Indicatif progress bar that increments as bytes are read.

The ProgressRead type is the one shown in the documentation of IndependentRead.

A conveniently runnable multithreaded form of this example is provided in this crate’s examples/ directory.

use chapter_tgz::TgzReader;
use indicatif::ProgressBar;
use std::fs::File;
use std::io::{self, Cursor};

fn main() -> io::Result<()> {
    let file = File::open("example.tar.gz")?;
    let pb = ProgressBar::no_length();
    let mut tgz = TgzReader::open(ProgressRead {
        inner: file,
        progress: &pb,
    })?;

    let n = tgz.chapters();
    let mut total_compressed_size = 0;
    for i in 0..n {
        total_compressed_size += tgz.compressed_size_of_chapter(i);
    }

    pb.reset();
    pb.set_length(total_compressed_size);

    for i in 0..n {
        for _entry in tgz.jump_to_chapter(i).entries()? {
            // ...
        }
    }
    Ok(())
}