Struct git_pack::index::File [−][src]
pub struct File { /* fields omitted */ }Expand description
A representation of a pack index file
Implementations
Instantiation
Iteration and access
Returns 20 bytes sha1 at the given index in our list of (sorted) sha1 hashes. The index ranges from 0 to self.num_objects()
Panics
If index is out of bounds.
Returns the offset into our pack data file at which to start reading the object at index.
Panics
If index is out of bounds.
Returns the CRC32 of the object at the given index.
Note: These are always present for index version 2 or higher.
Panics
If index is out of bounds.
Returns the index of the given SHA1 for use with the oid_at_index(),
pack_offset_at_index() or crc32_at_index().
An iterator over all Entries of this index file.
Return a vector of ascending offsets into our respective pack data file.
Useful to control an iteration over all pack entries in a cache-friendly way.
Traversal with index
pub fn traverse_with_index<P, Processor, E>(
&self,
check: SafetyCheck,
thread_limit: Option<usize>,
new_processor: impl Fn() -> Processor + Send + Sync,
progress: P,
pack: &File,
should_interrupt: Arc<AtomicBool>
) -> Result<(ObjectId, Outcome, P), Error<E>> where
P: Progress,
Processor: FnMut(Kind, &[u8], &Entry, &mut <<P as Progress>::SubProgress as Progress>::SubProgress) -> Result<(), E>,
E: Error + Send + Sync + 'static,
pub fn traverse_with_index<P, Processor, E>(
&self,
check: SafetyCheck,
thread_limit: Option<usize>,
new_processor: impl Fn() -> Processor + Send + Sync,
progress: P,
pack: &File,
should_interrupt: Arc<AtomicBool>
) -> Result<(ObjectId, Outcome, P), Error<E>> where
P: Progress,
Processor: FnMut(Kind, &[u8], &Entry, &mut <<P as Progress>::SubProgress as Progress>::SubProgress) -> Result<(), E>,
E: Error + Send + Sync + 'static,
Iterate through all decoded objects in the given pack and handle them with a Processor, using an index to reduce waste
at the cost of memory.
For more details, see the documentation on the traverse() method.
Verify and validate the content of the index file
pub fn traverse_with_lookup<P, C, Processor, E>(
&self,
new_processor: impl Fn() -> Processor + Send + Sync,
new_cache: impl Fn() -> C + Send + Sync,
progress: P,
pack: &File,
_: Options
) -> Result<(ObjectId, Outcome, P), Error<E>> where
P: Progress,
C: DecodeEntry,
E: Error + Send + Sync + 'static,
Processor: FnMut(Kind, &[u8], &Entry, &mut <<P as Progress>::SubProgress as Progress>::SubProgress) -> Result<(), E>,
pub fn traverse_with_lookup<P, C, Processor, E>(
&self,
new_processor: impl Fn() -> Processor + Send + Sync,
new_cache: impl Fn() -> C + Send + Sync,
progress: P,
pack: &File,
_: Options
) -> Result<(ObjectId, Outcome, P), Error<E>> where
P: Progress,
C: DecodeEntry,
E: Error + Send + Sync + 'static,
Processor: FnMut(Kind, &[u8], &Entry, &mut <<P as Progress>::SubProgress as Progress>::SubProgress) -> Result<(), E>,
Iterate through all decoded objects in the given pack and handle them with a Processor using a cache to reduce the amount of
waste while decoding objects.
For more details, see the documentation on the traverse() method.
Traversal of pack data files using an index file
pub fn traverse<P, C, Processor, E>(
&self,
pack: &File,
progress: Option<P>,
new_processor: impl Fn() -> Processor + Send + Sync,
new_cache: impl Fn() -> C + Send + Sync,
_: Options
) -> Result<(ObjectId, Outcome, Option<P>), Error<E>> where
P: Progress,
C: DecodeEntry,
E: Error + Send + Sync + 'static,
Processor: FnMut(Kind, &[u8], &Entry, &mut DoOrDiscard<<<P as Progress>::SubProgress as Progress>::SubProgress>) -> Result<(), E>,
pub fn traverse<P, C, Processor, E>(
&self,
pack: &File,
progress: Option<P>,
new_processor: impl Fn() -> Processor + Send + Sync,
new_cache: impl Fn() -> C + Send + Sync,
_: Options
) -> Result<(ObjectId, Outcome, Option<P>), Error<E>> where
P: Progress,
C: DecodeEntry,
E: Error + Send + Sync + 'static,
Processor: FnMut(Kind, &[u8], &Entry, &mut DoOrDiscard<<<P as Progress>::SubProgress as Progress>::SubProgress>) -> Result<(), E>,
Iterate through all decoded objects in the given pack and handle them with a Processor.
The return value is (pack-checksum, Outcome, progress), thus the pack traversal will always verify
the whole packs checksum to assure it was correct. In case of bit-rod, the operation will abort early without
verifying all objects using the interrupt mechanism mechanism.
Algorithms
Using the Options::algorithm field one can chose between two algorithms providing different tradeoffs. Both invoke
new_processor() to create functions receiving decoded objects, their object kind, index entry and a progress instance to provide
progress information.
Algorithm::DeltaTreeLookupbuilds an index to avoid any unnecessary computation while resolving objects, avoiding the need for a cache entirely, renderingnew_cache()unused. One could also calltraverse_with_index()directly.Algorithm::Lookupuses a cache created bynew_cache()to avoid having to re-compute all bases of a delta-chain while decoding objects. One could also calltraverse_with_lookup()directly.
Use thread_limit to further control parallelism and check to define how much the passed
objects shall be verified beforehand.
Verify and validate the content of the index file
Returns the trailing hash stored at the end of this index file.
It’s a hash over all bytes of the index.
Returns the hash of the pack data file that this index file corresponds to.
It should crate::data::File::checksum() of the corresponding pack data file.
pub fn verify_checksum(
&self,
progress: impl Progress,
should_interrupt: &AtomicBool
) -> Result<ObjectId, Error>
pub fn verify_checksum(
&self,
progress: impl Progress,
should_interrupt: &AtomicBool
) -> Result<ObjectId, Error>
Validate that our index_checksum() matches the actual contents
of this index file, and return it if it does.
The most thorough validation of integrity of both index file and the corresponding pack data file, if provided. Returns the checksum of the index file, the traversal outcome and the given progress if the integrity check is successful.
If pack is provided, it is expected (and validated to be) the pack belonging to this index.
It will be used to validate internal integrity of the pack before checking each objects integrity
is indeed as advertised via its SHA1 as stored in this index, as well as the CRC32 hash.
The last member of the Option is a function returning an implementation of crate::cache::DecodeEntry to be used if
the index::traverse::Algorithm is Lookup.
To set this to None, use None::<(_, _, _, fn() -> crate::cache::Never)>.
The thread_limit optionally specifies the amount of threads to be used for the pack traversal.
make_cache is only used in case a pack is specified, use existing implementations in the crate::cache module.
Tradeoffs
The given progress is inevitably consumed if there is an error, which is a tradeoff chosen to easily allow using ? in the
error case.
Various ways of writing an index file from pack entries
pub fn write_data_iter_to_stream<F, F2>(
kind: Version,
make_resolver: F,
entries: impl Iterator<Item = Result<Entry, Error>>,
thread_limit: Option<usize>,
root_progress: impl Progress,
out: impl Write,
should_interrupt: &AtomicBool
) -> Result<Outcome, Error> where
F: FnOnce() -> Result<F2>,
F2: for<'r> Fn(EntryRange, &'r mut Vec<u8>) -> Option<()> + Send + Sync,
pub fn write_data_iter_to_stream<F, F2>(
kind: Version,
make_resolver: F,
entries: impl Iterator<Item = Result<Entry, Error>>,
thread_limit: Option<usize>,
root_progress: impl Progress,
out: impl Write,
should_interrupt: &AtomicBool
) -> Result<Outcome, Error> where
F: FnOnce() -> Result<F2>,
F2: for<'r> Fn(EntryRange, &'r mut Vec<u8>) -> Option<()> + Send + Sync,
Write information about entries as obtained from a pack data file into a pack index file via the out stream.
The resolver produced by make_resolver must resolve pack entries from the same pack data file that produced the
entries iterator.
kind is the version of pack index to produce, use crate::index::Version::default() if in doubt.
tread_limit is used for a parallel tree traversal for obtaining object hashes with optimal performance.
root_progress is the top-level progress to stay informed about the progress of this potentially long-running
computation.
Remarks
- neither in-pack nor out-of-pack Ref Deltas are supported here, these must have been resolved beforehand.
make_resolver()will only be called after the iterator stopped returning elements and produces a function that provides all bytes belonging to a pack entry writing them to the given mutable outputVec. It should returnNoneif the entry cannot be resolved from the pack that produced theentriesiterator, causing the write operation to fail.
Basic file information
Trait Implementations
Auto Trait Implementations
impl RefUnwindSafe for File
impl UnwindSafe for File
Blanket Implementations
Mutably borrows from an owned value. Read more
fn fmt_binary(self) -> FmtBinary<Self> where
Self: Binary,
fn fmt_binary(self) -> FmtBinary<Self> where
Self: Binary,
Causes self to use its Binary implementation when Debug-formatted.
fn fmt_display(self) -> FmtDisplay<Self> where
Self: Display,
fn fmt_display(self) -> FmtDisplay<Self> where
Self: Display,
Causes self to use its Display implementation when
Debug-formatted. Read more
fn fmt_lower_exp(self) -> FmtLowerExp<Self> where
Self: LowerExp,
fn fmt_lower_exp(self) -> FmtLowerExp<Self> where
Self: LowerExp,
Causes self to use its LowerExp implementation when
Debug-formatted. Read more
fn fmt_lower_hex(self) -> FmtLowerHex<Self> where
Self: LowerHex,
fn fmt_lower_hex(self) -> FmtLowerHex<Self> where
Self: LowerHex,
Causes self to use its LowerHex implementation when
Debug-formatted. Read more
Causes self to use its Octal implementation when Debug-formatted.
fn fmt_pointer(self) -> FmtPointer<Self> where
Self: Pointer,
fn fmt_pointer(self) -> FmtPointer<Self> where
Self: Pointer,
Causes self to use its Pointer implementation when
Debug-formatted. Read more
fn fmt_upper_exp(self) -> FmtUpperExp<Self> where
Self: UpperExp,
fn fmt_upper_exp(self) -> FmtUpperExp<Self> where
Self: UpperExp,
Causes self to use its UpperExp implementation when
Debug-formatted. Read more
fn fmt_upper_hex(self) -> FmtUpperHex<Self> where
Self: UpperHex,
fn fmt_upper_hex(self) -> FmtUpperHex<Self> where
Self: UpperHex,
Causes self to use its UpperHex implementation when
Debug-formatted. Read more
impl<T> Pipe for T where
T: ?Sized,
impl<T> Pipe for T where
T: ?Sized,
Pipes by value. This is generally the method you want to use. Read more
Borrows self and passes that borrow into the pipe function. Read more
fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R where
R: 'a,
fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R where
R: 'a,
Mutably borrows self and passes that borrow into the pipe function. Read more
fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R where
Self: Borrow<B>,
R: 'a,
B: 'a + ?Sized,
fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R where
Self: Borrow<B>,
R: 'a,
B: 'a + ?Sized,
Borrows self, then passes self.borrow() into the pipe function. Read more
fn pipe_borrow_mut<'a, B, R>(
&'a mut self,
func: impl FnOnce(&'a mut B) -> R
) -> R where
Self: BorrowMut<B>,
R: 'a,
B: 'a + ?Sized,
fn pipe_borrow_mut<'a, B, R>(
&'a mut self,
func: impl FnOnce(&'a mut B) -> R
) -> R where
Self: BorrowMut<B>,
R: 'a,
B: 'a + ?Sized,
Mutably borrows self, then passes self.borrow_mut() into the pipe
function. Read more
fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R where
Self: AsRef<U>,
R: 'a,
U: 'a + ?Sized,
fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R where
Self: AsRef<U>,
R: 'a,
U: 'a + ?Sized,
Borrows self, then passes self.as_ref() into the pipe function.
fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R where
Self: AsMut<U>,
R: 'a,
U: 'a + ?Sized,
fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R where
Self: AsMut<U>,
R: 'a,
U: 'a + ?Sized,
Mutably borrows self, then passes self.as_mut() into the pipe
function. Read more
fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: Deref<Target = T>,
R: 'a,
T: 'a + ?Sized,
fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: Deref<Target = T>,
R: 'a,
T: 'a + ?Sized,
Borrows self, then passes self.deref() into the pipe function.
fn pipe_as_ref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: AsRef<T>,
R: 'a,
T: 'a,
fn pipe_as_ref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: AsRef<T>,
R: 'a,
T: 'a,
Pipes a trait borrow into a function that cannot normally be called in suffix position. Read more
fn pipe_as_mut<'a, T, R>(&'a mut self, func: impl FnOnce(&'a mut T) -> R) -> R where
Self: AsMut<T>,
R: 'a,
T: 'a,
fn pipe_as_mut<'a, T, R>(&'a mut self, func: impl FnOnce(&'a mut T) -> R) -> R where
Self: AsMut<T>,
R: 'a,
T: 'a,
Pipes a trait mutable borrow into a function that cannot normally be called in suffix position. Read more
fn pipe_borrow<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: Borrow<T>,
R: 'a,
T: 'a,
fn pipe_borrow<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R where
Self: Borrow<T>,
R: 'a,
T: 'a,
Pipes a trait borrow into a function that cannot normally be called in suffix position. Read more
fn pipe_borrow_mut<'a, T, R>(
&'a mut self,
func: impl FnOnce(&'a mut T) -> R
) -> R where
Self: BorrowMut<T>,
R: 'a,
T: 'a,
fn pipe_borrow_mut<'a, T, R>(
&'a mut self,
func: impl FnOnce(&'a mut T) -> R
) -> R where
Self: BorrowMut<T>,
R: 'a,
T: 'a,
Pipes a trait mutable borrow into a function that cannot normally be called in suffix position. Read more
fn pipe_deref<'a, R>(&'a self, func: impl FnOnce(&'a Self::Target) -> R) -> R where
Self: Deref,
R: 'a,
fn pipe_deref<'a, R>(&'a self, func: impl FnOnce(&'a Self::Target) -> R) -> R where
Self: Deref,
R: 'a,
Pipes a dereference into a function that cannot normally be called in suffix position. Read more
fn pipe_deref_mut<'a, R>(
&'a mut self,
func: impl FnOnce(&'a mut Self::Target) -> R
) -> R where
Self: DerefMut,
R: 'a,
fn pipe_deref_mut<'a, R>(
&'a mut self,
func: impl FnOnce(&'a mut Self::Target) -> R
) -> R where
Self: DerefMut,
R: 'a,
Pipes a mutable dereference into a function that cannot normally be called in suffix position. Read more
Pipes a reference into a function that cannot ordinarily be called in suffix position. Read more
fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self where
Self: Borrow<B>,
B: ?Sized,
fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self where
Self: Borrow<B>,
B: ?Sized,
Immutable access to the Borrow<B> of a value. Read more
fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self where
Self: BorrowMut<B>,
B: ?Sized,
fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self where
Self: BorrowMut<B>,
B: ?Sized,
Mutable access to the BorrowMut<B> of a value. Read more
Immutable access to the AsRef<R> view of a value. Read more
fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self where
Self: AsMut<R>,
R: ?Sized,
fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self where
Self: AsMut<R>,
R: ?Sized,
Mutable access to the AsMut<R> view of a value. Read more
Immutable access to the Deref::Target of a value. Read more
Mutable access to the Deref::Target of a value. Read more
Calls .tap() only in debug builds, and is erased in release builds.
fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
Calls .tap_mut() only in debug builds, and is erased in release
builds. Read more
fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self where
Self: Borrow<B>,
B: ?Sized,
fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self where
Self: Borrow<B>,
B: ?Sized,
Calls .tap_borrow() only in debug builds, and is erased in release
builds. Read more
fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self where
Self: BorrowMut<B>,
B: ?Sized,
fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self where
Self: BorrowMut<B>,
B: ?Sized,
Calls .tap_borrow_mut() only in debug builds, and is erased in release
builds. Read more
fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self where
Self: AsRef<R>,
R: ?Sized,
fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self where
Self: AsRef<R>,
R: ?Sized,
Calls .tap_ref() only in debug builds, and is erased in release
builds. Read more
fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self where
Self: AsMut<R>,
R: ?Sized,
fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self where
Self: AsMut<R>,
R: ?Sized,
Calls .tap_ref_mut() only in debug builds, and is erased in release
builds. Read more
Provides immutable access for inspection. Read more
Calls tap in debug builds, and does nothing in release builds.
Provides mutable access for modification. Read more
fn tap_mut_dbg<F, R>(self, func: F) -> Self where
F: FnOnce(&mut Self) -> R,
fn tap_mut_dbg<F, R>(self, func: F) -> Self where
F: FnOnce(&mut Self) -> R,
Calls tap_mut in debug builds, and does nothing in release builds.
impl<T, U> TapAsRef<U> for T where
U: ?Sized,
impl<T, U> TapAsRef<U> for T where
U: ?Sized,
Provides immutable access to the reference for inspection.
fn tap_ref_dbg<F, R>(self, func: F) -> Self where
Self: AsRef<T>,
F: FnOnce(&T) -> R,
fn tap_ref_dbg<F, R>(self, func: F) -> Self where
Self: AsRef<T>,
F: FnOnce(&T) -> R,
Calls tap_ref in debug builds, and does nothing in release builds.
fn tap_ref_mut<F, R>(self, func: F) -> Self where
Self: AsMut<T>,
F: FnOnce(&mut T) -> R,
fn tap_ref_mut<F, R>(self, func: F) -> Self where
Self: AsMut<T>,
F: FnOnce(&mut T) -> R,
Provides mutable access to the reference for modification.
fn tap_ref_mut_dbg<F, R>(self, func: F) -> Self where
Self: AsMut<T>,
F: FnOnce(&mut T) -> R,
fn tap_ref_mut_dbg<F, R>(self, func: F) -> Self where
Self: AsMut<T>,
F: FnOnce(&mut T) -> R,
Calls tap_ref_mut in debug builds, and does nothing in release builds.
impl<T, U> TapBorrow<U> for T where
U: ?Sized,
impl<T, U> TapBorrow<U> for T where
U: ?Sized,
fn tap_borrow<F, R>(self, func: F) -> Self where
Self: Borrow<T>,
F: FnOnce(&T) -> R,
fn tap_borrow<F, R>(self, func: F) -> Self where
Self: Borrow<T>,
F: FnOnce(&T) -> R,
Provides immutable access to the borrow for inspection. Read more
fn tap_borrow_dbg<F, R>(self, func: F) -> Self where
Self: Borrow<T>,
F: FnOnce(&T) -> R,
fn tap_borrow_dbg<F, R>(self, func: F) -> Self where
Self: Borrow<T>,
F: FnOnce(&T) -> R,
Calls tap_borrow in debug builds, and does nothing in release builds.
fn tap_borrow_mut<F, R>(self, func: F) -> Self where
Self: BorrowMut<T>,
F: FnOnce(&mut T) -> R,
fn tap_borrow_mut<F, R>(self, func: F) -> Self where
Self: BorrowMut<T>,
F: FnOnce(&mut T) -> R,
Provides mutable access to the borrow for modification.
fn tap_borrow_mut_dbg<F, R>(self, func: F) -> Self where
Self: BorrowMut<T>,
F: FnOnce(&mut T) -> R,
fn tap_borrow_mut_dbg<F, R>(self, func: F) -> Self where
Self: BorrowMut<T>,
F: FnOnce(&mut T) -> R,
Calls tap_borrow_mut in debug builds, and does nothing in release
builds. Read more
Immutably dereferences self for inspection.
fn tap_deref_dbg<F, R>(self, func: F) -> Self where
Self: Deref,
F: FnOnce(&Self::Target) -> R,
fn tap_deref_dbg<F, R>(self, func: F) -> Self where
Self: Deref,
F: FnOnce(&Self::Target) -> R,
Calls tap_deref in debug builds, and does nothing in release builds.
fn tap_deref_mut<F, R>(self, func: F) -> Self where
Self: DerefMut,
F: FnOnce(&mut Self::Target) -> R,
fn tap_deref_mut<F, R>(self, func: F) -> Self where
Self: DerefMut,
F: FnOnce(&mut Self::Target) -> R,
Mutably dereferences self for modification.
fn tap_deref_mut_dbg<F, R>(self, func: F) -> Self where
Self: DerefMut,
F: FnOnce(&mut Self::Target) -> R,
fn tap_deref_mut_dbg<F, R>(self, func: F) -> Self where
Self: DerefMut,
F: FnOnce(&mut Self::Target) -> R,
Calls tap_deref_mut in debug builds, and does nothing in release
builds. Read more