Skip to main content

Pack

Struct Pack 

Source
pub struct Pack {
    pub number: usize,
    pub signature: ObjectHash,
    pub objects: Vec<Box<dyn ObjectTrait>>,
    pub pool: Arc<ThreadPool>,
    pub waitlist: Arc<Waitlist>,
    pub caches: Arc<Caches>,
    pub mem_limit: Option<usize>,
    pub cache_objs_mem: Arc<AtomicUsize>,
    pub clean_tmp: bool,
}
Expand description

Representation of a Git pack file in memory.

Fields§

§number: usize§signature: ObjectHash§objects: Vec<Box<dyn ObjectTrait>>§pool: Arc<ThreadPool>§waitlist: Arc<Waitlist>§caches: Arc<Caches>§mem_limit: Option<usize>§cache_objs_mem: Arc<AtomicUsize>§clean_tmp: bool

Implementations§

Source§

impl Pack

Source

pub fn new( thread_num: Option<usize>, mem_limit: Option<usize>, temp_path: Option<PathBuf>, clean_tmp: bool, ) -> Self

§Parameters
  • thread_num: The number of threads to use for decoding and cache, None means use the number of logical CPUs. The requested value is capped by available parallelism to avoid over-threading in constrained containers. It can’t be zero, or panic
  • mem_limit: The maximum size of the memory cache in bytes, or None for unlimited. The 80% of it will be used for Caches. Very large limits use the direct in-memory cache path and skip per-object memory backpressure to avoid hot-loop cache accounting.
    Not very accurate, because of memory alignment and other reasons, overuse about 15%
  • temp_path: The path to a directory for temporary files, default is “./.cache_temp”
    For example, thread_num = 4 will use up to 8 threads (4 for decoding and 4 for cache)
  • clean_tmp: whether to remove temp directory when Pack is dropped
Source

pub fn check_header(pack: &mut impl BufRead) -> Result<(u32, Vec<u8>), GitError>

Checks and reads the header of a Git pack file.

This function reads the first 12 bytes of a pack file, which include the b“PACK“ magic identifier, the version number, and the number of objects in the pack. It verifies that the magic identifier is correct and that the version number is 2 (which is the version currently supported by Git). It also collects these header bytes for later use, such as for hashing the entire pack file.

§Parameters
  • pack - A mutable reference to an object implementing the Read trait, representing the source of the pack file data (e.g., file, memory stream).
§Returns

A Result which is:

  • Ok((u32, Vec<u8>)): On successful reading and validation of the header, returns a tuple where:
    • The first element is the number of objects in the pack file (u32).
    • The second element is a vector containing the bytes of the pack file header (Vec<u8>).
  • Err(GitError): On failure, returns a GitError with a description of the issue.
§Errors

This function can return an error in the following situations:

  • If the pack file does not start with the “PACK” magic identifier.
  • If the pack file’s version number is not 2.
  • If there are any issues reading from the provided pack source.
Source

pub fn decompress_data( pack: &mut (impl BufRead + Send), expected_size: usize, ) -> Result<(Vec<u8>, usize), GitError>

Decompresses data from a given Read and BufRead source using Zlib decompression.

§Parameters
  • pack: A source that implements both Read and BufRead traits (e.g., file, network stream).
  • expected_size: The expected decompressed size of the data.
§Returns

Returns a Result containing either:

  • A tuple with a Vec<u8> of the decompressed data and the total number of input bytes processed,
  • Or a GitError in case of a mismatch in expected size or any other reading error.
Source

pub fn decode_pack_object( pack: &mut (impl BufRead + Send), offset: &mut usize, ) -> Result<Option<CacheObject>, GitError>

Decodes a pack object from a given Read and BufRead source and returns the object as a CacheObject.

§Parameters
  • pack: A source that implements both Read and BufRead traits.
  • offset: A mutable reference to the current offset within the pack.
§Returns

Returns a Result containing either:

  • A tuple of the next offset in the pack and the original compressed data as Vec<u8>,
  • Or a GitError in case of any reading or decompression error.
Source

pub fn decode<F, C>( &mut self, pack: &mut (impl BufRead + Send), callback: F, pack_id_callback: Option<C>, ) -> Result<(), GitError>
where F: Fn(MetaAttached<Entry, EntryMeta>) + Sync + Send + 'static, C: FnOnce(ObjectHash) + Send + 'static,

Decodes a pack file from a given Read and BufRead source, for each object in the pack, it decodes the object and processes it using the provided callback function.

§Parameters
  • pack_id_callback: A callback that seed pack_file sha1 for updating database
Source

pub fn decode_without_callback<C>( &mut self, pack: &mut (impl BufRead + Send), pack_id_callback: Option<C>, ) -> Result<(), GitError>
where C: FnOnce(ObjectHash) + Send + 'static,

Decodes a pack file without materializing callback entries.

Use this when the caller only needs validation, hashing, and cache/delta reconstruction side effects. This preserves decode for callers that consume each decoded object, while avoiding one full object-data clone per completed object on the no-callback path.

Source

pub fn decode_file<F, C>( &mut self, pack_path: impl AsRef<Path>, callback: F, pack_id_callback: Option<C>, ) -> Result<(), GitError>
where F: Fn(MetaAttached<Entry, EntryMeta>) + Sync + Send + 'static, C: FnOnce(ObjectHash) + Send + 'static,

Decodes a pack file from disk and invokes a callback for each decoded object.

File-backed callers can use this path to reuse the same index-guided retention scan as decode_file_without_callback, while preserving the object metadata and CRC32 values needed by callback consumers such as index generation.

Source

pub fn decode_file_full_without_callback<C>( &mut self, pack_path: impl AsRef<Path>, pack_id_callback: Option<C>, ) -> Result<(), GitError>
where C: FnOnce(ObjectHash) + Send + 'static,

Decodes a pack file from disk without constructing callback entries, while still restoring every object in the pack.

Compared with decode_file, this avoids callback metadata and object-data clones. Compared with decode_file_without_callback, it does not skip leaf objects, so benchmark callers can measure a complete decode without producing per-object output.

Source

pub fn decode_file_without_callback<C>( &mut self, pack_path: impl AsRef<Path>, pack_id_callback: Option<C>, ) -> Result<(), GitError>
where C: FnOnce(ObjectHash) + Send + 'static,

Decodes a pack file from disk without retaining objects after their final delta user.

This is a no-callback fast path for validation/benchmark callers. It performs a light dependency scan first, then releases cache entries as soon as all later delta objects that need them have acquired an Arc to the base.

Source

pub fn decode_async( self, pack: impl BufRead + Send + 'static, sender: UnboundedSender<Entry>, ) -> JoinHandle<Pack>

Decode a Pack in a new thread and send the CacheObjects while decoding.
Attention: It will consume the pack and return in a JoinHandle.

Source

pub async fn decode_stream( self, stream: impl Stream<Item = Result<Bytes, Error>> + Unpin + Send + 'static, sender: UnboundedSender<MetaAttached<Entry, EntryMeta>>, pack_hash_send: Option<UnboundedSender<ObjectHash>>, ) -> Self

Decodes a Pack from a Stream of Bytes, and sends the Entry while decoding.

Source

pub fn rebuild_delta( delta_obj: CacheObject, base_obj: Arc<CacheObject>, ) -> CacheObject

Reconstruct the Delta Object based on the “base object” and return the new object.

Source

pub fn rebuild_zstdelta( delta_obj: CacheObject, base_obj: Arc<CacheObject>, ) -> CacheObject

Source§

impl Pack

Source

pub fn stats_pack(path: PathBuf) -> Result<PackStats, GitError>

Scans a pack file and returns statistics about the object types it contains.

This is a lightweight read-only utility that parses the pack header and every object header without fully reconstructing delta chains. It therefore runs much faster than a full Pack::decode call for large packs.

§Parameters
  • path - Path to the .pack file on disk.
§Returns
  • Ok(PackStats) – breakdown of object counts by type.
  • Err(GitError) – if the file cannot be opened or the pack header is invalid.
§Example
use std::path::PathBuf;
use git_internal::internal::pack::{Pack, decode::PackStats};

let stats = Pack::stats_pack(PathBuf::from("repo.pack")).unwrap();
println!("total={}, commits={}, blobs={}", stats.total, stats.commits, stats.blobs);

Trait Implementations§

Source§

impl Drop for Pack

Source§

fn drop(&mut self)

Executes the destructor for this type. Read more
Source§

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)
Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned. Read more

Auto Trait Implementations§

§

impl !RefUnwindSafe for Pack

§

impl !UnwindSafe for Pack

§

impl Freeze for Pack

§

impl Send for Pack

§

impl Sync for Pack

§

impl Unpin for Pack

§

impl UnsafeUnpin for Pack

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> ArchivePointee for T

Source§

type ArchivedMetadata = ()

The archived version of the pointer metadata for this type.
Source§

fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata

Converts some archived metadata to the pointer metadata for itself.
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<A, B, T> HttpServerConnExec<A, B> for T
where B: Body,

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> LayoutRaw for T

Source§

fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>

Returns the layout of the type.
Source§

impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
where T: SharedNiching<N1, N2>, N1: Niching<T>, N2: Niching<T>,

Source§

unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool

Returns whether the given value has been niched. Read more
Source§

fn resolve_niched(out: Place<NichedOption<T, N1>>)

Writes data to out indicating that a T is niched.
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Pointee for T

Source§

type Metadata = ()

The metadata type for pointers and references to this type.
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> ValueSize for T

Source§

fn value_size(&self) -> usize

The size of this value in bytes, excluding allocated data. Read more
Source§

fn value_size_sum_iter<'item>(iterator: impl Iterator<Item = &'item T>) -> usize
where T: 'item,

The total sum of the sizes of all values in the given iterator, in bytes. This is default-implemented by computing ValueSize::value_size on every element and summing them. For Sized types, a more potentially efficient implementation using Iterator::count is provided. If you are implementing this trait manually, it is unlikely to be more efficient to provide a manual implementation here. Read more
Source§

fn value_size_sum_exact_size_iter<'item>( iterator: impl ExactSizeIterator<Item = &'item T>, ) -> usize
where T: 'item,

The total sum of the sizes of all values in the given exact-size-iterator, in bytes. This is default-implemented by using ValueSize::value_size_sum_iter. For Sized types, a usually more efficient implementation using ExactSizeIterator::len is provided. If you are implementing this trait manually, it is unlikely to be more efficient to provide a manual implementation here. Read more
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more