Skip to main content

Crate znippy_common

Crate znippy_common 

Source

Re-exports§

pub use meta::BlobMeta;
pub use meta::ChunkMeta;
pub use meta::FileMeta;
pub use meta_sink_append::compact_archive;
pub use meta_sink_append::read_delta_map;
pub use meta_sink_append::supersede_as_delta;
pub use meta_sink_append::CompactReport;
pub use meta_sink_append::SupersedeOutcome;
pub use meta_sink_append::MAX_GENERATION_CHAIN;
pub use archive::BaseResolve;
pub use archive::ChunkSplitter;
pub use archive::DeltaSplitter;
pub use archive::Entry;
pub use archive::ReconstructCtx;
pub use archive::RegionDeltaSplitter;
pub use archive::Splitter;
pub use archive::StoredUnit;
pub use archive::UnitPayload;
pub use archive::ZnippyArchive;
pub use archive::ZnippyReader;
pub use views::CondaPackage;
pub use views::CondaView;
pub use views::GemPackage;
pub use views::GemView;
pub use views::MavenPackage;
pub use views::MavenView;
pub use views::NpmPackage;
pub use views::NpmView;
pub use views::PythonKind;
pub use views::PythonPackage;
pub use views::PythonView;
pub use views::RustPackage;
pub use views::RustView;
pub use views::CONDA_PKG_TYPE;
pub use views::GEM_PKG_TYPE;
pub use views::MAVEN_PKG_TYPE;
pub use views::NPM_PKG_TYPE;
pub use views::PYTHON_PKG_TYPE;
pub use views::RUST_PKG_TYPE;
pub use decompress::decompress_archive;
pub use decompress::decompress_archive_filtered;
pub use decompress::get_file;
pub use meta_sink::ArchiveMetaSink;
pub use meta_sink::ArrowIpcSink;
pub use meta_sink::GroupKey;
pub use meta_sink::LookupView;
pub use meta_sink::MetaSinkFactory;
pub use meta_sink::ReservedPayload;
pub use meta_sink::ReservedSection;
pub use meta_sink::ReservedSectionBuilder;
pub use hot::HotAppendReport;
pub use hot::HotArchive;
pub use hot::HotSealReport;
pub use hot::seal_hot;
pub use meta_sink_append::append_files_with_policy;
pub use meta_sink_append::AppendReport;
pub use meta_sink_append::ArrowIpcSinkAppend;
pub use meta_sink_append::append_files;
pub use meta_sink_append::append_files_with_meta;
pub use meta_sink_append::base_batch_from_rows;
pub use meta_sink_append::create_archive;
pub use meta_sink_append::create_archive_to_vec;
pub use meta_sink_append::create_archive_with_meta;
pub use meta_index::ArchiveMeta;
pub use meta_index::MetaEntry;
pub use meta_index::MetaIndex;
pub use meta_index::MetaSearch;
pub use meta_index::MetaTable;
pub use meta_index::MetaValue;
pub use meta_index::read_archive_meta;
pub use precompressed::SkipPolicy;
pub use index::ArchiveReader;
pub use index::ArtifactMeta;
pub use index::ChunkLoc;
pub use index::IndexFilter;
pub use index::VerifyReport;
pub use index::ZNIPPY_INDEX_SCHEMA;
pub use index::MULTI_INDEX_MAGIC;
pub use index::ManifestEntry;
pub use index::IndexFooter;
pub use index::META_MODULE;
pub use index::build_arrow_metadata_for_config;
pub use index::build_metadata_batch;
pub use index::data_subindex_schema;
pub use index::extract_config_from_arrow_metadata;
pub use index::get_all_files_meta;
pub use index::get_files_meta_with_prefix;
pub use index::is_probably_compressed;
pub use index::is_reserved_module;
pub use index::list_archive_contents;
pub use index::locate_file;
pub use index::read_manifest_bytes;
pub use index::read_reserved_section_bytes;
pub use index::read_znippy_full_manifest;
pub use index::read_znippy_index;
pub use index::read_znippy_index_filtered;
pub use index::read_znippy_manifest;
pub use index::should_skip_compression;
pub use index::verify_archive_integrity;
pub use index::write_manifest_bytes;
pub use index::znippy_index_schema;
pub use index::SIGN_ARCHIVE_MODULE;
pub use index::SIGN_ARTIFACTS_MODULE;
pub use index::GUNNAR_GRAPH_MODULE;
pub use index::GUNNAR_OID_MODULE;
pub use index::GUNNAR_REACH_MODULE;
pub use index::GUNNAR_REFS_MODULE;
pub use index::GUNNAR_SECRETS_MODULE;
pub use index::LOOKUP_MODULE;
pub use index::RESERVED_MODULES;
pub use index::RESERVED_PKG_TYPE;
pub use index::TRIE_MODULE;
pub use index::lookup_schema;
pub use arrow;
pub use blake3;

Modules§

archive
ZnippyArchive — trait and implementation for reading znippy archives.
codec
Codec layer: OpenZL compression/decompression. OpenZL is znippy’s codec — it wraps zstd+lz4 with improved framing.
common_config
decompress
hostinfo
Host memory and CPU facts read straight from /proc and /sys/fs/cgroup. Replaces the sysinfo runtime dependency, which reached this crate’s whole consumer set — and, via its multithread feature, put rayon in it. Host memory and CPU facts, read straight from /proc and /sys/fs/cgroup.
hot
The DYNAMIC half of the archive: append at O(1), seal once. See hot. HotArchive — the dynamic half of a znippy archive: append blobs and index rows at O(1) per append, and pay the static seal exactly once.
index
meta
meta_index
Searchable archive metadata: the META_MODULE sub-index, its typed key/value model, and the query API — “which entries carry key X” answered from an index, at a cost set by the metadata, not by the payload. Searchable archive metadata — answer “which entries in this archive carry key X, and what are their values?” by reading an index, never by streaming the payload.
meta_sink
ArchiveMetaSink — abstraction over the archive’s metadata layer.
meta_sink_append
ArrowIpcSinkAppend — the append/resume-capable v2 clone of ArrowIpcSink.
plugin
Plugin trait for archive-type-aware metadata extraction.
plugins
Plugin source, split the same way the repo-root plugins/ directory is split: native/ for handlers compiled into the host binary, wasm/ for the host-side loader of wasm32-unknown-unknown modules. No handler source sits loose beside this file — znippy-common/tests/plugin_source_layout.rs asserts that, and fails naming the stray file.
precompressed
Whether a payload is already compressed, and so should be stored raw. Resolution order: an explicit caller hint, then the extension table, then a magic-byte probe over the head of the real bytes. Deciding whether a payload is already compressed, and so should be stored raw instead of handed to the codec.
slotpool
Magazine — shared large-slot buffer pool for the no-barrier slice pipeline.
views
Typed specialized package views — the READ side of the plugin contract.

Structs§

CompressionReport

Enums§

ContentHint
The probe itself, re-exported so znippy_common::precompressed::* keeps naming exactly what it always named. These are ldeflate’s items, not copies of them — breaking one there breaks every znippy caller, which is the point of the move. What a caller already knows about its content, if anything.

Constants§

SNIFF_PREFIX_LEN
The probe itself, re-exported so znippy_common::precompressed::* keeps naming exactly what it always named. These are ldeflate’s items, not copies of them — breaking one there breaks every znippy caller, which is the point of the move. Bytes of prefix the probe reads. Sized by WebP, whose form type sits at offset 8..12 — the longest anchored pattern here.

Functions§

is_zlib_stream
The probe itself, re-exported so znippy_common::precompressed::* keeps naming exactly what it always named. These are ldeflate’s items, not copies of them — breaking one there breaks every znippy caller, which is the point of the move. Does this buffer begin with a raw zlib stream (RFC 1950)?
looks_compressed
The probe itself, re-exported so znippy_common::precompressed::* keeps naming exactly what it always named. These are ldeflate’s items, not copies of them — breaking one there breaks every znippy caller, which is the point of the move. Does this buffer begin with data that is already compressed?