pub struct ArchiveWriter { /* private fields */ }Expand description
Writer for creating an STT archive.
Two modes:
create— eager: each tile is compressed and written as it arrives, so peak memory is bounded (good for streaming huge builds).create_optimized— buffered: tiles are held untilfinalize, where a shared zstd dictionary is trained, byte-identical blobs are deduplicated, and blobs are written in Hilbert order. Smaller archives at the cost of holding payloads in memory.
Implementations§
Source§impl ArchiveWriter
impl ArchiveWriter
Sourcepub fn create<P: AsRef<Path>>(path: P, compression: Compression) -> Result<Self>
pub fn create<P: AsRef<Path>>(path: P, compression: Compression) -> Result<Self>
Create an archive in eager (streaming) mode. Tile blobs use compression.
Sourcepub fn create_optimized<P: AsRef<Path>>(path: P) -> Result<Self>
pub fn create_optimized<P: AsRef<Path>>(path: P) -> Result<Self>
Create an archive in buffered “optimized” mode: a shared zstd dictionary
is trained over the tile corpus, byte-identical blobs are deduplicated,
and blobs are laid down in 2D-spatial-Hilbert order for range-read
locality. Requires zstd. See create_optimized_with_ordering to pick a
different (e.g. space-time) blob order.
Sourcepub fn create_optimized_with_ordering<P: AsRef<Path>>(
path: P,
ordering: BlobOrdering,
) -> Result<Self>
pub fn create_optimized_with_ordering<P: AsRef<Path>>( path: P, ordering: BlobOrdering, ) -> Result<Self>
Buffered “optimized” mode with an explicit blob byte-ordering. The
directory index stays (zoom, hilbert, time_start) regardless; only the
physical blob layout changes, so readers are unaffected. The default
(BlobOrdering::Hilbert3) minimises range requests across the widest
range of datasets (see examples/simulate_layout.rs).
Sourcepub fn create_reordered<P: AsRef<Path>>(
path: P,
ordering: BlobOrdering,
) -> Result<Self>
pub fn create_reordered<P: AsRef<Path>>( path: P, ordering: BlobOrdering, ) -> Result<Self>
Buffered mode that reorders blobs but ships NO shared dictionary, so
the result stays decodable by readers without a dictionary-capable zstd
(today’s TS reader). Per-blob zstd + byte-identical dedup still apply.
This is the reader-safe way to get the range-request win from blob
ordering without the dict that the TS
reader can’t yet decode.
Sourcepub fn add_tile(
&mut self,
id: &TileId,
time_start: i64,
time_end: i64,
feature_count: u32,
payload: &[u8],
) -> Result<()>
pub fn add_tile( &mut self, id: &TileId, time_start: i64, time_end: i64, feature_count: u32, payload: &[u8], ) -> Result<()>
Add a tile. payload is the uncompressed tile payload (the layer frame
produced by crate::arrow_tile::encode_tile).
Sourcepub fn add_tile_with_bucket(
&mut self,
id: &TileId,
time_start: i64,
time_end: i64,
feature_count: u32,
temporal_bucket_ms: Option<u64>,
payload: &[u8],
) -> Result<()>
pub fn add_tile_with_bucket( &mut self, id: &TileId, time_start: i64, time_end: i64, feature_count: u32, temporal_bucket_ms: Option<u64>, payload: &[u8], ) -> Result<()>
Add a tile, tagging the directory entry with the temporal bucket size it represents (used when emitting temporal-LOD aggregate tiles).
Sourcepub fn add_tile_full(
&mut self,
id: &TileId,
time_start: i64,
time_end: i64,
cover_t_min: Option<i64>,
feature_count: u32,
temporal_bucket_ms: Option<u64>,
payload: &[u8],
) -> Result<()>
pub fn add_tile_full( &mut self, id: &TileId, time_start: i64, time_end: i64, cover_t_min: Option<i64>, feature_count: u32, temporal_bucket_ms: Option<u64>, payload: &[u8], ) -> Result<()>
Add a tile carrying the full directory metadata, including the tight
lower covering bound cover_t_min (the earliest feature start time
actually present — see TileEntry::cover_t_min). None leaves the
entry without a covering bound (clients fall back to time_start).
Sourcepub fn tile_count(&self) -> usize
pub fn tile_count(&self) -> usize
Number of tiles staged so far (buffered or written).
Auto Trait Implementations§
impl Freeze for ArchiveWriter
impl RefUnwindSafe for ArchiveWriter
impl Send for ArchiveWriter
impl Sync for ArchiveWriter
impl Unpin for ArchiveWriter
impl UnsafeUnpin for ArchiveWriter
impl UnwindSafe for ArchiveWriter
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more