Struct MemoryClient 

pub struct MemoryClient { /* private fields */ }

In-memory client for testing purposes. Stores all data in memory using hash tables.

Implementations

impl MemoryClient

pub fn new(ctx: XetContext) -> Arc<Self>

Create a new in-memory client.

pub fn set_lifecycle_tag_deletion(&self, on: bool)

Toggle lifecycle-tag deletion mode (see Self::lifecycle_tag_deletion).

pub async fn insert_random_xorb(&self, xorb: RandomXorb) -> Result<MerkleHash>

Inserts a RandomXorb into the client’s storage and registers it in the shard for XORB block lookups. Returns the xorb hash.

This allows storing XORBs that generate their data on-the-fly, enabling testing with massive files without storing all data in memory.

pub async fn insert_random_lazy_file( &self, term_spec: &[(u64, (u64, u64))], chunk_size: usize, ) -> Result<RandomFileContents>

Upload a random file using lazy generation with RandomXorb.

This uses RandomXorb to generate XORB data on-the-fly, allowing testing with massive files without storing all data in memory.

Each term is defined as (xorb_seed, (chunk_start, chunk_end)) where:

• xorb_seed determines the random data for that XORB
• chunk_start and chunk_end define the range of chunks to include

impl MemoryClient

pub async fn get_reconstruction_v1( &self, file_id: &MerkleHash, bytes_range: Option<FileRange>, ) -> Result<Option<QueryReconstructionResponse>>

V1 reconstruction: returns per-range presigned URLs.

pub async fn get_reconstruction_v2( &self, file_id: &MerkleHash, bytes_range: Option<FileRange>, ) -> Result<Option<QueryReconstructionResponseV2>>

V2 reconstruction: returns per-xorb multi-range fetch descriptors.

Trait Implementations

impl Client for MemoryClient

fn get_file_reconstruction_info<'life0, 'life1, 'async_trait>( &'life0 self, file_hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<Option<(MDBFileInfo, Option<MerkleHash>)>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

fn query_for_global_dedup_shard<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _prefix: &'life1 str, chunk_hash: &'life2 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<Option<Bytes>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

fn acquire_upload_permit<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<ConnectionPermit>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Acquire an upload permit.

fn upload_shard<'life0, 'async_trait>( &'life0 self, shard_data: Bytes, _permit: ConnectionPermit, progress_callback: Option<ShardUploadProgressCallback>, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Upload a new shard. The optional callback receives v2 NDJSON progress events.

fn upload_xorb<'life0, 'life1, 'async_trait>( &'life0 self, _prefix: &'life1 str, serialized_xorb_object: SerializedXorbObject, progress_callback: Option<ProgressCallback>, _permit: ConnectionPermit, ) -> Pin<Box<dyn Future<Output = Result<u64>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Upload a new xorb. Optional progress callback receives (delta, completed, total) in transfer bytes.

fn get_reconstruction<'life0, 'life1, 'async_trait>( &'life0 self, file_id: &'life1 MerkleHash, bytes_range: Option<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<Option<QueryReconstructionResponseV2>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Returns reconstruction info always in V2 format. Implementations may try V2 first and fall back to V1 + convert.

fn batch_get_reconstruction<'life0, 'life1, 'async_trait>( &'life0 self, file_ids: &'life1 [MerkleHash], ) -> Pin<Box<dyn Future<Output = Result<BatchQueryReconstructionResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

fn acquire_download_permit<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<ConnectionPermit>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn get_file_term_data<'life0, 'async_trait>( &'life0 self, url_info: Box<dyn URLProvider>, _download_permit: ConnectionPermit, progress_callback: Option<ProgressCallback>, uncompressed_size_if_known: Option<usize>, ) -> Pin<Box<dyn Future<Output = Result<(Bytes, Vec<u32>)>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Optional progress callback receives (delta, completed, total) in transfer bytes. When [uncompressed_size_if_known] is Some, the returned Bytes must have len() equal to that value.

fn get_file_chunk_hashes<'life0, 'life1, 'async_trait>( &'life0 self, file_id: &'life1 MerkleHash, dirty_ranges: Vec<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<FileChunkHashesResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Compute chunk-aligned dirty windows + opaque gap [MerkleHashSubtree] summaries for the given file, narrowed to dirty_ranges.

impl DeletionControlableClient for MemoryClient

fn list_shard_entries<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<MerkleHash>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns all shard hashes from shard files on disk.

fn get_shard_bytes<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<Bytes>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Returns a shard’s raw bytes by its hash.

fn delete_shard_entry<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Deletes a shard file by its hash.

fn list_file_shard_entries<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<(MerkleHash, MerkleHash)>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns (file_hash, shard_hash) tuples for all files across all shards.

fn delete_file_entry<'life0, 'life1, 'async_trait>( &'life0 self, file_hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Soft-deletes a file entry by hash. The file is hidden from reconstruction and listing paths without rewriting shard files.

fn remove_shard_dedup_entries<'life0, 'life1, 'async_trait>( &'life0 self, shard_hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Removes all global-dedup table entries contributed by the given shard. Called by GC Stage 4 before replacing or discarding a shard.

fn delete_xorb<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Deletes a XORB by hash.

fn list_xorbs_and_tags<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<(MerkleHash, ObjectTag)>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns all XORB hashes with their associated object tags.

fn delete_xorb_if_tag_matches<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, tag: &'life2 ObjectTag, ) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Deletes a XORB only if its current tag matches the provided tag. Returns Ok(true) if deleted, Ok(false) if the tag did not match.

fn list_shards_with_tags<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<(MerkleHash, ObjectTag)>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns all shard hashes with their associated object tags.

fn delete_shard_if_tag_matches<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, tag: &'life2 ObjectTag, ) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Deletes a shard only if its current tag matches the provided tag. Returns Ok(true) if deleted, Ok(false) if the tag did not match.

fn verify_integrity<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Verifies referential integrity of all shards on disk.

fn verify_all_reachable<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Verifies completeness: after GC convergence, all on-disk data must be reachable.

impl DirectAccessClient for MemoryClient

fn set_fetch_term_url_expiration(&self, expiration: Duration)

Sets the expiration duration for fetch term URLs.

fn set_global_dedup_shard_expiration(&self, expiration: Option<Duration>)

Sets the expiration duration for global dedup shards.

fn set_max_ranges_per_fetch(&self, max_ranges: usize)

Sets the maximum number of byte ranges per XorbMultiRangeFetch entry in V2 reconstruction responses.

fn disable_v2_endpoints(&self, status_code: u16)

Disables V2 endpoints (reconstruction and shard upload) with the given HTTP status code. When disabled, /v2/* handlers return this status, forcing clients to fall back to V1. Pass 0 to re-enable.

fn v2_disabled_status_code(&self) -> u16

Returns the HTTP status code V2 endpoints should return when disabled, or 0 if V2 is enabled.

fn get_reconstruction_v1<'life0, 'life1, 'async_trait>( &'life0 self, file_id: &'life1 MerkleHash, bytes_range: Option<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<Option<QueryReconstructionResponse>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

V1 reconstruction: returns per-range presigned URLs.

fn get_reconstruction_v2<'life0, 'life1, 'async_trait>( &'life0 self, file_id: &'life1 MerkleHash, bytes_range: Option<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<Option<QueryReconstructionResponseV2>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

V2 reconstruction: returns per-xorb multi-range fetch descriptors.

fn set_api_delay_range(&self, delay_range: Option<Range<Duration>>)

Sets a random delay range for all Client API calls.

fn apply_api_delay<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Applies the configured API delay if set.

fn list_xorbs<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<MerkleHash>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns all XORB hashes stored in this client.

fn get_full_xorb<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<Bytes>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get all uncompressed bytes from a XORB.

fn get_xorb_ranges<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, chunk_ranges: Vec<(u32, u32)>, ) -> Pin<Box<dyn Future<Output = Result<Vec<Bytes>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get uncompressed bytes from a XORB within chunk ranges. Each tuple represents a chunk index range [start, end).

fn xorb_length<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<u32>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get the length of the uncompressed XORB data.

fn xorb_exists<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<bool>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Check if a XORB exists.

fn xorb_footer<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<XorbObject>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get the XorbObject footer/metadata for a XORB.

fn get_file_size<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<u64>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get the file size for a given file hash.

fn get_file_data<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, byte_range: Option<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<Bytes>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get file data, optionally within a byte range.

fn get_xorb_raw_bytes<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, byte_range: Option<FileRange>, ) -> Pin<Box<dyn Future<Output = Result<Bytes>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get raw (serialized) bytes from a XORB, optionally within a byte range.

fn xorb_raw_length<'life0, 'life1, 'async_trait>( &'life0 self, hash: &'life1 MerkleHash, ) -> Pin<Box<dyn Future<Output = Result<u64>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Get the total length of the raw (serialized) XORB data.

fn fetch_term_data<'life0, 'async_trait>( &'life0 self, hash: MerkleHash, fetch_term: XorbReconstructionFetchInfo, ) -> Pin<Box<dyn Future<Output = Result<(Bytes, Vec<u32>)>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Fetches term data for a given hash and fetch term. Returns (data bytes, chunk byte indices) matching Client::get_file_term_data.