Struct HyperFileRuntimeConfig 

pub struct HyperFileRuntimeConfig {

    pub forward_origin_concurrency: usize,
    pub forward_origin_threshold: usize,
    pub forward_origin_chunk_size: usize,
    pub segment_buffer_size: usize,
    pub segment_mpu_chunk_size: usize,
    pub segment_mpu_max_concurrency: usize,
    pub segment_max_object_bytes: usize,
    pub data_cache_blocks: usize,
    pub data_cache_dirty_max_bytes_threshold: usize,
    pub data_cache_dirty_max_blocks_threshold: usize,
    pub data_cache_dirty_max_flush_interval: u64,
    pub node_cache_blocks: usize,
    pub read_get_max_bytes: usize,
    pub parted_segment_enabled: bool,
    pub data_cache_dirty_min_bytes_to_part: usize,
    pub read_max_concurrency: usize,
    pub flush_conflict_policy: FlushConflictPolicy,
}

Fields

forward_origin_concurrency: usize

forward_origin_threshold: usize

forward_origin_chunk_size: usize

segment_buffer_size: usize

segment_mpu_chunk_size: usize

segment_mpu_max_concurrency: usize

The most parts of one segment’s multipart upload in flight at once; the next part starts as one finishes. Default 8. Zero is taken as one.

A segment longer than segment_mpu_chunk_size goes up in parts of that size. Each part in flight is a request with its own connection and TLS and HTTP buffers, and with wal also a copy of its range of the segment. In the default build a part is a view of the segment’s blocks, not a copy.

It does not bound the segment itself, which is held in memory until the upload ends whatever this is.

segment_max_object_bytes: usize

The most bytes one stored object of a checkpoint may hold. Default, and upper limit, 1 GiB: what a block pointer can address within one object.

A checkpoint larger than this is written as several objects under the parted block-pointer format — data-only parts first, then the part holding the index and the inode, which is what makes it a checkpoint — and is published as one. Under the older formats, and with wal, a checkpoint is always one object, and a flush that would need more is refused before anything is written.

Values above the limit are taken as the limit; below 16 KiB, as 16 KiB.

data_cache_blocks: usize

data_cache_dirty_max_bytes_threshold: usize

data_cache_dirty_max_blocks_threshold: usize

data_cache_dirty_max_flush_interval: u64

node_cache_blocks: usize

read_get_max_bytes: usize

Cap on the number of bytes a single S3 GET issued by the read path’s coalescing logic may cover. Larger contiguous regions are split into this-sized sub-ranges.

parted_segment_enabled: bool

Whether this container writes partial segments when memory pressure asks for it, instead of publishing a checkpoint nobody asked for.

On by default, which changes nothing for a container whose format is not crate::meta_format::BlockPtrFormat::PartedSegment — a partial segment is addressable only under that format, and the format is fixed when the container is created. So asking for the format is asking for the behaviour, and this exists to take it back: a container can be created with the format and made to behave exactly as before by clearing this.

What it changes: a dirty-data threshold, or the flush interval, writes the dirty set out as a partial segment and keeps accumulating, rather than publishing. Under a transaction or WalRecoveryMode::Barrier, where crossing a threshold is refused with OutOfMemory because nothing may publish, it writes a partial instead — which is the case this exists for.

What it does not change: an explicit flush, fdatasync, release or commit_txn still writes a consistency point, and with no partial outstanding it writes exactly the single segment it always did.

data_cache_dirty_min_bytes_to_part: usize

Least dirty data worth writing as a partial segment. Below it, a threshold crossing behaves as it always did.

A partial segment costs an object and a break in what a sequential read can coalesce, so a small one buys little memory for a lasting cost in fragmentation. Defaults to data_cache_dirty_max_bytes_threshold’s own default, which is the amount that triggers a write in the first place.

read_max_concurrency: usize

Backpressure cap on in-flight S3 GETs spawned by a single fs_read / fh_read after Level-A coalescing.

flush_conflict_policy: FlushConflictPolicy

How to resolve flush conflicts when another writer has modified the same file between our read and our write. See FlushConflictPolicy for details.

Implementations

impl HyperFileRuntimeConfig

pub fn default_large() -> Self

pub fn default_middle() -> Self

Trait Implementations

impl Clone for HyperFileRuntimeConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HyperFileRuntimeConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for HyperFileRuntimeConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for HyperFileRuntimeConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for HyperFileRuntimeConfig

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for HyperFileRuntimeConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for HyperFileRuntimeConfig