Struct ConfigValueGroup 

pub struct ConfigValueGroup {
    pub min_reconstruction_fetch_size: ByteSize,
    pub max_reconstruction_fetch_size: ByteSize,
    pub download_buffer_size: ByteSize,
    pub download_buffer_perfile_size: ByteSize,
    pub download_buffer_limit: ByteSize,
    pub completion_rate_estimator_half_life: f64,
    pub target_block_completion_time: Duration,
    pub min_prefetch_buffer: ByteSize,
    pub use_vectored_write: bool,
}

ConfigValueGroup struct containing all configurable values

Fields

min_reconstruction_fetch_size: ByteSize

The minimum size of a single fetch request during reconstruction. Individual fetches will request reconstruction terms representing at least this amount of data.

The default value is 256MB.

Use the environment variable HF_XET_RECONSTRUCTION_MIN_RECONSTRUCTION_FETCH_SIZE to set this value.

max_reconstruction_fetch_size: ByteSize

The maximum size of a single fetch request during reconstruction. Individual fetches will not request reconstruction terms representing more than this amount of data.

The default value is 8GB.

Use the environment variable HF_XET_RECONSTRUCTION_MAX_RECONSTRUCTION_FETCH_SIZE to set this value.

download_buffer_size: ByteSize

The amount of download buffer always available for file reconstruction. The full buffer size will be this plus the number of simultaneous active file downloads times the per file size up to the global limit of download_buffer_limit.

The default value is 2GB.

Use the environment variable HF_XET_RECONSTRUCTION_DOWNLOAD_BUFFER_SIZE to set this value.

download_buffer_perfile_size: ByteSize

The additional download buffer allocated per active file download. Each active file download increases the total buffer by this amount.

The default value is 512MB.

Use the environment variable HF_XET_RECONSTRUCTION_DOWNLOAD_BUFFER_PERFILE_SIZE to set this value.

download_buffer_limit: ByteSize

The maximum total download buffer allowed during file reconstruction. The buffer will not grow beyond this limit regardless of the number of concurrent downloads.

The default value is 8GB.

Use the environment variable HF_XET_RECONSTRUCTION_DOWNLOAD_BUFFER_LIMIT to set this value.

completion_rate_estimator_half_life: f64

The half-life in count of observations for the exponentially weighted moving average used to estimate completion rate during reconstruction prefetching.

The default value is 4 observations..

Use the environment variable HF_XET_RECONSTRUCTION_COMPLETION_RATE_ESTIMATOR_HALF_LIFE to set this value.

target_block_completion_time: Duration

The target time for completing a prefetch block during reconstruction. This is used to determine how much data to prefetch ahead.

The default value is 15 minutes.

Use the environment variable HF_XET_RECONSTRUCTION_TARGET_BLOCK_COMPLETION_TIME to set this value.

min_prefetch_buffer: ByteSize

The minimum size of the prefetch buffer during reconstruction. The prefetch system will maintain terms representing at least this much always prefetched, no matter the estimated completion time.

The default value is 1gb.

Use the environment variable HF_XET_RECONSTRUCTION_MIN_PREFETCH_BUFFER to set this value.

use_vectored_write: bool

Whether to use vectorized writes (write_vectored) during file reconstruction. When true, multiple pending writes are batched and written using write_vectored. When false, standard sequential writes are used.

The default value is true.

Use the environment variable HF_XET_RECONSTRUCTION_USE_VECTORED_WRITE to set this value.

Implementations

impl ConfigValueGroup

pub fn new() -> Self

Create a new instance with default values only (no environment variable overrides).

pub fn apply_env_overrides(&mut self)

Apply environment variable overrides to this configuration group.

The group name is derived from the module path. For example, in module xet_config::groups::data, the env var for TEST_INT would be HF_XET_DATA_TEST_INT.

pub fn field_names() -> &'static [&'static str]

Returns the list of field names in this configuration group.

pub fn get(&self, name: &str) -> Result<String, ConfigError>

Get a configuration field’s string representation by name.

Trait Implementations

impl AsRef<ConfigValueGroup> for ConfigValueGroup

fn as_ref(&self) -> &ConfigValueGroup

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for ConfigValueGroup

fn clone(&self) -> ConfigValueGroup

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ConfigValueGroup

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ConfigValueGroup

fn default() -> Self

Returns the “default value” for a type.