Type Alias XorbConfig 

pub type XorbConfig = ConfigValueGroup;

Aliased Type

pub struct XorbConfig {
    pub compression_scheme_retest_interval: usize,
    pub compression_policy: ConfigEnum,
    pub simulation_max_bytes: Option<ByteSize>,
    pub simulation_max_chunks: Option<usize>,
}

Fields

compression_scheme_retest_interval: usize

How often should we retest the compression scheme? Determining the optimal compression scheme takes time, but it also minimizes the storage costs of the data.

If set to zero, it’s set once per file block per xorb.

The default value is 32.

Use the environment variable HF_XET_XORB_COMPRESSION_SCHEME_RETEST_INTERVAL to set this value.

compression_policy: ConfigEnum

Compression policy for xorb data. Valid values: “” or “auto” for automatic detection, “none”, “lz4”, “bg4-lz4”. When set to “” or “auto”, the best compression scheme is chosen based on data analysis.

The default value is “auto” (auto-detect).

Use the environment variable HF_XET_XORB_COMPRESSION_POLICY to set this value.

simulation_max_bytes: Option<ByteSize>

Override the maximum xorb size in bytes for simulation mode. When set to Some(value), this overrides the hard-coded MAX_XORB_BYTES cutting threshold in simulation builds. When None (default), the standard constant is used.

Only effective when the simulation feature is enabled.

Use the environment variable HF_XET_XORB_SIMULATION_MAX_BYTES to set this value.

simulation_max_chunks: Option<usize>

Override the maximum xorb chunk count for simulation mode. When set to Some(value), this overrides the hard-coded MAX_XORB_CHUNKS cutting threshold in simulation builds. When None (default), the standard constant is used.

Only effective when the simulation feature is enabled.

Use the environment variable HF_XET_XORB_SIMULATION_MAX_CHUNKS to set this value.