Struct Configuration 

pub struct Configuration { /* private fields */ }

Fully resolved crawler configuration.

Unlike the interface’s non-optional storage getter, core exposes an optional client because it cannot depend on millipede-storage-memory without a dependency cycle. The crawler builder takes the client from CrawlerBuilder::storage_client or Configuration::storage_client and returns CrawlerBuildError::MissingStorage when neither is set; default in-memory wiring lives with the umbrella crate’s examples.

Implementations

impl Configuration

pub fn builder() -> ConfigurationBuilder

Creates an empty configuration builder.

pub fn events(&self) -> &EventBus

Returns the crawler event bus.

pub fn default_dataset_id(&self) -> &str

Returns the default dataset identifier.

pub fn default_key_value_store_id(&self) -> &str

Returns the default key-value store identifier.

pub fn default_request_queue_id(&self) -> &str

Returns the default request queue identifier.

pub fn storage_dir(&self) -> &Path

Returns the storage directory.

pub fn max_used_cpu_ratio(&self) -> Option<f32>

Returns the maximum used CPU ratio, when configured.

pub fn available_memory_ratio(&self) -> Option<f32>

Returns the available memory ratio, when configured.

pub fn memory_bytes(&self) -> Option<u64>

Returns the memory limit in bytes, when configured.

pub fn persist_state_interval(&self) -> Duration

Returns the interval between state persistence events.

pub fn purge_on_start(&self) -> bool

Returns whether storage is purged at startup.

pub fn log_level(&self) -> LogLevel

Returns the logging verbosity.

pub fn storage_client(&self) -> Option<&Arc<dyn StorageClient>>

Returns the configured storage client, when one was injected.

Trait Implementations

impl Debug for Configuration

fn fmt(&self, formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Configuration

fn default() -> Self

Returns the “default value” for a type.