Struct SyncConfig 

pub struct SyncConfig {
    pub concurrency: usize,
    pub embedding_batch_size: usize,
    pub upsert_batch_size: usize,
    pub force_full_sync: bool,
    pub dry_run: bool,
    pub circuit_breaker: CircuitBreakerConfig,
}

Portal synchronization configuration (combined harvest + embed).

This is a convenience type that composes HarvestConfig and EmbeddingServiceConfig for the common harvest-then-embed workflow. Used by crate::HarvestPipeline and kept for backward compatibility.

TODO(config): Support CLI arg --concurrency and env var SYNC_CONCURRENCY Optimal value depends on portal rate limits and system resources. Consider auto-tuning based on API response times.

Fields

concurrency: usize

Number of concurrent dataset processing tasks.

embedding_batch_size: usize

Maximum number of texts per embedding API batch call. The actual batch size is min(this, provider.max_batch_size()).

upsert_batch_size: usize

Maximum number of datasets per DB upsert batch.

force_full_sync: bool

Force full sync even if incremental sync is available.

dry_run: bool

Preview mode: fetch and compare datasets without writing to DB or calling embedding API.

circuit_breaker: CircuitBreakerConfig

Circuit breaker configuration for API resilience.

Implementations

impl SyncConfig

pub fn with_full_sync(self) -> Self

Creates a new SyncConfig with force_full_sync enabled.

pub fn with_dry_run(self) -> Self

Creates a new SyncConfig with dry_run enabled.

pub fn with_embedding_batch_size(self, size: usize) -> Self

Creates a new SyncConfig with a custom embedding batch size.

pub fn with_circuit_breaker(self, config: CircuitBreakerConfig) -> Self

Creates a new SyncConfig with custom circuit breaker configuration.

pub fn harvest_config(&self) -> HarvestConfig

Extracts the harvest-only configuration.

pub fn embedding_service_config(&self) -> EmbeddingServiceConfig

Extracts the embedding service configuration.

Trait Implementations

impl Clone for SyncConfig

fn clone(&self) -> SyncConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for SyncConfig

fn default() -> Self

Returns the “default value” for a type.