Crate ceres_core 

Ceres Core - Domain types, business logic, and services.

This crate provides the core functionality for Ceres, including:

• Domain models: Dataset, SearchResult, [Portal], etc.
• Business logic: Delta detection, statistics tracking
• Services: HarvestService for portal synchronization, SearchService for semantic search, ExportService for streaming exports
• Traits: EmbeddingProvider, DatasetStore, PortalClient for dependency injection
• Progress reporting: ProgressReporter trait for decoupled logging/UI

Architecture

This crate is designed to be reusable by different frontends (CLI, server, etc.). Business logic is decoupled from I/O concerns through traits:

REST API (ceres-server)

The REST API uses utoipa for automatic OpenAPI documentation with Swagger UI at /swagger-ui. See ceres_server::openapi for the implementation.

• EmbeddingProvider - abstracts embedding generation (e.g., Gemini API)
• DatasetStore - abstracts database operations (e.g., PostgreSQL)
• PortalClient - abstracts portal access (e.g., CKAN API)

Example

use ceres_core::{HarvestService, PortalType, SearchService};
use ceres_core::progress::TracingReporter;

// Create services with your implementations
let harvest = HarvestService::new(store, embedding, portal_factory);
let reporter = TracingReporter;
let stats = harvest
    .sync_portal_with_progress("https://data.gov/api/3", None, "en", &reporter, PortalType::Ckan)
    .await?;

// Semantic search
let search = SearchService::new(store, embedding);
let results = search.search("climate data", 10).await?;

Re-exports

pub use circuit_breaker::CircuitBreaker;

pub use circuit_breaker::CircuitBreakerConfig;

pub use circuit_breaker::CircuitBreakerError;

pub use circuit_breaker::CircuitBreakerStats;

pub use circuit_breaker::CircuitState;

pub use config::DbConfig;

pub use config::HttpConfig;

pub use config::PortalEntry;

pub use config::PortalType;

pub use config::PortalsConfig;

pub use config::SyncConfig;

pub use config::default_config_path;

pub use config::load_portals_config;

pub use error::AppError;

pub use i18n::LocalizedField;

pub use models::DatabaseStats;

pub use models::Dataset;

pub use models::NewDataset;

pub use models::SearchResult;

pub use sync::AlwaysReprocessDetector;

pub use sync::AtomicSyncStats;

pub use sync::BatchHarvestSummary;

pub use sync::ContentHashDetector;

pub use sync::DeltaDetector;

pub use sync::PortalHarvestResult;

pub use sync::ReprocessingDecision;

pub use sync::SyncOutcome;

pub use sync::SyncResult;

pub use sync::SyncStats;

pub use sync::SyncStatus;

pub use sync::needs_reprocessing;

pub use progress::HarvestEvent;

pub use progress::ProgressReporter;

pub use progress::SilentReporter;

pub use progress::TracingReporter;

pub use traits::DatasetStore;

pub use traits::EmbeddingProvider;

pub use traits::PortalClient;

pub use traits::PortalClientFactory;

pub use export::ExportFormat;

pub use export::ExportService;

pub use harvest::HarvestService;

pub use parquet_export::ParquetExportConfig;

pub use parquet_export::ParquetExportResult;

pub use parquet_export::ParquetExportService;

pub use search::SearchService;

pub use job::CreateJobRequest;

pub use job::HarvestJob;

pub use job::JobStatus;

pub use job::RetryConfig;

pub use job::WorkerConfig;

pub use job_queue::JobQueue;

pub use worker::SilentWorkerReporter;

pub use worker::TracingWorkerReporter;

pub use worker::WorkerEvent;

pub use worker::WorkerReporter;

pub use worker::WorkerService;

Modules

circuit_breaker

Circuit breaker pattern for API resilience.

config

Configuration types for Ceres components.

error

export

Export service for streaming dataset exports.

harvest

Harvest service for portal synchronization.

i18n

Multilingual field support for open data portals.

job

Job queue types for persistent harvest job management.

job_queue

Job queue trait for abstracting job persistence.

models

parquet_export

Parquet export service for publishing a curated open data index.

progress

Progress reporting for harvest operations.

search

Search service for semantic dataset queries.

sync

Sync service layer for portal synchronization logic.

traits

Trait definitions for external dependencies.

worker

Worker service for processing harvest jobs from the queue.