Crate ceres_core 

Ceres Core - Domain types, business logic, and services.

This crate provides the core functionality for Ceres, including:

• Domain models: Dataset, SearchResult, Portal, etc.
• Business logic: Delta detection, statistics tracking
• Services: HarvestService for portal synchronization, SearchService for semantic search, ExportService for streaming exports
• Traits: EmbeddingProvider, DatasetStore, PortalClient for dependency injection
• Progress reporting: ProgressReporter trait for decoupled logging/UI

Architecture

This crate is designed to be reusable by different frontends (CLI, server, etc.). Business logic is decoupled from I/O concerns through traits:

Server Development Notes (ceres-server)

TODO(server): Use utoipa for automatic OpenAPI documentation When creating ceres-server, use the utoipa crate (compatible with Axum/Actix) to auto-generate OpenAPI specs from Rust types and handlers via derive macros. Expose /swagger-ui endpoint for interactive API documentation. This makes the API immediately usable by frontend developers without reading code.

• EmbeddingProvider - abstracts embedding generation (e.g., Gemini API)
• DatasetStore - abstracts database operations (e.g., PostgreSQL)
• PortalClient - abstracts portal access (e.g., CKAN API)

Example

use ceres_core::{HarvestService, SearchService};
use ceres_core::progress::TracingReporter;

// Create services with your implementations
let harvest = HarvestService::new(store, embedding, portal_factory);
let reporter = TracingReporter;
let stats = harvest.sync_portal_with_progress("https://data.gov/api/3", &reporter).await?;

// Semantic search
let search = SearchService::new(store, embedding);
let results = search.search("climate data", 10).await?;

Re-exports

pub use circuit_breaker::CircuitBreaker;

pub use circuit_breaker::CircuitBreakerConfig;

pub use circuit_breaker::CircuitBreakerError;

pub use circuit_breaker::CircuitBreakerStats;

pub use circuit_breaker::CircuitState;

pub use config::DbConfig;

pub use config::HttpConfig;

pub use config::PortalEntry;

pub use config::PortalsConfig;

pub use config::SyncConfig;

pub use config::default_config_path;

pub use config::load_portals_config;

pub use error::AppError;

pub use models::DatabaseStats;

pub use models::Dataset;

pub use models::NewDataset;

pub use models::Portal;

pub use models::SearchResult;

pub use sync::AtomicSyncStats;

pub use sync::BatchHarvestSummary;

pub use sync::PortalHarvestResult;

pub use sync::ReprocessingDecision;

pub use sync::SyncOutcome;

pub use sync::SyncResult;

pub use sync::SyncStats;

pub use sync::SyncStatus;

pub use sync::needs_reprocessing;

pub use progress::HarvestEvent;

pub use progress::ProgressReporter;

pub use progress::SilentReporter;

pub use progress::TracingReporter;

pub use traits::DatasetStore;

pub use traits::EmbeddingProvider;

pub use traits::PortalClient;

pub use traits::PortalClientFactory;

pub use export::ExportFormat;

pub use export::ExportService;

pub use harvest::HarvestService;

pub use search::SearchService;

pub use job::CreateJobRequest;

pub use job::HarvestJob;

pub use job::JobStatus;

pub use job::RetryConfig;

pub use job::WorkerConfig;

pub use job_queue::JobQueue;

pub use worker::SilentWorkerReporter;

pub use worker::TracingWorkerReporter;

pub use worker::WorkerEvent;

pub use worker::WorkerReporter;

pub use worker::WorkerService;

Modules

circuit_breaker

Circuit breaker pattern for API resilience.

config

Configuration types for Ceres components.

error

export

Export service for streaming dataset exports.

harvest

Harvest service for portal synchronization.

job

Job queue types for persistent harvest job management.

job_queue

Job queue trait for abstracting job persistence.

models

progress

Progress reporting for harvest operations.

search

Search service for semantic dataset queries.

sync

Sync service layer for portal synchronization logic.

traits

Trait definitions for external dependencies.

worker

Worker service for processing harvest jobs from the queue.