Expand description
Shared runtime for surreal-sync: process setup, SurrealDB connection settings, transform loading, and the apply pipeline.
Origin from-* crates re-export the pieces embedders need. The stock CLI
may still auto-detect SurrealDB major version (linking both SDKs); the
documented embed path uses a pre-built sink or run::<OneSinkType>.
Sink crates use the default features (no CLI argument parsing). Enable cli
only if you need [SurrealCliOpts].
Enable checkpoint_fs (default) for checkpoint_fs::FilesystemStore.
§Sink + checkpoint typing
SinkConnect/SinkWithCheckpoints/SurrealConfig— defined insurreal_sync_core, re-exported here for existing importsSinkWithCheckpointsopens a matching Surreal-tableCheckpointStorefor--checkpoints-surreal-table(implemented onSurreal2Sink/Surreal3Sink)
Re-exports§
pub use pipeline::apply_changes;pub use pipeline::apply_changes_with;pub use pipeline::apply_relation_changes;pub use pipeline::apply_relation_changes_with;pub use pipeline::ensure_command_resolvable;pub use pipeline::load_pipeline_and_opts;pub use pipeline::load_transforms_config;pub use pipeline::parse_humantime;pub use pipeline::parse_transforms_toml;pub use pipeline::relation_wire_batch_id;pub use pipeline::run_adhoc_snapshot_tables;pub use pipeline::run_adhoc_snapshot_tables_with_transforms;pub use pipeline::run_change_feed;pub use pipeline::run_change_feed_with;pub use pipeline::run_interleaved_snapshot;pub use pipeline::run_interleaved_snapshot_with_resume;pub use pipeline::run_interleaved_snapshot_with_resume_and_transforms;pub use pipeline::run_interleaved_snapshot_with_transforms;pub use pipeline::run_source_runtime;pub use pipeline::run_source_runtime_with;pub use pipeline::write_relations;pub use pipeline::write_relations_with;pub use pipeline::write_rows;pub use pipeline::write_rows_with;pub use pipeline::AdhocApply;pub use pipeline::ApplyContext;pub use pipeline::ApplyEvent;pub use pipeline::ApplyOpts;pub use pipeline::BatchTransformer;pub use pipeline::ChangeFeed;pub use pipeline::ChangeFeedDriver;pub use pipeline::ChangeFeedRef;pub use pipeline::CheckpointPolicy;pub use pipeline::ChildStdioMode;pub use pipeline::CommandStageConfig;pub use pipeline::ConfiguredStage;pub use pipeline::ControlSignal;pub use pipeline::CowBatch;pub use pipeline::ExternalTransform;pub use pipeline::ExternalTransport;pub use pipeline::FailurePolicy;pub use pipeline::FlattenId;pub use pipeline::FlattenIdStageConfig;pub use pipeline::Framer;pub use pipeline::FramerKind;pub use pipeline::InterleavedSnapshotConfig;pub use pipeline::InterleavedSnapshotResult;pub use pipeline::ManagerCheckpointer;pub use pipeline::NdjsonFramer;pub use pipeline::NoopCheckpointer;pub use pipeline::PersistentChildStdio;pub use pipeline::Pipeline;pub use pipeline::PipelineSection;pub use pipeline::PkTuple;pub use pipeline::PositionedChange;pub use pipeline::PositionedEvent;pub use pipeline::ReconciliationEvent;pub use pipeline::ReconciliationPos;pub use pipeline::RelationChunkDriver;pub use pipeline::RelationChunkSource;pub use pipeline::RequestHeader;pub use pipeline::ResponseHeader;pub use pipeline::RetryPolicy;pub use pipeline::RowChunkDriver;pub use pipeline::RowChunkSource;pub use pipeline::RuntimeExit;pub use pipeline::SnapshotSignal;pub use pipeline::SnapshotTransforms;pub use pipeline::SourceDriver;pub use pipeline::SourceRuntimeOpts;pub use pipeline::Stage;pub use pipeline::StdioConfig;pub use pipeline::StopReason;pub use pipeline::TableSpec;pub use pipeline::TransformsConfig;pub use pipeline::TransientChildStdio;pub use pipeline::WatermarkKind;pub use pipeline::WatermarkSource;pub use pipeline::WireItemKind;pub use pipeline::WireResponse;pub use pipeline::DEFAULT_CHUNK_SIZE;pub use pipeline::DEFAULT_FLATTEN_ID_SEPARATOR;pub use pipeline::RELATION_WIRE_BATCH_ID_BIT;
Modules§
- checkpoint_
fs - Store checkpoints on the filesystem. Filesystem checkpoint storage for surreal-sync.
- pipeline
- Shared apply loop, transform pipeline, and interleaved snapshot engine. Shared apply loop for surreal-sync: transform → ordered SurrealDB write → advance watermark.
Structs§
- Interleaved
Snapshot Checkpoint - Resumable checkpoint describing the state of an in-progress interleaved snapshot.
- Passthrough
- No-op
InPlaceTransform. Useful for tests and library completeness. - Snapshot
Table Progress - Progress for a single table within an interleaved snapshot.
- Surreal
Config - Plain-field SurrealDB connection and write options (no clap).
Traits§
- InPlace
Transform - Mutate-only, same-length transform over sync docs.
- Sink
Connect - Connect a single Surreal sink implementation from plain config.
- Sink
With Checkpoints - Sink that can open a SurrealDB-table
CheckpointStorefor the same SDK major. - Snapshot
Checkpointer - Receives resumable snapshot checkpoints emitted per chunk.
Functions§
- init
- Install the TLS crypto provider and initialize tracing.
- load_
transforms_ from_ args - Load
--transforms-configinto aPipeline+ApplyOpts. - merge_
inplace_ boxed - Like
merge_inplace_transforms, but accepts mixed concrete stage types viaBox<dyn InPlaceTransform>(needed for heterogeneous lists). - merge_
inplace_ transforms - Load optional
--transforms-config, then append in-processInPlaceTransformstages (TOML stages run first). - parse_
duration_ to_ secs - Parse a duration string like “1h”, “30m”, “300s”, “300” into seconds.