Skip to main content

Crate surreal_sync_runtime

Crate surreal_sync_runtime 

Source
Expand description

Shared runtime for surreal-sync: process setup, SurrealDB connection settings, transform loading, and the apply pipeline.

Origin from-* crates re-export the pieces embedders need. The stock CLI may still auto-detect SurrealDB major version (linking both SDKs); the documented embed path uses a pre-built sink or run::<OneSinkType>.

Sink crates use the default features (no CLI argument parsing). Enable cli only if you need [SurrealCliOpts].

Enable checkpoint_fs (default) for checkpoint_fs::FilesystemStore.

§Sink + checkpoint typing

Re-exports§

pub use pipeline::apply_changes;
pub use pipeline::apply_changes_with;
pub use pipeline::apply_relation_changes;
pub use pipeline::apply_relation_changes_with;
pub use pipeline::ensure_command_resolvable;
pub use pipeline::load_pipeline_and_opts;
pub use pipeline::load_transforms_config;
pub use pipeline::parse_humantime;
pub use pipeline::parse_transforms_toml;
pub use pipeline::relation_wire_batch_id;
pub use pipeline::run_adhoc_snapshot_tables;
pub use pipeline::run_adhoc_snapshot_tables_with_transforms;
pub use pipeline::run_change_feed;
pub use pipeline::run_change_feed_with;
pub use pipeline::run_interleaved_snapshot;
pub use pipeline::run_interleaved_snapshot_with_resume;
pub use pipeline::run_interleaved_snapshot_with_resume_and_transforms;
pub use pipeline::run_interleaved_snapshot_with_transforms;
pub use pipeline::run_source_runtime;
pub use pipeline::run_source_runtime_with;
pub use pipeline::write_relations;
pub use pipeline::write_relations_with;
pub use pipeline::write_rows;
pub use pipeline::write_rows_with;
pub use pipeline::AdhocApply;
pub use pipeline::ApplyContext;
pub use pipeline::ApplyEvent;
pub use pipeline::ApplyOpts;
pub use pipeline::BatchTransformer;
pub use pipeline::ChangeFeed;
pub use pipeline::ChangeFeedDriver;
pub use pipeline::ChangeFeedRef;
pub use pipeline::CheckpointPolicy;
pub use pipeline::ChildStdioMode;
pub use pipeline::CommandStageConfig;
pub use pipeline::ConfiguredStage;
pub use pipeline::ControlSignal;
pub use pipeline::CowBatch;
pub use pipeline::ExternalTransform;
pub use pipeline::ExternalTransport;
pub use pipeline::FailurePolicy;
pub use pipeline::FlattenId;
pub use pipeline::FlattenIdStageConfig;
pub use pipeline::Framer;
pub use pipeline::FramerKind;
pub use pipeline::InterleavedSnapshotConfig;
pub use pipeline::InterleavedSnapshotResult;
pub use pipeline::ManagerCheckpointer;
pub use pipeline::NdjsonFramer;
pub use pipeline::NoopCheckpointer;
pub use pipeline::PersistentChildStdio;
pub use pipeline::Pipeline;
pub use pipeline::PipelineSection;
pub use pipeline::PkTuple;
pub use pipeline::PositionedChange;
pub use pipeline::PositionedEvent;
pub use pipeline::ReconciliationEvent;
pub use pipeline::ReconciliationPos;
pub use pipeline::RelationChunkDriver;
pub use pipeline::RelationChunkSource;
pub use pipeline::RequestHeader;
pub use pipeline::ResponseHeader;
pub use pipeline::RetryPolicy;
pub use pipeline::RowChunkDriver;
pub use pipeline::RowChunkSource;
pub use pipeline::RuntimeExit;
pub use pipeline::SnapshotSignal;
pub use pipeline::SnapshotTransforms;
pub use pipeline::SourceDriver;
pub use pipeline::SourceRuntimeOpts;
pub use pipeline::Stage;
pub use pipeline::StdioConfig;
pub use pipeline::StopReason;
pub use pipeline::TableSpec;
pub use pipeline::TransformsConfig;
pub use pipeline::TransientChildStdio;
pub use pipeline::WatermarkKind;
pub use pipeline::WatermarkSource;
pub use pipeline::WireItemKind;
pub use pipeline::WireResponse;
pub use pipeline::DEFAULT_CHUNK_SIZE;
pub use pipeline::DEFAULT_FLATTEN_ID_SEPARATOR;
pub use pipeline::RELATION_WIRE_BATCH_ID_BIT;

Modules§

checkpoint_fs
Store checkpoints on the filesystem. Filesystem checkpoint storage for surreal-sync.
pipeline
Shared apply loop, transform pipeline, and interleaved snapshot engine. Shared apply loop for surreal-sync: transform → ordered SurrealDB write → advance watermark.

Structs§

InterleavedSnapshotCheckpoint
Resumable checkpoint describing the state of an in-progress interleaved snapshot.
Passthrough
No-op InPlaceTransform. Useful for tests and library completeness.
SnapshotTableProgress
Progress for a single table within an interleaved snapshot.
SurrealConfig
Plain-field SurrealDB connection and write options (no clap).

Traits§

InPlaceTransform
Mutate-only, same-length transform over sync docs.
SinkConnect
Connect a single Surreal sink implementation from plain config.
SinkWithCheckpoints
Sink that can open a SurrealDB-table CheckpointStore for the same SDK major.
SnapshotCheckpointer
Receives resumable snapshot checkpoints emitted per chunk.

Functions§

init
Install the TLS crypto provider and initialize tracing.
load_transforms_from_args
Load --transforms-config into a Pipeline + ApplyOpts.
merge_inplace_boxed
Like merge_inplace_transforms, but accepts mixed concrete stage types via Box<dyn InPlaceTransform> (needed for heterogeneous lists).
merge_inplace_transforms
Load optional --transforms-config, then append in-process InPlaceTransform stages (TOML stages run first).
parse_duration_to_secs
Parse a duration string like “1h”, “30m”, “300s”, “300” into seconds.