Crate cognee_core 

Core runtime primitives for the cognee pipeline.

This crate provides:

• AsyncRuntime — a thin wrapper around a Tokio tokio::runtime::Runtime.
• RayonThreadPool / CpuPool / CpuPoolExt — a Rayon-backed CPU thread pool with an async interface for offloading CPU-intensive work from Tokio workers.
• CancellationHandle / CancellationToken — cooperative task cancellation via a tokio::sync::watch channel pair.
• ProgressToken — a lock-free, clone-able progress counter.
• TaskContext / TaskContextBuilder — a bundle of all runtime services (thread pool, databases, cancellation, progress) passed into pipeline tasks.
• Task / Pipeline — composable unit of work and ordered executor with fan-out batching and retry support.

Re-exports

pub use cancellation::CancellationHandle;

pub use cancellation::CancellationToken;

pub use cancellation::cancellation_pair;

pub use error::CoreError;

pub use exec_status::ExecStatusManager;

pub use exec_status::NoopExecStatusManager;

pub use pipeline::DataIdFn;

pub use pipeline::ExecutionError;

pub use pipeline::NoopWatcher;

pub use pipeline::Pipeline;

pub use pipeline::PipelineBuilder;

pub use pipeline::PipelineRunHandle;

pub use pipeline::PipelineRunInfo;

pub use pipeline::PipelineRunResult;

pub use pipeline::PipelineRunStatus;

pub use pipeline::PipelineStatus;

pub use pipeline::PipelineWatcher;

pub use pipeline::RetryDelay;

pub use pipeline::RetryPolicy;

pub use pipeline::TaskStatus;

pub use pipeline::execute;

pub use pipeline::execute_blocking;

pub use pipeline::execute_in_background;

pub use progress::ProgressToken;

pub use provenance::ProvenanceContext;

pub use provenance::extract_content_hash_from_value;

pub use provenance::extract_node_set_from_value;

pub use provenance::stamp_tree;

pub use provenance::stamp_tree_dyn;

pub use rate_limiter::RateLimiter;

pub use rate_limiter::SemaphoreLimiter;

pub use rate_limiter::TokenBucketLimiter;

pub use runtime::AsyncRuntime;

pub use sentinels::DroppedSentinel;

pub use sentinels::PassthroughSentinel;

pub use sentinels::is_dropped;

pub use sentinels::is_passthrough;

pub use task::AsyncBatchFn;

pub use task::AsyncFn;

pub use task::AsyncStreamBatchFn;

pub use task::AsyncStreamFn;

pub use task::SyncBatchFn;

pub use task::SyncFn;

pub use task::SyncIterBatchFn;

pub use task::SyncIterFn;

pub use task::Tagged;

pub use task::TaggedMeta;

pub use task::Task;

pub use task::TaskCall;

pub use task::TaskError;

pub use task::TaskInfo;

pub use task::TypedTask;

pub use task::Value;

pub use task::ValueIter;

pub use task::ValueStream;

pub use task::downcast_value;

pub use task::extract_node_set;

pub use task_context::PipelineContext;

pub use task_context::TaskContext;

pub use task_context::TaskContextBuilder;

pub use thread_pool::CpuPool;

pub use thread_pool::CpuPoolExt;

pub use thread_pool::RayonThreadPool;

Modules

cancellation

error

exec_status

pipeline

progress

provenance

Provenance stamping for DataPoints emitted by pipeline tasks.

rate_limiter

Proactive request-rate throttling for pipeline tasks.

runtime

sentinels

Control-flow sentinel values that pipeline tasks return to steer the executor. Sentinels are ordinary Values (via the blanket impl<T> Value for T in task.rs), so no manual trait impl is needed.

task

task_context

thread_pool

Traits

HasDataPoint

Read / write access to the embedded DataPoint of a typed container, plus a hook to recurse into nested DataPoint-bearing children.

PipelineRunRepository

Persistence abstraction for pipeline run status rows.