Module core 

Core pipeline orchestration primitives.

Modules

cancellation

error

exec_status

pipeline

pipeline_run_registry

progress

provenance

Provenance stamping for DataPoints emitted by pipeline tasks.

rate_limiter

Proactive request-rate throttling for pipeline tasks.

runtime

sentinels

Control-flow sentinel values that pipeline tasks return to steer the executor. Sentinels are ordinary Values (via the blanket impl<T> Value for T in task.rs), so no manual trait impl is needed.

task

task_context

thread_pool

Structs

AsyncRuntime

A wrapper around a Tokio Runtime.

CancellationHandle

Allows the owner of a task to request cancellation.

CancellationToken

Passed into a running task so it can observe cancellation requests.

DbPipelineWatcher

PipelineWatcher that writes pipeline_runs rows through a PipelineRunRepository. Does not broadcast RunEvents.

DefaultPipelineRunRegistry

Concrete in-memory PipelineRunRegistry backed by a PipelineRunRepository for durable persistence.

DroppedSentinel

Returned by a task to discard the current item: it is not forwarded to downstream tasks and does not appear in the pipeline output.

NoopExecStatusManager

No-op implementation used when incremental loading is disabled.

NoopWatcher

PassthroughSentinel

Returned by an enriching task to forward its input unchanged.

Pipeline

PipelineBuilder

A compile-time type-safe builder for Pipeline.

PipelineContext

Identity of the running pipeline and the data item being processed.

PipelineRunHandle

Handle to a pipeline run spawned in the background via execute_in_background.

PipelineRunInfo

Identity and metadata of a pipeline run, passed to PipelineWatcher event methods.

PipelineRunResult

The successful output of a pipeline run.

ProgressToken

A cheaply-cloneable progress token representing a portion of overall progress.

ProvenanceContext

What we know at the call site of stamp_tree.

RayonThreadPool

A CpuPool backed by a dedicated rayon::ThreadPool.

RegistryConfig

Configurable bounds for the in-memory registry.

RunEvent

One event in a run’s lifecycle emitted on the registry channel.

RunHandle

Per-run handle returned by register_*. Cheap to clone and share.

RunOutcome

The value returned by register_inline once the work future completes.

RunSpec

Builder-style metadata for a new run.

ScopedRunWatcher

PipelineWatcher proxy that forwards lifecycle events to a run’s slot.

SemaphoreLimiter

Admission-style concurrency limit: at most max_per_sec starts may be issued per second.

Tagged

A value wrapper that carries arbitrary string metadata alongside the inner value.

TaggedMeta

Lightweight metadata carrier that tasks can attach to any Arc<dyn Value> when they need to propagate node_set without wrapping in Tagged<T>.

TaskContext

Runtime dependencies and control tokens for a single pipeline task.

TaskContextBuilder

Fluent builder for TaskContext.

TaskInfo

A Task bundled with optional per-task configuration.

TokenBucketLimiter

Caps the number of starts allowed per refill window (token-bucket algorithm).

Enums

CoreError

ExecutionError

PipelineRunStatus

High-level status of a pipeline run.

PipelineStatus

RegistryError

Errors returned by registry operations.

RetryDelay

Delay strategy between retry attempts.

RetryPolicy

RunEventKind

Discriminant for a RunEvent.

RunPhase

Snapshot of a run’s high-level phase. Cheap to read; never blocks the producer.

Task

A single reusable unit of work in a cognee pipeline.

TaskCall

The pending (or already-resolved) output of Task::call.

TaskStatus

TypedTask

A typed pipeline task whose input and output types are tracked at the type level.

Traits

CpuPool

Dyn-compatible interface for a CPU-bound thread pool.

CpuPoolExt

Ergonomic extension for CpuPool that adds a generic spawn with a return value. Auto-implemented for every T: CpuPool.

ExecStatusManager

Per-data-item status tracking for incremental pipeline execution.

HasDataPoint

Read / write access to the embedded DataPoint of a typed container, plus a hook to recurse into nested DataPoint-bearing children.

PipelineRunRegistry

Runtime-agnostic registry for pipeline run lifecycle tracking.

PipelineRunRepository

Persistence abstraction for pipeline run status rows.

PipelineWatcher

Observer for pipeline and task lifecycle events.

RateLimiter

Admission throttle: acquire().await returns when the caller is permitted to start an external call. Object-safe; hold as Arc<dyn RateLimiter>.

Value

Type-erased value passed between pipeline tasks.

Functions

cancellation_pair

Creates a linked (CancellationHandle, CancellationToken) pair.

downcast_value

Attempt to downcast a Box<dyn Value> to a concrete type.

execute

Execute pipeline against a set of inputs.

execute_blocking

Run execute synchronously, blocking the calling thread until the pipeline completes.

execute_in_background

Spawn execute on the current Tokio runtime and return a PipelineRunHandle immediately.

extract_content_hash_from_value

Walk a type-erased Value looking for the first non-empty Data.content_hash (raw ingestion artefact) or DataPoint.source_content_hash. Mirrors Python’s _extract_content_hash.

extract_node_set

Try to extract a node_set metadata value from an Arc<dyn Value>.

extract_node_set_from_value

Walk a type-erased Value looking for the first non-empty source_node_set on an embedded DataPoint. Mirrors Python’s _extract_node_set.

is_dropped

Returns true if value is a DroppedSentinel.

is_passthrough

True if value is a PassthroughSentinel. See is_dropped for the &dyn Value (dereference-the-pointer) contract.

stamp_tree

Stamp a tree of HasDataPoint values in place.

stamp_tree_dyn

Type-erased dispatch for stamp_tree.

Type Aliases

AsyncBatchFn

Async batch task: slice of values in → one value out (via a future).

AsyncFn

Async task: one value in → one value out (via a future).

AsyncStreamBatchFn

Async batch task: slice of values in → async stream of values out.

AsyncStreamFn

Async task: one value in → async stream of values out.

DataIdFn

Function that extracts a stable, content-addressed identifier from a type-erased Value.

PipelineFuture

A boxed, Send pipeline future whose output is a generic Result.

SyncBatchFn

Sync batch task: slice of values in → one value out.

SyncFn

Sync task: one value in → one value out.

SyncIterBatchFn

Sync batch task: slice of values in → lazy iterator of values out.

SyncIterFn

Sync task: one value in → lazy iterator of values out.

TaskError

Boxed error returned by a failing task.

ValueIter

Boxed, type-erased iterator yielded by SyncIter tasks.

ValueStream

Boxed, type-erased async stream yielded by AsyncStream tasks.