1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
//! Transport-agnostic [OpenLineage](https://openlineage.io) event model and emit client.
//!
//! This crate owns the emission side of OpenLineage: the [`RunEvent`] model and
//! its typed facets, a pluggable [`Transport`] sink, and a non-blocking
//! [`OpenLineageClient`] that hands events to a background drain task. It has
//! **no DataFusion (or any engine) dependency** — engine integrations such as
//! [`datafusion-openlineage`](https://docs.rs/datafusion-openlineage) build on
//! top of it, and so can any other emitter.
//!
//! # The transport seam
//!
//! How events are published is deliberately unspecified. The [`Transport`] trait
//! is the only seam: an implementation might POST to a spec-compliant OpenLineage
//! REST endpoint, publish to a Kafka topic, or do anything else. The crate ships
//! [`NoopTransport`], [`ConsoleTransport`], and — behind the `http` feature —
//! [`CloudClientTransport`], which posts to an (optionally authenticated) HTTP
//! endpoint via `olai-http`. To target something else, implement [`Transport`].
//!
//! # Non-blocking emission
//!
//! [`OpenLineageClient::emit`] never blocks and never applies back-pressure: it
//! hands the event to a bounded channel drained by a background task. If the
//! queue is full the event is dropped with a warning — lineage must never slow or
//! break the host workload. Call [`OpenLineageClient::shutdown`] before exit to
//! drain queued events and flush the transport.
//!
//! # Example
//!
//! ```no_run
//! use std::sync::Arc;
//! use openlineage_client::{ConsoleTransport, OpenLineageClient};
//!
//! # async fn run(event: openlineage_client::RunEvent) {
//! let client = OpenLineageClient::new(Arc::new(ConsoleTransport));
//! client.emit(event); // non-blocking
//! client.shutdown().await; // drain + flush before exit
//! # }
//! ```
//!
//! # Environment
//!
//! [`OpenLineageClient::from_env`] and [`OpenLineageConfig::from_env`] read the
//! standard OpenLineage environment variables, so an integration can wire itself
//! up from the environment the rest of the ecosystem already uses:
//!
//! | Variable | Read by | Meaning |
//! |---|---|---|
//! | `OPENLINEAGE_URL` | [`OpenLineageClient::from_env`] | Base URL of the endpoint (unset → no-op client) |
//! | `OPENLINEAGE_ENDPOINT` | [`OpenLineageClient::from_env`] | Path appended to the URL (default `/api/v1/lineage`) |
//! | `OPENLINEAGE_API_KEY` | [`OpenLineageClient::from_env`] | Bearer token, if set |
//! | `OPENLINEAGE_NAMESPACE` | [`OpenLineageConfig::from_env`] | Default job namespace |
//! | `OPENLINEAGE_TIMEOUT_MS` | [`OpenLineageConfig::from_env`] | Per-request transport timeout |
//! | `OPENLINEAGE_PARENT_ID` / `OPENLINEAGE_PARENT_*` | [`LineageContext::from_env`] | Parent run facet |
pub use ;
pub use ;
pub use LineageContext;
pub use ;
pub use DatasetName;
pub use ;
pub use CloudClientTransport;