pipe-audit-core 0.1.3

Universal data validation and compliance audit engine
Documentation

Pipe Audit

Universal data validation and compliance audit engine for structured data pipelines.

pipe audit helps data engineers, developers, and technical managers enforce reproducible, compliance‑grade validation without heavy setup.
Use it as a Rust library (pipe-audit-core), a standalone CLI (pipe-audit → installs the pipa binary), or both.

✨ Features

  • Cloud storage integration: Azure Blob, Google Cloud Storage, Amazon S3
  • Compliance‑grade audit logging: tamper‑resistant, JSON‑structured logs for every action
  • Simple TOML configuration: define contracts, profiles, and validation rules in plain TOML
  • Flexible usage: embed as a Rust library or run as a CLI tool

🚀 Installation

CLI

Install the CLI globally (requires a recent Rust toolchain, edition 2024):

cargo install pipe-audit

This provides the pipa binary:

pipa init
pipa run --all

Library

If you want to embed the engine in your own Rust project, add the core library:

[dependencies]
pipe-audit-core = "0.1"

🛠️ Quick Start (CLI)

Validate a contract:

pipa contract validate contracts/example.toml

Check system health:

pipa health

List available profiles:

pipa profile list

Verify logs:

pipa logs verify ./examples/logs/test.log

👉 For full guides, examples, and contributor onboarding, see the 📚 Documentation.

📦 Library Usage

use pipe_audit_core::engine::run_contract_validation;

fn main() {
    run_contract_validation("contracts/example.toml").unwrap();
}

📄 License

  • pipe-audit-core (the library) is licensed under the Mozilla Public License 2.0 (MPL‑2.0).
  • pipe-audit (the CLI, installs as pipa) is licensed under the GNU General Public License v3.0 or later (GPL‑3.0‑or‑later).

See the LICENSES/ directory for full texts.

📚 Documentation & Community

🔮 Roadmap

  • Richer docs with end‑to‑end examples
  • Expanded connectors (databases, streaming sources)
  • Containerized API service built on pipe-audit-core