DTCS — Data Transformation Contract Standard

Vendor-neutral contracts for data transformation semantics — describe what a transformation means without locking you to Spark, SQL, Polars, or any single runtime.

This repository contains:

• SPEC.md — normative DTCS 1.0 draft (26 chapters)
• Reference tools — parse, validate, and analyze contracts in Rust and Python

What you can do today: validate YAML/JSON contracts, compare versions for compatibility, analyze evolution between revisions, and trace dataset lineage — all read-only, no execution engine required.

Quick start · User docs · Examples · Changelog · Roadmap

Install

Requirements: Python 3.9+ (PyPI package); Rust 1.75+ (cargo install or building from source).

pip install dtcs
cargo install dtcs

dtcs version
dtcs validate examples/customer_normalize.dtcs.yaml

Both packages install the dtcs CLI on PATH:

validate · inspect · diagnostics · compat · evolve · lineage · version

Develop from source (requires Rust + maturin): see CONTRIBUTING.md.

Quick start

# Validate a contract (exit 0 = valid)
dtcs validate examples/customer_normalize.dtcs.yaml

# Human-readable summary
dtcs inspect examples/customer_normalize.dtcs.yaml

# Compare two contract versions
dtcs compat examples/analysis/backward_old.yaml examples/analysis/backward_new.yaml

# Trace lineage impact
dtcs lineage examples/analysis/lineage_multi.yaml --impact customers

import dtcs

report = dtcs.parse_and_validate(
    open("examples/customer_normalize.dtcs.yaml", "rb").read()
)
assert dtcs.is_valid(report)

Read docs/user/getting-started.md for a full walkthrough. For normative definitions, see SPEC.md — start with Chapter 3 (COM) and Chapter 9 (Validation).

Pipeline

The reference implementation through Phase 0.3:

DTCS Document
        │
        ▼
Parser → Canonical Object Model
        │
        ├──────────────────────────────┐
        ▼                              ▼
Validator (0.1–0.2)              Analyzer (0.3)
        │                              │
        ▼                              ├─ compatibility::analyze
Diagnostics                            ├─ analyze_evolution
        │                              ├─ versioning::validate
        │                              └─ lineage::analyze
        │                              │
        │                              ▼
        │                         Analysis reports
        ▼
   (valid contracts only for analysis)

Phase 0.2 adds metadata validation, extended type system checks, expression typing, and I/O interface depth. Phase 0.3 adds compatibility classification, evolution analysis, versioning validation, and dataset-level lineage analysis.

Execution, backend compilation, and runtime behavior remain out of scope. See docs/implementation/non-goals.md.

Repository layout

Contributing

See CONTRIBUTING.md for editorial conventions, implementation guidelines, and the review process.

When implementation guidance conflicts with the specification, SPEC.md wins. See docs/implementation/spec-usage.md.

License

Licensed under the Apache License, Version 2.0. See LICENSE.