dtcs 0.11.0

Reference implementation of the Data Transformation Contract Standard (DTCS)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

# Python API reference

Package: [`dtcs` on PyPI](https://pypi.org/project/dtcs/). Source: [`python/dtcs/__init__.py`](https://github.com/eddiethedean/dtcs/blob/main/python/dtcs/__init__.py).

CLI JSON shapes in [json-output.md](../user/json-output.md) use the same camelCase keys as Python dicts.

## Install

```bash
pip install 'dtcs==0.11.0'
```

```python
import dtcs
print(dtcs.__version__)   # package version, e.g. "0.11.0"
print(dtcs.SPEC_VERSION)  # "1.0.0-draft"
```

## Errors and validity

Most APIs return **dicts** that may include a `diagnostics` list (and sometimes other fields). They do not raise for validation failures.

| Helper | Behavior |
|--------|----------|
| `is_valid(report)` | `True` when no diagnostic has error severity. Missing `severity` is treated as an error. |

```python
report = dtcs.parse_and_validate(yaml_bytes)
if not dtcs.is_valid(report):
    for d in report["diagnostics"]:
        print(d["severity"], d["id"], d["message"])
```

`analyze` returns nested reports — check each:

```python
result = dtcs.analyze(contract)
assert dtcs.is_valid(result["validation"])
assert dtcs.is_valid(result["analysis"])
```

## Parse and validate

| Function | Arguments | Returns |
|----------|-----------|---------|
| `parse(content, format="yaml")` | `bytes`/`str`, format `yaml`\|`json` | `{contract, report: {diagnostics}}` |
| `parse_file(path)` | filesystem path | `{contract, report: {diagnostics}}` |
| `validate(contract, registry_path=None)` | COM dict | `{diagnostics}` |
| `validate_with_registry(contract, registry_path)` | COM dict + vendor registry path | `{diagnostics}` |
| `parse_and_validate(content, format="yaml")` | document bytes | `{diagnostics}` |
| `validate_result(result, registry_path=None)` | parse result dict | `{diagnostics}` |
| `is_valid(report)` | any report-like dict with `diagnostics` | `bool` |

### Example

```python
import urllib.request
import dtcs

url = "https://raw.githubusercontent.com/eddiethedean/dtcs/main/examples/minimal.dtcs.yaml"
content = urllib.request.urlopen(url).read()
report = dtcs.parse_and_validate(content)
assert dtcs.is_valid(report)
contract = dtcs.parse(content)["contract"]
assert contract["dtcsVersion"] == "1.0.0"
```

## Analysis and planning

| Function | Returns (typical) |
|----------|-------------------|
| `analyze(contract, registry_path=None)` | `{validation, analysis}` (each has `diagnostics`) |
| `plan_lower(contract, registry_path=None)` | `{plan, diagnostics}` |
| `plan_validate(plan, registry_path=None)` | `{diagnostics}` |
| `plan_optimize(plan, registry_path=None, *, validate=True)` | `{plan, diagnostics}` |
| `plan_equivalent(before, after)` | `bool` |
| `plan_topological_order(contract, plan)` | ordered node id list |
| `metadata_validate(contract)` / `version_validate(contract)` | focused `{diagnostics}` reports |

## Compatibility and lineage

| Function | Notes |
|----------|-------|
| `compat_analyze(source, target, scope=None)` | classification of target vs source |
| `evolve_analyze(older, newer)` | same-identity revision analysis |
| `lineage_analyze(contract, impact=None, dependency=None)` | dataset lineage |
| `inspect(contract)` | human-readable **string** summary (not a dict) |

## Registry

| Function | Notes |
|----------|-------|
| `registry_list(registry_path=None)` | entry list |
| `registry_resolve(id, registry_path=None)` | single entry or `None` |
| `registry_load(path)` | load document |

## Execution pipeline

| Function | Notes |
|----------|-------|
| `capability_reference_profile()` | embedded `dtcs:reference` |
| `capability_match(plan, profile=None)` | `{supported, diagnostics, …}` |
| `compile_plan(plan)` | `{plan, diagnostics}` execution plan |
| `execution_validate(plan)` | `{diagnostics}` |
| `runtime_execute(execution_plan, inputs)` | `{outputs, diagnostics}` |

`inputs` map interface id → list of row dicts. Cell values may be JSON `null`, or `{"$dtcs":"missing"}` / `{"$dtcs":"invalid"}`. **Do not coerce** missing/invalid to `None`.

```python
import json, dtcs

parsed = dtcs.parse_file("examples/customer_pipeline.dtcs.yaml")
contract = parsed["contract"]
plan = dtcs.plan_lower(contract)["plan"]
compiled = dtcs.compile_plan(plan)["plan"]
inputs = json.loads(open("tests/fixtures/runtime/customer_pipeline_input.json").read())
result = dtcs.runtime_execute(compiled, inputs)
assert dtcs.is_valid(result)
assert len(result["outputs"]["customer_clean"]) == 2
```

## Conformance

| Function | Notes |
|----------|-------|
| `conformance_declare(profile=None)` | Ch 23 capability declaration |
| `conformance_run(profile=None)` | offline report (`None`/`"all"` runs every profile). Fixtures are embedded in the wheel; optional `DTCS_FIXTURES` overrides the on-disk search path. |

## Typing

The wheel does not yet ship `py.typed` / `.pyi` stubs. Treat return values as JSON-compatible dicts/lists (except `inspect`, which returns `str`).

## See also

- [Rust API](rust.md) · [WASM](wasm.md) · [Node](node.md)
- [migration-0.11.md](../user/migration-0.11.md)