julienne 0.1.0

Range-preserving Rust text chunkers for retrieval and embedding pipelines
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# Release And Quality Gates

The canonical local quality gate is:

```bash
prek run --all-files
```

It runs:

- `cargo fmt --all -- --check`
- required tool checks for `cargo-deny`, `cargo-machete`, and
  `cargo-semver-checks`
- `cargo check --locked --all-targets --all-features`
- feature matrix checks for default, no-default, each optional feature, and all
  features
- `cargo test --locked --all-targets --all-features`
- `cargo clippy --locked --all-targets --all-features -- -D warnings`
- `cargo deny check`
- `cargo machete`
- `RUSTDOCFLAGS="-D warnings" cargo doc --locked --all-features --no-deps`
- `cargo package --locked --allow-dirty`

Before publishing, also run:

```bash
prek run cargo-semver-checks --all-files --stage manual
cargo package
cargo publish --dry-run
```

For the first crates.io release, the semver check passes explicitly when no
published baseline exists. After the first release, the same gate compares
against the latest published baseline.

Benchmark commands:

```bash
cargo bench --bench splitters_bench
cargo bench --bench splitters_bench --features code -- splitters_code
```