creature_feature 0.2.0

Composable n-gram combinators that are ergonomic and bare-metal fast.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

# Changelog

All notable changes to `creature_feature` are documented here.
This project adheres to [Semantic Versioning](https://semver.org/).

## 0.2.0

This release fixes several correctness bugs, repairs the `serde1` feature, and
replaces the internal macro-based input dispatch with a single public trait.
Some changes are behavioral; the version bump to 0.2.0 reflects that.

### Added
- **`traits::AsTokens`** — a public trait (element type as the associated
  `Token`) that lets any type be viewed as a slice of tokens. Implementing it
  for your own type makes that type featurizable by every featurizer and
  combinator at once. This replaces the internal `impl_ftrzs!` /
  `impl_ftrzs_2!` macros: each featurizer now has a single blanket
  `IterFtzr<&D> where D: AsTokens` impl instead of one hand-written forward per
  container type (`&str`, `&String`, `&[T]`, `&Vec<T>`, `&[T; N]`).
- `ForEach` now implements `IterFtzr` (previously `unimplemented!()` — a
  runtime panic on the iterator path). The impl requires `F: IterFtzr + Clone`.
- Integration tests (`tests/regressions.rs`) and a criterion benchmark
  (`benches/featurization.rs`) reproducing the output-type performance spread.
- `rust-version = "1.65"` (verified MSRV).

### Fixed
- **`serde1` feature now compiles.** The borrowing iterator structs
  (`NGramIter`, `SliceGramIter`, `GapGramIter`, `BookEndsIter`, `ForEachIter`)
  no longer derive `Serialize`/`Deserialize`, which previously failed to build.
- **`From<GapPair<T, V>> for HashedAs<_>`** hashed the first component twice and
  never hashed the second, so pairs differing only in the second component
  collided. (Latent on the normal `featurize` path, where a blanket impl won.)
- **`gap_gram` no longer panics** on input shorter than `a.chunk_size() + gap`;
  it yields no features instead.
- **`bookends` no longer panics** on input shorter than a window; each window
  clamps to the available input.
- **`featurizers!`** now expands using `$crate` paths instead of a hardcoded
  `creature_feature::` path, so it works when the crate is renamed or
  re-exported.
- Numerous doctests that never compiled now build and run.

### Changed (behavior)
- The `Accumulates<&[&str]> for String` join separator changed from two spaces
  (`"  "`) to a single space (`" "`).
- Short-input handling changed from panic to clamp / empty output (see Fixed).
- `ForEach`'s `IterFtzr` impl adds a `Clone` bound on the inner featurizer.

### Changed (internal / hygiene)
- Renamed the fragile implicit `serde` cargo feature to `serde1` using
  `dep:serde`, so the `#[cfg(feature = ...)]` gates key off the real feature.
- Edition 2018 → 2021.
- Removed the dead shipped binary (`main.rs`, `bench.rs`), the unused
  `tokengroup`, `skip_schema`, and `Collisions` code, and the `internal` macro
  module. Clippy-clean; builds with zero warnings.

### Not included (possible future work)
- A `--safe`-style GAT/RPITIT rewrite of the `IterFtzr` traits was evaluated
  and rejected: it would reduce downstream expressiveness (a parameterized
  input type and nameable associated iterators are load-bearing) without a
  compensating gain.
- Collision inspection (which hashes collided) was removed rather than exposed,
  since the prior sketch's semantics couldn't be pinned down; it remains
  available as a deliberate future design.