token-lang 1.0.0

# token-lang v1.0.0 — API freeze

**Stable.** v1.0.0 freezes the public surface under Semantic Versioning until 2.0.
There are no code changes from `0.2.0` — this release marks the contract: the token
type, the classification trait, and the re-exported coordinate and interning types
are now stable, and no breaking change will be made without a major version bump.

## What is token-lang?

The token type that sits between a lexer and a parser. A `Token<K>` is a *classified
span* — a language-specific kind `K` paired with the `Span` of source it covers — and
the `TokenKind` trait gives generic code the small set of questions a parser asks of
any token: skip trivia, detect the end, read an interned lexeme. The language supplies
`K`; token-lang owns the pairing and the seam, so a lexer and a parser depend on each
other only through this crate.

## The 1.0 surface

The whole public API, frozen as of this release:

- **`Token<K>`** — a kind paired with a `Span`. `new`, the `kind` / `span` /
  `into_kind` accessors, `map` (rewrite the kind, keep the span), and `as_ref`.
  `Copy` when `K` is; orders by span first, so a stream sorts into source order;
  `Display` as `kind @ start..end`. Converts to and from `span-lang`'s `Spanned<K>`
  and a `(K, Span)` tuple.
- **`TokenKind`** — the classification trait a language implements for its kind:
  `is_trivia`, `is_eof`, and `symbol` (the interned lexeme, as a `Symbol`), all
  default-implemented so an empty `impl` works. `Token` forwards all three when
  `K: TokenKind`.
- **Re-exports** — `Span`, `Spanned` (`span-lang`) and `Symbol` (`intern-lang`), so a
  consumer names only `token-lang`.

The design held from the start: a token is a kind plus a span, the kind is generic so
the crate serves every language, and the trait is the three universal questions and
nothing more. The `symbol` accessor — the one open question across the 0.x line —
stays on `TokenKind`, wiring `intern-lang` into the seam.

## Guarantees, now under SemVer

The invariants are part of the contract and are property-tested:

- **Lossless construction** — a token built from `(kind, span)` reads both back.
- **Spanned round-trip** — `Token` and `Spanned` hold the same two fields and convert
  both ways without loss.
- **Source order** — token ordering equals `(span, kind)` ordering, so sorting a
  stream sorts it by position.
- **`map` keeps the span** — only the kind changes.
- **Forwarders, not a second truth** — `Token::is_trivia` / `is_eof` / `symbol`
  return exactly what the kind returns.
- **Total and panic-free** — no constructor or accessor can fail, and a token adds no
  allocation beyond what `K` costs.

SemVer's additive rule still permits new defaulted trait methods, new `Token`
helpers, and new trait impls later — so the freeze locks the existing surface without
precluding growth.

## Breaking changes

**None.** v1.0.0 is identical in behaviour to v0.2.0.

## Verification

Run green on **Windows x86_64** against Rust stable and MSRV 1.85, default and all
features:

```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test
cargo test --all-features
cargo build --no-default-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo +1.85 test --all-features
cargo bench --no-run
cargo deny check
```

All green. Counts at this tag: 13 unit + 7 property + 7 integration tests and 13
doctests on default features; the 3-test `serde` suite adds under `--all-features`.
The `--no-default-features` build is clean as `no_std`.

## What's next

1.0 is the stability line. Future work is bug fixes, performance, documentation, and
additive API only — anything breaking waits for a 2.0.

## Installation

```toml
[dependencies]
token-lang = "1"

# optional: serde support for `Token`
token-lang = { version = "1", features = ["serde"] }
```

MSRV: Rust 1.85.

## Documentation

- [README](https://github.com/jamesgober/token-lang/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/token-lang/blob/main/docs/API.md)
- [CHANGELOG](https://github.com/jamesgober/token-lang/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.2.0...v1.0.0`](https://github.com/jamesgober/token-lang/compare/v0.2.0...v1.0.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/token-lang/blob/main/CHANGELOG.md#100---2026-06-28).