# intern-lang v0.2.0 — Core interner & symbol
**The first release with domain logic.** v0.2.0 ships the two types the whole
crate exists for: `Interner`, which maps a string to a small `Copy` handle and
stores the bytes once, and `Symbol`, the four-byte handle itself. The hard part
of the design — keeping a symbol valid after the backing store grows — is built
and property-tested here, not deferred. No breaking changes; this is the first
public surface, added on top of the v0.1.0 scaffold.
## What is intern-lang?
A string interner. It maps each distinct string to a small, copyable `Symbol`,
stores the bytes once in a contiguous backing store, and hands back integer
handles so that comparing two names is an integer comparison rather than a byte
walk. It is the deduplication layer beneath the lexer and the symbol table:
intern an identifier once, then pass cheap `Copy` handles everywhere a string name
would otherwise travel. It owns interning and nothing else.
## What's new in 0.2.0
### `Symbol` — a four-byte handle
`Symbol` is a newtype over a `NonZeroU32`. It is `Copy`, four bytes, and its
equality, ordering, and hashing are integer operations that never touch the
underlying string. The `NonZeroU32` representation gives `Option<Symbol>` the same
four-byte size through niche optimisation, which matters when a symbol sits in a
large side table.
```rust
use intern_lang::Interner;
let mut interner = Interner::new();
let first = interner.intern("alpha");
let second = interner.intern("beta");
assert_eq!(first.as_u32(), 1); // ids are assigned in interning order
assert_eq!(second.as_u32(), 2);
assert_eq!(interner.intern("alpha"), first); // re-interning never mints a new id
```
`as_u32` exposes the raw 1-based id for diagnostics, stable ordering, and external
side tables. A symbol is only meaningful with the interner that issued it.
### `Interner` — contiguous store, deduplicating index
`Interner::intern(&str) -> Symbol` deduplicates: the same string always returns
the same symbol, and a repeat hit allocates and copies nothing. `resolve(Symbol)
-> Option<&str>` borrows the bytes back out of the store. `get(&str) ->
Option<Symbol>` is the read-only lookup that never interns. `with_capacity`
pre-sizes the index when the rough identifier count is known.
```rust
use intern_lang::Interner;
let mut interner = Interner::new();
let a = interner.intern("while");
let b = interner.intern("while");
let c = interner.intern("until");
assert_eq!(a, b); // deduplicated
assert_ne!(a, c); // distinct strings, distinct symbols
assert_eq!(interner.resolve(a), Some("while"));
assert_eq!(interner.get("until"), Some(c));
assert_eq!(interner.get("never"), None);
```
Under the surface, bytes live once in a single contiguous buffer, appended end to
end. A symbol indexes a side table of `(start, len)` spans, so a symbol stays four
bytes no matter how long its string is. Deduplication runs through an
open-addressing hash index that stores symbol ids — not a second copy of the
bytes — and caches a 32-bit fingerprint per slot so a probe rejects a non-match
without touching the buffer.
### Symbol stability across growth
The invariant everything above depends on: a symbol issued early must keep
resolving to the same string after the store grows. The buffer only ever appends
and the span table only ever grows, and `resolve` recomputes the slice from the
current buffer on each call rather than holding a borrowed pointer — so a
reallocation can never dangle a previously issued symbol.
```rust
use intern_lang::Interner;
let mut interner = Interner::new();
let early = interner.intern("first");
// Force the backing store and the dedup index to reallocate many times.
for i in 0..10_000 {
let _ = interner.intern(&format!("filler_{i}"));
}
assert_eq!(interner.resolve(early), Some("first")); // still valid, still correct
```
This is proven, not assumed: a `proptest` property interns a batch, remembers the
symbols, floods the interner with thousands more strings to force growth, and
asserts every early symbol still resolves to its original string.
### Property tests against a reference interner
`tests/properties.rs` cross-checks the real interner against a
`HashMap<String, Symbol>` reference over a wide input space:
- **Dedup, distinctness, and round-trip** in one pass — the same string always
returns the same symbol, distinct strings never collide, and
`resolve(intern(s))` is always `Some(s)`.
- **`get` agrees with `intern`** without ever creating a symbol.
- **Symbols stay valid after growth.**
Integration tests in `tests/interner.rs` cover the edge cases the directive calls
for: the empty string, arbitrary Unicode, 100 KB strings, foreign and
out-of-range symbols, fifty thousand distinct interns, and `Symbol` used as a
`HashMap` key.
### Benchmarks
`benches/bench.rs` establishes a Criterion baseline for the intern (hit and miss),
resolve, and lookup-miss paths, so later optimisation is measured against a
recorded baseline rather than a guess. Local means on the development machine
(Windows x86_64, Rust stable):
- `resolve` (id → `&str`): ~1.1 ns
- `intern`, repeat hit (allocation-free): ~0.23 µs per 1024-string batch element
- `intern`, new string (amortised growth): ~0.62 µs
## Breaking changes
**None.** v0.2.0 adds the first public surface on top of the v0.1.0 scaffold.
## Verification
Run on Windows x86_64, Rust stable 1.95.x and MSRV 1.85; the same commands run on
Linux (WSL2 Ubuntu) and across the CI matrix (Linux / macOS / Windows × stable /
1.85):
```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test
cargo test --all-features
cargo build --no-default-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo +1.85 test --all-features
cargo audit
cargo deny check
```
All green. Counts at this tag:
- Default features: 19 unit + 9 integration + 3 property + 11 doctests.
- `--all-features`: same, plus the `serde` feature builds clean (no `Symbol`
serialisation yet — that lands in v0.4.0).
## What's next
- **0.3.0 — Concurrent interner.** A thread-safe interner many front-end threads
can intern into at once, behind the same trait seam as `Interner`, with
contention behaviour benchmarked rather than assumed.
## Installation
```toml
[dependencies]
intern-lang = "0.2"
```
MSRV: Rust 1.85.
## Documentation
- [README](https://github.com/jamesgober/intern-lang/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/intern-lang/blob/main/docs/API.md)
- [CHANGELOG](https://github.com/jamesgober/intern-lang/blob/main/CHANGELOG.md)
---
**Full diff:** [`v0.1.0...v0.2.0`](https://github.com/jamesgober/intern-lang/compare/v0.1.0...v0.2.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/intern-lang/blob/main/CHANGELOG.md#020---2026-06-19).