source-lang 0.3.0

Source file and buffer management with multi-file source maps.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179

# source-lang v0.3.0 — File loading & line integration

**From a map to a loader.** v0.2.0 built the core — the `SourceMap`, stable ids, and
the `O(log files)` resolver. v0.3.0 makes it loadable and diagnostic-ready: sources
now come from disk and from raw byte buffers through one set of checks, every bad
input is a defined error instead of a panic, and a global position resolves to file
plus 1-based line and column in a single call. This is the layer a front-end
actually reads through — load the files, then point at `file:line:col`.

## What is source-lang?

The multi-file coordinate layer of a compiler front-end. It loads files and
in-memory buffers, gives each a stable `SourceId`, lays them out in one global
position space, and maps any global position back to the source and local offset it
came from. It is the layer above `span-lang`: a `Span` says *where in a buffer* an
error is, source-lang says *which buffer*, so a diagnostic can name the file as well
as the position. It owns source storage and coordinate mapping only.

## What's new in 0.3.0

### Loading behind one seam

Sources reach the map three ways, all funnelling through the same insertion path so
that a buffer from the network and a file from disk are checked, sized, and stored
identically:

- `add` — text already in hand (a `String` or `&str`), as in 0.2.0.
- `add_bytes` — raw bytes, validated as UTF-8 before they are stored.
- `add_file` — a path read from disk (with the default `std` feature).

```rust
use source_lang::SourceMap;

let mut map = SourceMap::new();

// Raw bytes are validated as UTF-8, then stored once.
let cfg = map.add_bytes("config.toml", b"name = \"demo\"")?;
assert_eq!(map.source(cfg).unwrap().text(), "name = \"demo\"");
# Ok::<(), source_lang::SourceMapError>(())
```

`add_file` reads a path the same way, naming the source by its path:

```rust,ignore
let main = map.add_file("src/main.rs")?;
assert_eq!(map.source(main).unwrap().name(), "src/main.rs");
```

### Defined errors for every bad input

Bad input never panics; it is a value the caller can match on. `SourceMapError`
gains three variants alongside `SpaceExhausted`, each naming the source it concerns:

- `Oversize` — the source exceeds the per-source size ceiling.
- `NotUtf8` — the bytes are not valid UTF-8.
- `Io` — the file could not be read (missing path, a directory, permission denied);
  carries the `std::io::ErrorKind`, under the `std` feature.

```rust
use source_lang::{SourceMap, SourceMapError};

let mut map = SourceMap::new();
let err = map.add_bytes("blob.bin", &[0xff, 0xfe]).unwrap_err();
assert!(matches!(err, SourceMapError::NotUtf8 { .. }));
# let _ = err;
```

### A configurable size ceiling

`set_max_source_len` bounds how much a single untrusted source may load. For a file
the limit is checked against the path's metadata *before* a byte is read, so an
oversize file is rejected without ever being pulled into memory — a hostile input is
turned away at the boundary, not after the fact.

```rust
use source_lang::{SourceMap, SourceMapError};

let mut map = SourceMap::new();
map.set_max_source_len(8);
let err = map.add("big", "123456789").unwrap_err(); // 9 bytes
assert!(matches!(err, SourceMapError::Oversize { len: 9, .. }));
# let _ = err;
```

### `line_col` — file, line, and column in one step

`line_col` composes `locate` with `span-lang`'s `LineIndex`: a global position
resolves to its source and a 1-based `LineCol`, with the column counted in Unicode
scalar values rather than bytes. It is the call a diagnostic renderer makes to print
`file:line:col`.

```rust
use source_lang::{BytePos, LineCol, SourceMap};

let mut map = SourceMap::new();
map.add("a.rs", "fn a() {}")?;                       // 0..9
let b = map.add("b.rs", "let x = 1;\nlet y = 2;")?;  // 9..30

let (id, lc) = map.line_col(BytePos::new(20)).expect("in range");
assert_eq!(id, b);
assert_eq!(lc, LineCol::new(2, 1)); // second line of b.rs
# Ok::<(), source_lang::SourceMapError>(())
```

For resolving many positions in one source, `SourceFile::line_index` returns a
reusable index, so the `O(source len)` scan happens once instead of per call.
`LineCol` and `LineIndex` are re-exported from the crate root.

### Tests and benchmarks

- `tests/loading.rs` exercises the disk boundary on a real filesystem: a written
  file round-trips, and a missing path, a directory, non-UTF-8 contents, and a file
  over the size ceiling are each the expected defined error.
- A property test holds `line_col` to per-file `LineIndex` resolution across a wide
  `proptest` input space, including multi-byte characters and newlines.
- A `criterion` benchmark for `line_col` joins the existing one for `locate`.

## Breaking changes

- **`SourceMapError` is no longer `Copy`.** Its file-loading variants carry the
  offending source's name (a `Box<str>`) so a failure is actionable when logged far
  from its origin, which makes the type `Clone` but not `Copy`. Code that relied on
  implicit copies needs an explicit `.clone()`; matching is unaffected, and the enum
  remains `#[non_exhaustive]`.

## Verification

Run on Windows x86_64 (Rust stable and 1.85 MSRV); the same commands run on Linux
(WSL2 Ubuntu) and across the CI matrix on Linux, macOS, and Windows:

```bash
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test
cargo test --all-features
cargo build --no-default-features
cargo +1.85 build --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo audit
cargo deny check
```

All green. Counts at this tag:

- Default features: 27 unit + 6 file-loading + 5 property + 23 doctests.
- `--all-features`: identical (serde adds no runtime surface yet).

Benchmarks (1024 probes per iteration, 256-byte sources, on the reference machine):
`locate` runs about 4 ns per lookup at 16 sources and about 40 ns at 65,536 — a 9×
rise across a 4096× increase in file count, the shape of `O(log files)`. `line_col`
runs a few hundred nanoseconds per lookup, roughly flat across the same range,
because its cost is the per-source line-index build rather than the lookup;
`SourceFile::line_index` amortises that build when resolving many positions.

## What's next

- **0.4.0 — serde, feature freeze.** Optional `serde` for the source-map metadata
  (ids and ranges) and a declared frozen public surface.

## Installation

```toml
[dependencies]
source-lang = "0.3"
```

MSRV: Rust 1.85.

## Documentation

- [README](https://github.com/jamesgober/source-lang/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/source-lang/blob/main/docs/API.md)
- [CHANGELOG](https://github.com/jamesgober/source-lang/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.2.0...v0.3.0`](https://github.com/jamesgober/source-lang/compare/v0.2.0...v0.3.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/source-lang/blob/main/CHANGELOG.md#030---2026-06-20).