Skip to main content

soml/
lib.rs

1/*!
2SOML, a config format for humans: a strict reader, serde support, canonical form, and a lossless syntax tree.
3
4SOML keeps JSON's tree of objects and arrays and its strictness, and adds comments, bare keys, literal strings, block strings, dates, instants, durations, and line breaks as separators, with optional commas. A value's type comes from its syntax, never from its content. See the [specification](https://github.com/soml-lang/soml), v0.1.
5
6```rust
7use serde::Deserialize;
8
9#[derive(Deserialize)]
10#[serde(rename_all = "kebab-case")]
11struct Config {
12	name: String,
13	replicas: u32,
14	timeout: std::time::Duration,
15	deployed_at: soml::Instant,
16	launch_date: soml::Date,
17	labels: Vec<String>,
18}
19
20let config: Config = soml::from_str("
21## The edge service.
22name: 'api-gateway'
23replicas: 3
24timeout: 1m30s
25deployed-at: 2026-09-19T14:00:00Z
26launch-date: 2027-09-19
27labels: ['prod', 'eu-west']
28")?;
29
30assert_eq!(config.replicas, 3);
31assert_eq!(config.timeout.as_secs(), 90);
32# Ok::<(), soml::Error>(())
33```
34
35# Reading
36
37[`from_str`] and [`from_slice`] read a document into any type that implements `Deserialize`, and [`Value`] holds a document of unknown shape. Every rule in the spec is checked: duplicate keys, the int64 range, date, instant, and duration ranges, raw control characters, and more. An error has a line and a column, also when the document is valid but does not fit the type:
38
39```rust
40#[derive(serde::Deserialize, Debug)]
41struct Server {
42	port: u16,
43}
44
45let error = soml::from_str::<Server>("port: 70000").unwrap_err();
46assert_eq!(error.to_string(), "invalid value: integer `70000`, expected u16 at line 1, column 7");
47```
48
49# Writing
50
51[`to_string`] writes a document for people to read: one member or item per line, tab indentation, and no commas, because a line break separates members and items. Members keep the order the value gives them, so a struct is written in the order of its fields.
52
53[`to_string_canonical`] writes canonical form, the one text the spec defines for a value. It is the same, except that members are sorted by key, so the same value always gives the same bytes. Use it to hash, sign, or compare documents.
54
55```rust
56#[derive(serde::Serialize)]
57struct Server {
58	port: u16,
59	hosts: Vec<&'static str>,
60}
61
62let server = Server { port: 8080, hosts: vec!["a", "b"] };
63assert_eq!(soml::to_string(&server)?, "port: 8080\nhosts: [\n\t'a'\n\t'b'\n]\n");
64assert_eq!(soml::to_string_canonical(&server)?, "hosts: [\n\t'a'\n\t'b'\n]\nport: 8080\n");
65# Ok::<(), soml::Error>(())
66```
67
68A map is written in the order it iterates in. A `HashMap` iterates in a different order on each run, so use [`to_string_canonical`], a `BTreeMap`, or an `IndexMap` when the order must not change.
69
70# Types
71
72| Rust | SOML |
73|---|---|
74| `bool` | bool |
75| `i8` to `i128`, `u8` to `u128` | int, which is 64-bit, so a larger value cannot be written |
76| `f32`, `f64` | float. An int reads into a float only when it converts exactly, and a float never reads into an int. |
77| `String`, `&str`, `char` | string |
78| `Option<T>` | `null` or the value |
79| `()`, a unit struct | `null` |
80| `Vec<T>`, a tuple | array |
81| a struct, `HashMap`, `BTreeMap` | object. A map key can also be a char, an integer, a bool, or a unit enum variant. |
82| an enum | a string for a unit variant, and a one-member object for the others |
83| [`Date`] | Gregorian date, without a time or timezone |
84| [`Instant`] | instant |
85| [`Duration`], `std::time::Duration` | duration. A negative duration does not fit a `std::time::Duration`. serde gives a `std::time::Duration` no type of its own, so it is recognized by its shape: a struct named `Duration` with the fields `secs` and `nanos`. A struct of your own with that shape is written as a duration too, and it reads back. |
86| [`Value`] | any value |
87
88Fields of jiff, chrono, and humantime types, such as `jiff::Timestamp`, `jiff::SignedDuration`, and `chrono::DateTime<Utc>`, read instants and durations too, because they ask for text, and an instant or duration gives its canonical text. humantime reads only instants from 1970 on. The cost of this is that a `String` field also accepts an instant or duration. `chrono::TimeDelta` does not ask for text, so it needs `soml::chrono::time_delta` to read a duration too. Those types write themselves as strings, so use [`Instant`] and [`Duration`] to write native instants and durations. Native dates require [`Date`] or the explicit `soml::chrono::naive_date` and `soml::jiff::date` adapters, and do not read into a `String` field.
89
90`#[serde(flatten)]`, untagged enums, and internally tagged enums read values through serde's buffer, which has no date, instant, or duration type and no positions. There, a date, instant, or duration is its text: a [`Value`] holds it as a string, [`Date`], [`Instant`], and [`Duration`] still read it, and a `std::time::Duration` cannot be read. An int reads into a float even when the float does not hold it exactly. An `f32` is read through an `f64`, as in `serde_json`, so 2 of the 4.3 billion `f32` values read back one step away. A [`Spanned`] value has no span.
91
92# Syntax tree
93
94[`Document`] is a lossless syntax tree, for tools that read and change documents. An unchanged document prints the exact text it was parsed from, and a changed one keeps the comments and the author's spelling outside the change. Every parsed node has its position in the text.
95
96```rust
97let mut document: soml::Document = "port: 0x1F90 # The default.\n".parse()?;
98document.set(["port"], 8080)?;
99assert_eq!(document.to_string(), "port: 8080 # The default.\n");
100# Ok::<(), soml::Error>(())
101```
102
103See [`tree`] for the nodes, and for how whitespace and comments are kept.
104
105[`format()`] and [`Document::format`] are the spec's formatter: they change layout and nothing else.
106
107# Positions and errors
108
109[`Spanned`] reads any value with the byte range it came from, so a check your app makes after reading can point at the right place, with [`Error::with_position`]. Every [`Error`] has an [`ErrorKind`], and [`Error::code_frame`] shows where it is, for a terminal.
110
111# More
112
113- [`soml!`] builds a [`Value`] with JSON-like syntax.
114- [`from_reader`], [`to_writer`], and [`to_writer_canonical`] read from an `io::Read` and write to an `io::Write`.
115- The `jiff` and `chrono` features add conversions, and modules for `#[serde(with)]` that write their types as native dates, instants, and durations: `soml::jiff` and `soml::chrono`.
116
117# Format
118
119- File extension: `.soml`
120- Encoding: UTF-8 without a byte order mark, with LF line endings
121- Media type: `application/soml`
122- Uniform type identifier: `com.sindresorhus.soml`
123
124# Limits
125
126- Nesting is limited to 100 levels, as the spec requires. Every object and array counts, including the document's own.
127- Keys are compared byte for byte, so two keys that differ only in Unicode normalization are different keys, as the spec requires.
128*/
129
130#![forbid(unsafe_code)]
131#![allow(clippy::tabs_in_doc_comments)]
132#![warn(missing_docs)]
133#![warn(clippy::missing_errors_doc)]
134#![warn(clippy::missing_panics_doc)]
135#![warn(clippy::doc_markdown)]
136
137/**
138A module for `#[serde(with = "…")]` that writes a type of another crate as a native SOML date, instant, or duration, and reads it only from one, through a conversion to this crate's type. It has an `option` module inside for an `Option` of the type.
139*/
140#[cfg(any(feature = "jiff", feature = "chrono"))]
141macro_rules! serde_with_module {
142	($(#[$attribute:meta])* $name:ident, $type:ty, $soml:ty) => {
143		$(#[$attribute])*
144		pub mod $name {
145			use serde_core::de::{Deserialize, Deserializer, Error as _};
146			use serde_core::ser::{Error as _, Serialize, Serializer};
147
148			/**
149			Writes the value as a native SOML value.
150
151			# Errors
152
153			Returns an error when the value is outside the range of a SOML value.
154			*/
155			pub fn serialize<S: Serializer>(value: &$type, serializer: S) -> Result<S::Ok, S::Error> {
156				<$soml>::try_from(*value).map_err(S::Error::custom)?.serialize(serializer)
157			}
158
159			/**
160			Reads the value from a native SOML value.
161
162			# Errors
163
164			Returns an error when the document has another type of value there, or a value outside the range of the type.
165			*/
166			pub fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result<$type, D::Error> {
167				<$type>::try_from(<$soml>::deserialize(deserializer)?).map_err(D::Error::custom)
168			}
169
170			/**
171			The same, for an `Option`, which is `null` when it is `None`. With `#[serde(default)]`, a missing member reads as `None` too.
172			*/
173			pub mod option {
174				use super::*;
175
176				/**
177				Writes the value as a native SOML value, or `null`.
178
179				# Errors
180
181				Returns an error when the value is outside the range of a SOML value.
182				*/
183				pub fn serialize<S: Serializer>(value: &Option<$type>, serializer: S) -> Result<S::Ok, S::Error> {
184					value.map(<$soml>::try_from).transpose().map_err(S::Error::custom)?.serialize(serializer)
185				}
186
187				/**
188				Reads the value from a native SOML value, or `null`.
189
190				# Errors
191
192				Returns an error when the document has another type of value there, or a value outside the range of the type.
193				*/
194				pub fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Option<$type>, D::Error> {
195					Option::<$soml>::deserialize(deserializer)?.map(<$type>::try_from).transpose().map_err(D::Error::custom)
196				}
197			}
198		}
199	};
200}
201
202/**
203Native temporal scalars share their token wrapper and canonical-text serde representation.
204*/
205macro_rules! serde_temporal_scalar {
206	($type:ty, $expected:literal) => {
207		impl serde_core::ser::Serialize for $type {
208			fn serialize<S: serde_core::ser::Serializer>(
209				&self,
210				serializer: S,
211			) -> Result<S::Ok, S::Error> {
212				serializer.serialize_newtype_struct(TOKEN, &crate::value::CanonicalText(self))
213			}
214		}
215
216		impl<'de> serde_core::de::Deserialize<'de> for $type {
217			fn deserialize<D: serde_core::de::Deserializer<'de>>(
218				deserializer: D,
219			) -> Result<Self, D::Error> {
220				deserializer.deserialize_newtype_struct(
221					TOKEN,
222					crate::value::TextVisitor {
223						expecting: $expected,
224						parse: Self::parse,
225					},
226				)
227			}
228		}
229	};
230}
231
232#[cfg(feature = "chrono")]
233pub mod chrono;
234mod date;
235mod de;
236mod document;
237mod duration;
238mod error;
239mod instant;
240#[cfg(feature = "jiff")]
241pub mod jiff;
242mod macros;
243mod parse;
244mod scalar;
245mod ser;
246mod spanned;
247pub mod tree;
248mod value;
249mod write;
250
251pub use date::Date;
252pub use document::{Document, PathSegment};
253pub use duration::Duration;
254pub use error::{Error, ErrorKind, LineColumn};
255pub use instant::Instant;
256pub use spanned::Spanned;
257pub use value::{Index, Object, Value};
258
259use serde_core::de::{Deserialize, DeserializeOwned};
260use serde_core::ser::Serialize;
261
262/**
263Pulls the readme in as a doctest, so its examples are compiled and run by `cargo test`. Nothing is rendered; this only exists so the readme cannot drift from the API.
264*/
265// The readme shows the `jiff` feature, so its examples need it.
266#[cfg(all(doctest, feature = "jiff"))]
267#[doc = include_str!("../readme.md")]
268#[doc(hidden)]
269pub struct ReadmeDoctests;
270
271/**
272Reads a document into `T`.
273
274A `'...'` or `"..."` string without escapes is borrowed from `text`, so `T` can hold `&str` fields. Other strings are copied, so use `String` or `Cow<str>` when a document may hold them.
275
276# Errors
277
278Returns an error when `text` is not a valid document, or when the document does not fit `T`. The error has the line and column of the problem. To find the error that comes first, such a document is read a second time, so `T`'s `Deserialize` impl runs twice for it.
279*/
280pub fn from_str<'de, T: Deserialize<'de>>(text: &'de str) -> Result<T, Error> {
281	// The parser itself assumes these source-wide restrictions, so both reading paths share this check.
282	parse::check_characters(text)?;
283	// Most documents are valid and fit their type, so they are read in one pass, without a tree. After any error, the document is read again through the tree, which gives the error that a check of the whole document gives first.
284	if let Some(value) = de::stream::read(text) {
285		return Ok(value);
286	}
287
288	// An error that a type makes after reading, such as a check in `#[serde(try_from)]`, is about the whole document.
289	de::NodeDeserializer {
290		node: parse::parse_after_character_check(text)?,
291		source: Some(text),
292	}
293	.located(T::deserialize)
294}
295
296/**
297Reads a document from UTF-8 bytes into `T`.
298
299# Errors
300
301Returns an error when `bytes` is not UTF-8, when it is not a valid document, or when the document does not fit `T`.
302*/
303pub fn from_slice<'de, T: Deserialize<'de>>(bytes: &'de [u8]) -> Result<T, Error> {
304	from_str(parse::decode(bytes)?)
305}
306
307/**
308Reads a document from a reader into `T`.
309
310It reads everything first, because a document is only valid as a whole: an error at its end, such as a duplicate key, makes all of it invalid. So this is a convenience for `from_slice` on the bytes, and when you have the text already, [`from_str`] can borrow strings from it.
311
312```rust
313let config: std::collections::BTreeMap<String, u16> = soml::from_reader("port: 8080".as_bytes())?;
314assert_eq!(config["port"], 8080);
315# Ok::<(), soml::Error>(())
316```
317
318# Errors
319
320Returns an error of kind [`ErrorKind::Io`] when reading fails, and the errors of [`from_slice`] otherwise.
321*/
322pub fn from_reader<T: DeserializeOwned>(mut reader: impl std::io::Read) -> Result<T, Error> {
323	let mut bytes = Vec::new();
324	reader.read_to_end(&mut bytes).map_err(Error::io)?;
325	from_slice(&bytes)
326}
327
328/**
329Converts a [`Value`] into `T`.
330
331```rust
332let value: soml::Value = "port: 8080".parse()?;
333
334#[derive(serde::Deserialize)]
335struct Server {
336	port: u16,
337}
338
339let server: Server = soml::from_value(value)?;
340assert_eq!(server.port, 8080);
341# Ok::<(), soml::Error>(())
342```
343
344# Errors
345
346Returns an error when the value does not fit `T`. The error has no position, because a `Value` does not remember where it came from.
347
348A `Value` holds a float as an `f64`, so an `f32` is read through the `f64`'s shortest digits. For a document's text with more than 15 significant digits, that can be the `f32` next to the one [`from_str`] gives, which rounds the text itself.
349
350A `Value` from a document is never nested more than 100 levels deep. One built by hand that is nested thousands of levels deep can overflow the stack here.
351*/
352pub fn from_value<T: DeserializeOwned>(value: Value) -> Result<T, Error> {
353	T::deserialize(de::NodeDeserializer {
354		node: value.into(),
355		source: None,
356	})
357}
358
359/**
360Writes `value` as a document, for people to read. It follows every rule of canonical form except the order of members, which stay in the order the value gives them: a struct's fields in the order they are declared, and a map's entries in the order it iterates in. So a `BTreeMap` and a [`Value`] are sorted by key, an `IndexMap` keeps its insertion order, and a `HashMap` has a different order on each run. Use [`to_string_canonical`] for the same bytes every time.
361
362```rust
363#[derive(serde::Serialize)]
364struct Package {
365	name: &'static str,
366	description: &'static str,
367}
368
369let text = soml::to_string(&Package { name: "soml", description: "A config format" })?;
370assert_eq!(text, "name: 'soml'\ndescription: 'A config format'\n");
371# Ok::<(), soml::Error>(())
372```
373
374# Errors
375
376Returns an error when the value is not an object or an array, because a document is always a collection, or when it holds something SOML cannot represent: NaN, an integer outside the 64-bit range, a carriage return in a string or a key, two members with the same key, a map key that is not a string, a char, a bool, an integer, or a unit enum variant, or nesting deeper than 100 levels. To give the same error as [`to_string_canonical`], such a value is serialized a second time, so its `Serialize` impl runs twice for it. That is also the case for a `Serialize` impl that catches an error and goes on.
377*/
378pub fn to_string<T: ?Sized + Serialize>(value: &T) -> Result<String, Error> {
379	// Writing the text as the value is serialized is faster than building a tree first. When that fails, or a type caught an error, the tree serializer writes the value, because the stream finds errors in another order, so its error can be a different one, and it keeps the text that was written before an error.
380	match ser::stream::write(value) {
381		Some(text) => Ok(text),
382		None => write::document(&value.serialize(ser::NodeSerializer { depth: 0 })?),
383	}
384}
385
386/**
387Writes `value` as a document in canonical form, the one text the spec defines for a value: the same as [`to_string`], with the members of every object sorted by key. Two equal values always give the same bytes, so use it to hash, sign, or compare documents.
388
389```rust
390#[derive(serde::Serialize)]
391struct Package {
392	name: &'static str,
393	description: &'static str,
394}
395
396let text = soml::to_string_canonical(&Package { name: "soml", description: "A config format" })?;
397assert_eq!(text, "description: 'A config format'\nname: 'soml'\n");
398# Ok::<(), soml::Error>(())
399```
400
401# Errors
402
403Returns the errors of [`to_string`].
404*/
405pub fn to_string_canonical<T: ?Sized + Serialize>(value: &T) -> Result<String, Error> {
406	let mut node = value.serialize(ser::NodeSerializer { depth: 0 })?;
407	node.sort_members();
408	write::document(&node)
409}
410
411/**
412Writes `value` as a document to a writer, as [`to_string`] does.
413
414```rust
415let mut output = Vec::new();
416soml::to_writer(&mut output, &std::collections::BTreeMap::from([("port", 8080)]))?;
417assert_eq!(output, b"port: 8080\n");
418# Ok::<(), soml::Error>(())
419```
420
421# Errors
422
423Returns the errors of [`to_string`], and an error of kind [`ErrorKind::Io`] when writing fails. Nothing is written when the value cannot be written.
424*/
425pub fn to_writer<T: ?Sized + Serialize>(
426	mut writer: impl std::io::Write,
427	value: &T,
428) -> Result<(), Error> {
429	writer
430		.write_all(to_string(value)?.as_bytes())
431		.map_err(Error::io)
432}
433
434/**
435Writes `value` as a document in canonical form to a writer, as [`to_string_canonical`] does.
436
437# Errors
438
439Returns the errors of [`to_string`], and an error of kind [`ErrorKind::Io`] when writing fails. Nothing is written when the value cannot be written.
440*/
441pub fn to_writer_canonical<T: ?Sized + Serialize>(
442	mut writer: impl std::io::Write,
443	value: &T,
444) -> Result<(), Error> {
445	writer
446		.write_all(to_string_canonical(value)?.as_bytes())
447		.map_err(Error::io)
448}
449
450/**
451Formats a document: it changes layout and nothing else, so comments, member order, and the spelling of every value stay. See [`Document::format`].
452
453```rust
454assert_eq!(soml::format("a: {b: 1, c: [2,3]}  # Note.")?, "a: {b: 1, c: [2, 3]} # Note.\n");
455assert_eq!(soml::format("a: {\nb: 1, c: [2,3]}")?, "a: {\n\tb: 1\n\tc: [2, 3]\n}\n");
456# Ok::<(), soml::Error>(())
457```
458
459# Errors
460
461Returns an error when `text` is not a valid document.
462*/
463pub fn format(text: &str) -> Result<String, Error> {
464	let parts = tree::build(text, parse::parse(text)?);
465	Ok(tree::format::document(
466		&parts.outer,
467		&parts.root,
468		text.len(),
469	))
470}
471
472/**
473Converts `value` into a [`Value`].
474
475```rust
476let value = soml::to_value(&[1, 2, 3])?;
477assert_eq!(value, soml::Value::Array(vec![1.into(), 2.into(), 3.into()]));
478# Ok::<(), soml::Error>(())
479```
480
481# Errors
482
483Returns an error when the value holds something SOML cannot represent, as for [`to_string`]. A top-level scalar is not an error here, because a `Value` does not have to be a document.
484*/
485pub fn to_value<T: ?Sized + Serialize>(value: &T) -> Result<Value, Error> {
486	value
487		.serialize(ser::NodeSerializer { depth: 0 })
488		.map(parse::Node::into_value)
489}
490
491/**
492Shortens text quoted in an error message, so a huge token does not make a huge message. The length counts UTF-16 code units, as the JS reference implementation does, so that the messages are the same. The cut never splits a character.
493*/
494pub(crate) fn abbreviate(text: &str, maximum_length: usize) -> std::borrow::Cow<'_, str> {
495	if text.len() <= maximum_length {
496		return text.into();
497	}
498	let end = utf16_limit(text, 0, maximum_length);
499	if end < text.len() {
500		let mut shortened = String::with_capacity(end + '…'.len_utf8());
501		shortened.push_str(&text[..end]);
502		shortened.push('…');
503		return shortened.into();
504	}
505	text.into()
506}
507
508/**
509The byte offset in `text` that is `length` UTF-16 code units after `start`, or the end of `text`, so that a diagnosed length counts as in the JS reference implementation. A character that would cross the limit is left out.
510*/
511pub(crate) fn utf16_limit(text: &str, start: usize, length: usize) -> usize {
512	let mut units = 0;
513
514	for (index, character) in text[start..].char_indices() {
515		units += character.len_utf16();
516
517		if units > length {
518			return start + index;
519		}
520	}
521
522	text.len()
523}