Lisp S-Expressions Parser (lispexp)

A pure-Rust reader (lexer + parser) for S-expression syntax across many Lisp-family dialects, producing a faithful, position-annotated, code-vs-data-aware parse tree.

lispexp is deliberately reader-only: it does not evaluate, expand macros, or interpret the numeric tower. It reads source text into data — the shape, positions, and reader-macro structure needed to statically analyze Lisp code.

Features

One reader, many dialects. Scheme (R7RS-small), Guile, Racket, Common Lisp, Emacs Lisp, Clojure, Hy, Phel, Fennel, LFE (Lisp Flavoured Erlang), ISLisp, AutoLISP, Janet, and EDN — plus a tolerant .scm "Scheme superset" — selected via Options presets built from orthogonal, individually-toggleable syntax settings.
Position-annotated. Every datum carries a byte span and 1-based start line; a LineIndex maps offsets to 1-based (line, byte-column).
Code vs. data aware. Quote/quasiquote/unquote structure is preserved, and a pruning walk classifies each node as Code or Data so consumers descend into code and skip quoted data.
Fault-tolerant. Malformed input never panics; the reader returns a partial tree plus structured diagnostics, resynchronizing at the next top-level form, with a bounded recursion depth.
Zero-copy. The parse tree borrows &str slices from the source; verbatim round-trip is lossless via source spans.
Two layers. A tree reader (parse) and an independent token stream (lex) that tiles the input — for consumers such as a parinfer backend that need lexical state rather than a tree.
Definition-aware utilities. An opt-in annotate module tags definition forms (name, arglist, docstring, body, method dispatch) across dialects, and an indent module harvests Emacs Lisp indent specs.
Pure Rust, no unsafe, zero dependencies — cross-compiles cleanly. MSRV 1.70.

Install

cargo add lispexp

Usage

use lispexp::{parse, Options};

let parsed = parse("(define (square x) (* x x))", &Options::scheme());
assert!(parsed.errors.is_empty());
assert_eq!(parsed.data[0].head_symbol(), Some("define"));
assert_eq!(parsed.data[0].items().unwrap().len(), 3);

Pick a dialect with a preset (Options::clojure(), Options::emacs_lisp(), Options::edn(), …) or Options::for_dialect(Dialect::Racket), then adjust individual fields by assignment. The reader is fault-tolerant, so always inspect parsed.errors alongside parsed.data.

Beyond the core reader, the crate exposes: lex / Lexer (the token layer), walk (a code-vs-data pruning visitor), parse_form_at (positioned single-form reparse for incremental validation), LineIndex (offset ↔ line/column), and the annotate and indent utility modules.

Documentation

API docs are on docs.rs. The design is recorded in docs/design.md, the domain vocabulary in CONTEXT.md, and the decisions behind it in the ADRs under docs/adr/.

Copyright