Expand description
The alchemy language: declarative streaming transducers and renderers for the tabnas engine, parsed by a tabnas grammar plugin; and the types every tabnas transducer and renderer shares.
shared is the types: the event and table protocols, sinks,
failures and their codes, limits, selectors, datums, the renderers’
options and text boundary, and the two interfaces a program is lowered
through, shared::Routers and shared::Renderers. It depends on
nothing else in the fleet; with default-features = false it is the
whole crate, and tabnas-transduce and tabnas-render build on it.
Everything else is the language, behind the default language
feature. The reader:
lexandgrammar: the grammar plugin, installed on an engine withalchemy, or built withmake;parse_valueanswers the reader’s tagged tree andparsetheExprforms.ast:ExprwithSourceSpans, thecanonicaland layout [format] printers, andMAX_NESTING, the bound every form this crate builds respects.desugar: the core-form rewrites (defwith parameters,pipe, the shapes oflet,ifandmatch).
The language, reading Expr after desugar has run:
resolve: scopes and linking;typesandcheck: inference, affine streams, protocols, strict mode;effects: the effect summary and theexplainreport.value: runtime values, every oneSend, with streams and texts as plans;interp: the evaluator;lower: plans to the stages the host’s routers and renderers make;stdlib: the natives and the embedded library.program:compileandProgram, the API a host embeds. The host passes the routers (tabnas-transduce’s) and the renderers (tabnas-render’s) in; thedmogcommand, which does, is tabnas-dmog.
use tabnas_alchemy::{canonical, desugar, parse};
let src = "def export [input]\n pipe input\n table-from-json api-binding\n csv csv-options";
let core = desugar::program(parse(src)?, src)?;
assert_eq!(
canonical(&core),
"(def export (fn [input] (csv csv-options (table-from-json api-binding input))))"
);Re-exports§
pub use ast::canonical;pub use ast::canonical_form;pub use ast::format;pub use ast::same_program;pub use ast::Expr;pub use ast::SourceSpan;pub use ast::Sources;pub use ast::MAX_NESTING;pub use check::MAX_APPLIED;pub use grammar::alchemy;pub use grammar::make;pub use grammar::parse;pub use grammar::parse_file;pub use grammar::parse_value;pub use grammar::UNNAMED;pub use interp::MAX_EVAL_DEPTH;pub use interp::MAX_PLAN_STEPS;pub use program::compile;pub use program::compile_sources;pub use program::compile_sources_with;pub use program::compile_with;pub use program::Input;pub use program::Output;pub use program::Program;pub use program::Renderer;pub use program::Source;
Modules§
- ast
- The syntax tree:
Exprwith source spans, built from the reader’s tagged value tree, and the two printers, canonical and layout. - check
- The checker: conservative inference over the core forms (spec sections 10.2 and 15; design brief 4.3).
- desugar
- Desugaring: the conveniences rewritten into the core forms (design brief section 4.2; spec sections 9.3, 9.4 and Appendix B).
- effects
- The effect summary and the plan report (spec sections 15.1, 15.2 and 15.5; design brief 4.3).
- grammar
- The alchemy grammar plugin: the grammar document (embedded from
alchemy-grammar.jsonicat the repository root, the one source every runtime embeds), the native actions it names, and the parse entry points. - interp
- The evaluator (design brief section 4.4; spec sections 10.1 and 10.3).
- lex
- The layout lex matcher: indentation as tokens, and the word tokens of the language.
- lower
- Lowering: a
Planbecomes a chain of sinks, the routing stages made by the host’sRoutersand the rendering ones by itsRenderers(transduce’s and render’s, which the host passes in). - program
- The public API a host embeds:
compilea program (orcompile_sources, several files linked into one), ask it what it produces, and take aSinkto push the source’s events into. - pvec
- A persistent vector: the items in leaves of
LEAFunder a trie, and the last up toLEAFof them in a tail, as Clojure’s vector keeps them. A push copies the tail, and once inLEAFpushes the path to the leaf the full tail becomes, so it costs a bounded amount whatever the vector’s length, and the vector it pushed onto is unchanged and shares every leaf with the new one. Every node keeps theSummaryof the items under it once something has asked for it, so a walk that sums the items reads a leaf’s sum instead of visiting its items again, and a vector nothing sums costs nothing for it. - resolve
- Scopes and linking (design brief section 4.2).
- rope
- A string value: its text in one piece, or, once
string-joinmakes one ofROPE_MINbytes or more, a rope: the chunks it was joined from, kept in a persistent vector, and its length. A join onto a rope copies at mostCHUNK_MAXbytes whatever the rope’s length, and the rope it joined onto is unchanged and shares every chunk with the new one, so a text a program builds a piece at a time (a:textrule’s text, a pass at a time) costs each piece its own bytes, not the text’s so far. A rope’s length is known without reading its text; the text in one piece is made the first time something reads it, and kept. - shared
- The types every tabnas transducer and renderer shares, and the two interfaces a compiled program is lowered through.
- stdlib
- The standard library: the natives (
registry) and the definitions written in alchemy itself. The crate embeds copies underrs/stdlib/*.alc, because a crates.io package cannot contain the canonicalstdlib/*.alcfiles above its package root;make embed(scripts/embed.js) writes them. A unit test holds each packaged copy byte-for-byte to its canonical file, andtests/shared_sources_test.rsholds the two directories to the same files andSOURCESto all of them. - translate
- Translation between formats, composed from per-format parts (admin
ADR-27; the design is tabnas/transduce’s
docs/translation.md). - types
- The checker’s types (spec sections 10.2 and 15; design brief 4.3).
- value
- Runtime values (design brief section 4.3; spec sections 10.2 and 10.3).
Constants§
- STACK_
BYTES - The stack the checker and the evaluator are given:
compileruns on a thread of this size, and so does thedmogcommand, so a program atMAX_NESTINGand an evaluation atMAX_EVAL_DEPTHfit in a debug build as in a release one. A host that pushes events into aProgram::sinkruns the program’s per-item functions on its own thread and gives it at least this much (aless’s parse thread has it). - VERSION
- This crate’s version, as
Cargo.tomldeclares it.