; Expressions' render: a tree's events written as an expression document,
; jsonic text with the plugin's default operators. This is a library of
; definitions, each named `expr-...`, which a host links with its own
; program (alchemy's `compile_sources`); its entry point is `expr-render`,
; and nothing else here is the host's to call.
;
; The tree is the one the reader builds with its default options:
; jsonic's values, in which an operation is a list whose first element is
; its operator and whose other elements are its terms. The reader's own
; value puts the operator's description there, an object whose `src`
; member is the operator's source text (TypeScript's parse, and Rust's
; once realized, give it so); the shared fixtures' simplifier, Go's
; `Simplify` and Rust's `simplify` reduce each description to that text,
; `+`, `-`, `*`, `/` or `%` (`1+2*3` is `["+", 1, ["*", 2, 3]]`, `-1` is
; `["-", 1]`) or `(` for a group (`(1)` is `["(", 1]`, `()` is `["("]`).
; The render reads both: the first element of a list is the operator its
; text is, or the operator an object describes, read by its `src` member
; as those simplifiers read it, when that is a default operator's source.
;
; The simplifiers read any list led by an object with a `src` member that
; way, and keep that member's value alone: TypeScript's `simplify` where
; the `src` is not empty, zero, false or null, and Go's `SimplifyOrdered`
; and Rust's `simplify` where it is a string other than the empty one. So
; a list led by an object whose `src` they read and is not a default
; operator's source, as the tokens a C syntax tree puts first in a node's
; children are, is no tree of the reader's: no document reads back as
; it, and the render refuses it (below).
;
; Every value is written so that the reader builds the same tree from it:
;
; - An operation is written in infix: `a+b` for a list of an infix
; operator and two terms, `-a` or `+a` for a list of `-` or `+` and one
; term, `(a)` for a group and `()` for an empty one. The default
; operators' binding powers decide where a term needs grouping: `*`,
; `/` and `%` bind tighter than `+` and `-`, every infix operator binds
; to the left, and a prefix operator binds tighter than any infix one.
; Parentheses would read back as a group, a list `["(", ...]`, so a
; term that needs grouping (`["+", 1, 2]` under `*`, or on the right of
; `-`) is written as a list instead, `["+",1,2]*3`, which the reader
; builds as the same list.
; - Any other list, one whose first element is no operator or which has
; another number of terms (`["*", 1]`, `["+", 1, 2, 3]`), is written as
; a list, `[a,b]`, each element in infix where it is an operation.
; - An object is written `{"k":v}`, every key quoted.
; - A string is quoted (JSON's escapes, which jsonic reads), so a word is
; written `"a"`; a number by its lexeme, or as JSON writes it; a
; boolean and null by their names. A negative number is written with
; its sign, which the reader reads as the operator `-` applied to the
; number's magnitude, and a number that is not finite, which jsonic
; has no spelling for, as the string `Infinity`, `-Infinity` or `NaN`.
; - A list whose first element is an object with a `src` member that the
; simplifiers read is written as an operation only where that `src` is
; a default operator's source text, as the reader's own value describes
; the operator (the manifest's loss list says the object's other
; members are not kept). With any other such `src` the list fails with
; TARGET_VALUE_UNREPRESENTABLE, naming the object's path, in jq's
; syntax, and its `src` as the render writes it. The `src` is judged as
; it is written, since that is what the reader reads: a number that is
; not finite as a string, and a negative number as the operation its
; sign makes, a list. An object whose `src` is empty, zero, false or
; null is written as it is, and reads back as itself.
;
; No white space is written, since jsonic reads a space between two
; values as an implicit list, and the document ends with one line break.
;
; Whether a list is written as an operation is known from its first
; element and its count, so a list whose first element is a default
; operator is held until it ends, its elements' texts with it, and then
; written. A list whose first element is an object is held until that
; object ends, since the object may describe an operator: if it does, the
; list is held to its end as an operation is, and if not, the list is
; written as it is read from then on. Every other list, and every object,
; is written as it is read, one element at a time, so a document's own
; lists and objects stream and only its operations, and the first object
; of a list, are held. Events no tree has fail with PROTOCOL_ORDER_ERROR.
;
; The state is a stack of markers, one for each container the render is
; inside:
;
; [:pending mode] a list whose first element is due, which
; decides how it is written; mode is
; :stream where it is written as it is
; read and :hold inside a held list
; [:list :first] a list whose first element, an object,
; is being held, which decides how the
; list is written
; [:list :stream index] a list written as it is read; index is
; [] until an element is written, and then
; the decimal digits of the last one's
; index, most significant first
; [:list :hold items] a held list, and the items read so far
; [:map :stream :first :key] an object written as it is read, its
; first key or its end due
; [:map :stream [:value k]] the same, the value of the key k due
; [:map :stream [:after k] :key] the same, after the member of the key
; k, its next key or its end due
; [:map :hold parts :key] an object inside a held list, or the
; first element of a list, and its
; members' texts
; [:map :hold parts [:value k]] the same, the value of the key k due,
; k quoted
;
; The markers say where the render is, which only a refusal reads: the
; list a value is in, and its index, or the object and its key. alchemy
; has no arithmetic, so an index goes up a digit at a time.
;
; An item is `[text class list head]`: the text a value is written as;
; its class, `:infix2` for an operation of `+` or `-`, `:infix3` for one
; of `*`, `/` or `%`, and `:atom` for any other value, which binds as
; tightly as a term can; the text of the value as a list, for a term that
; needs grouping; and the operator a string is or an object describes, or
; `:none`. A list's first item that is an operator is held as the
; operator's text, so a description is written as its source, as the
; simplifiers reduce it, wherever its list is written. Once the root
; value is written the state is `[:done]`, and nothing may follow it.
; The operator a value is, when it is the source of a default one.
def expr-op [value]
match (kind value)
case :string
match value
case "+" value
case "-" value
case "*" value
case "/" value
case "%" value
case "(" value
case _ :none
case _ :none
; The operator a held object's member describes: the default operator a
; `src` member's text names, or :none for any other member.
def expr-src-op [part]
match part
case "\"src\":\"+\"" "+"
case "\"src\":\"-\"" "-"
case "\"src\":\"*\"" "*"
case "\"src\":\"/\"" "/"
case "\"src\":\"%\"" "%"
case "\"src\":\"(\"" "("
case _ :none
; Whether a held object's member is its `src`: a member's text is its
; quoted key, a colon and its value's text, and a quoted key ends at its
; first unescaped quote, so the member is `src` exactly when its text
; begins with "src":.
def expr-is-src [part]
expr-leads "\"src\":" part
; The text of a `src` member's value, what follows its key and colon.
def expr-src-value [part]
let [pieces (split "\"src\":" part)]
string-join "\"src\":" (map (fn [i] (expr-at pieces i)) (filter expr-after-first (indices pieces)))
def expr-after-first [i]
match i
case 0 false
case _ true
; Whether a simplifier takes a list led by an object for an operation,
; the object's `src` member's value written as this text: TypeScript's
; does where the value is not empty, zero, false or null, and Go's and
; Rust's where it is a string other than the empty one, a narrower set.
; The value is judged as written, which is what the reader reads: a
; number that is not finite is written as a string, and a negative one
; with its sign, which reads back as an operation, a list.
def expr-src-read [text]
match text
case "\"\"" false
case "false" false
case "null" false
case _
match (is-number text)
case false true
case true
match (expr-at (split "-" text) 0)
case "" true
case _
match (compare (number text) 0)
case :equal false
case _ true
; What a held object makes of a list it is the first element of, from its
; members' texts: the default operator its `src` member names, as the
; reader's description of an operator does; :none where it has no `src`,
; or one no simplifier reads; and `[:foreign text]` for any other `src`,
; text its value as written. Where a key repeats, the last member is the
; one the reader keeps.
def expr-src-of [parts]
let [srcs (filter expr-is-src parts)]
match (count srcs)
case 0 :none
case _
let [part (top srcs)]
match (expr-src-op part)
case :none
let [text (expr-src-value part)]
match (expr-src-read text)
case true [:foreign text]
case false :none
case op op
; An item as its list holds it: the first one, where it is an operator,
; as that operator's text, so that a description is written as the
; source it describes; any other as it is.
def expr-lead [items item]
match (count items)
case 0
match item
case [_ _ _ :none] item
case [_ _ _ op] (expr-scalar-item op)
case _ item
; An infix operator's class.
def expr-level [op]
match op
case "+" :infix2
case "-" :infix2
case _ :infix3
; A number: its lexeme, or as JSON writes it; one that is not finite as
; a string.
def expr-number [value]
match (number-class value)
case :finite (scalar-text csv-options value)
case :infinity (quoted "Infinity")
case :negative-infinity (quoted "-Infinity")
case :nan (quoted "NaN")
def expr-scalar-text [value]
match (kind value)
case :string (quoted value)
case :number (expr-number value)
case :boolean (scalar-text csv-options value)
case :null "null"
def expr-scalar-item [value]
let [text (expr-scalar-text value)]
[text :atom text (expr-op value)]
def expr-text [item]
match item
case [text _ _ _] text
; A held list's elements written as a list.
def expr-list [items]
string-join "" ["[" (string-join "," (map expr-text items)) "]"]
; A left term of an infix operator of the class given: an operation that
; binds as tightly or more, or the term as a list.
def expr-left [item level]
match item
case [text :atom _ _] text
case [text :infix3 _ _] text
case [text :infix2 list _]
match level
case :infix2 text
case _ list
; A right term: an operation that binds more tightly, or the term as a
; list.
def expr-right [item level]
match item
case [text :atom _ _] text
case [text :infix3 list _]
match level
case :infix2 text
case _ list
case [_ :infix2 list _] list
; A prefix operator's term: one that binds as tightly, or the term as a
; list.
def expr-term [item]
match item
case [text :atom _ _] text
case [_ _ list _] list
; An infix operation, when the first element is an infix operator.
def expr-infix [op a b list]
match op
case :none [list :atom list :none]
case "(" [list :atom list :none]
case _
let [level (expr-level op)]
[(string-join "" [(expr-left a level) op (expr-right b level)]) level list :none]
; A held list, ended: an operation where its first element and its count
; make one, and a list otherwise.
def expr-array [items]
let [list (expr-list items)]
match items
case [[_ _ _ "("]] ["()" :atom list :none]
case [[_ _ _ "("] x] [(string-join "" ["(" (expr-text x) ")"]) :atom list :none]
case [[_ _ _ "+"] x] [(string-join "" ["+" (expr-term x)]) :atom list :none]
case [[_ _ _ "-"] x] [(string-join "" ["-" (expr-term x)]) :atom list :none]
case [[_ _ _ op] a b] (expr-infix op a b list)
case _ [list :atom list :none]
; The top marker, or :none before the root value.
def expr-top [s]
match (count s)
case 0 :none
case _ (top s)
; A vector's item at a position.
def expr-at [v i]
get-path (path i) v
def expr-mark [marker s]
push marker (pop s)
; A value written where it is read: at the root, which then is done, in
; a list written as it is read, after a comma unless it is the first,
; or as an object's member, whose next key is then due. `text` is what
; the value writes now, and `inner` the marker it opens, or :none.
def expr-stream-value [s text inner]
match (expr-top s)
case :none (expr-push-or-done [] inner [text])
case [:list :stream index]
match index
case [] (expr-push-or-done (expr-mark [:list :stream ["0"]] s) inner [text])
case _ (expr-push-or-done (expr-mark [:list :stream (expr-count-up index)] s) inner ["," text])
case [:map :stream [:value k]] (expr-push-or-done (expr-mark [:map :stream [:after k] :key] s) inner [text])
def expr-push-or-done [s inner out]
match inner
case :none
match (count s)
case 0 (transition [[:done]] (push "\n" out))
case _ (transition s out)
case _ (transition (push inner s) out)
; A value completed inside a held list or object: its item is added.
def expr-hold-value [s item]
match (expr-top s)
case [:list :hold items] (transition (expr-mark [:list :hold (push (expr-lead items item) items)] s) [])
case [:map :hold parts [:value k]] (transition (expr-mark [:map :hold (push (string-join "" [k ":" (expr-text item)]) parts) :key] s) [])
; A held value has ended: it is added to the held container around it,
; decides the list it is the first element of, or is written where
; values are written as they are read, where what led it (a comma, a
; key) was written when it began.
def expr-held [s item]
match (expr-top s)
case [:list :hold _] (expr-hold-value s item)
case [:map :hold _ _] (expr-hold-value s item)
case [:list :first] (expr-first-held s item)
case _ (expr-closed s (expr-text item))
; A list whose first element, an object, has ended: one that describes
; an operator makes the list an operation, held as one whose first
; element is that operator's text is; any other is written, the list's
; `[` first, and the rest of the list as it is read.
def expr-first-held [s item]
match item
case [text _ _ :none] (transition (expr-mark [:list :stream ["0"]] s) ["[" text])
case _ (transition (expr-mark [:list :hold [(expr-lead [] item)]] s) [])
; Whether the top of the stack holds what is read.
def expr-holding [s]
match (expr-top s)
case [:list :hold _] true
case [:map :hold _ _] true
case [:list :first] true
case _ false
; A list whose first element is due decides on that element's first
; event, and answers its marker and the text it writes: one whose first
; element is a default operator may be an operation and is held; one
; whose first element is an object waits for the object, which may
; describe an operator; any other is written as it is read, `[` first,
; unless it is inside a held one.
def expr-decide [s mode event]
match [mode event]
case [:stream (scalar value)]
match (expr-op value)
case :none [(expr-mark [:list :stream []] s) "["]
case _ [(expr-mark [:list :hold []] s) ""]
case [:stream object-start] [(expr-mark [:list :first] s) ""]
case [:stream _] [(expr-mark [:list :stream []] s) "["]
case _ [(expr-mark [:list :hold []] s) ""]
; A value where one is due: a scalar, or a container's start.
def expr-value [s event]
match (expr-holding s)
case true
match event
case (scalar value) (expr-hold-value s (expr-scalar-item value))
case object-start (transition (push [:map :hold [] :key] s) [])
case array-start (transition (push [:pending :hold] s) [])
case false
match event
case (scalar value) (expr-stream-value s (expr-scalar-text value) :none)
case object-start (expr-stream-value s "{" [:map :stream :first :key])
case array-start (expr-stream-value s "" [:pending :stream])
; The first element of a list, once the list has decided: what the
; decision writes, then what the element does.
def expr-first [s mode event]
match (expr-decide s mode event)
case [s2 lead]
match (expr-value s2 event)
case (transition s3 []) (transition s3 [lead])
case (transition s3 [text]) (transition s3 [(string-join "" [lead text])])
; A container ends; a held one's item is added or written.
def expr-close [s marker]
match marker
case [:pending :stream] (expr-closed (pop s) "[]")
case [:pending :hold] (expr-held (pop s) ["[]" :atom "[]" :none])
case [:list :stream _] (expr-closed (pop s) "]")
case [:list :hold items] (expr-held (pop s) (expr-array items))
; A container written as it was read has ended: its closing text, and
; the document's end at the root.
def expr-closed [s text]
match (count s)
case 0 (transition [[:done]] [text "\n"])
case _ (transition s [text])
def expr-key [s name]
match (expr-top s)
case [:map :stream :first :key] (transition (expr-mark [:map :stream [:value name]] s) [(quoted name) ":"])
case [:map :stream [:after _] :key] (transition (expr-mark [:map :stream [:value name]] s) ["," (quoted name) ":"])
case [:map :hold parts :key] (transition (expr-mark [:map :hold parts [:value (quoted name)]] s) [])
case _ (fail :protocol-order "the events hold a key where a value is due, which a tree's never do")
; An object ends. A held one becomes an item, which records the operator
; it describes; one that is the first element of a list, with a `src` a
; simplifier would read as that list's operator and no default one, is
; refused.
def expr-object-end [s]
match (expr-top s)
case [:map :stream _ :key] (expr-closed (pop s) "}")
case [:map :hold parts :key]
let [text (string-join "" ["{" (string-join "," parts) "}"])]
match (expr-src-of parts)
case [:foreign src]
match (expr-leads-list (pop s))
case true (expr-refuse (pop s) src)
case false (expr-held (pop s) [text :atom text :none])
case described (expr-held (pop s) [text :atom text described])
case _ (fail :protocol-order "the events end an object that is not open, or one whose member has no value, which a tree's never do")
; Whether the top of the stack is a list whose first element is due, so
; that an object ending there is its first element.
def expr-leads-list [s]
match (expr-top s)
case [:list :first] true
case [:list :hold []] true
case _ false
; A list led by an object whose `src` a simplifier would read as the
; list's operator, and which is no default operator's source text: no
; document reads back as the tree, so it is refused, naming the object,
; by the markers of the containers it is in, and its `src` as the render
; writes it.
def expr-refuse [s src]
fail :unrepresentable (string-join "" ["the object at " (expr-path-text s) " is the first element of a list and has the src " src ", which expr's reader would read back as the list's operator, keeping only the src; the render writes such a list only where the src is a default operator's source text"])
; The path to the value being read, from the markers of the containers it
; is in, in jq's syntax as aless prints and reads one: `.name` for a
; member whose key is a word, `."key"` for one whose key is not, and `[i]`
; for an element, `.[i]` at the root. While a value is read, the list it
; is in is held, written as it is read or deciding on its first element,
; and the object it is in has the value's key, held or written.
def expr-path-text [s]
let [text (string-join "" (map expr-path-part s))]
match (expr-at s 0)
case [:map _ _ _] text
case _ (string-join "" ["." text])
def expr-path-part [marker]
match marker
case [:list :first] "[0]"
case [:list :stream index] (string-join "" ["[" (string-join "" index) "]"])
case [:list :hold items] (string-join "" ["[" (scalar-text csv-options (count items)) "]"])
case [:map :stream [:after k] :key] (expr-path-key k)
case [:map :hold _ [:value k]] (expr-path-key (unquoted k))
; A key as a path writes it: bare where it is a word, letters, digits and
; underscores not led by a digit, and quoted otherwise.
def expr-path-key [name]
match [(length name) (chars-within [[48 57] [65 90] [95 95] [97 122]] name)]
case [0 _] (string-join "" ["." (quoted name)])
case [_ false] (string-join "" ["." (quoted name)])
case _
match (count (filter (fn [digit] (expr-leads digit name)) ["0" "1" "2" "3" "4" "5" "6" "7" "8" "9"]))
case 0 (string-join "" ["." name])
case _ (string-join "" ["." (quoted name)])
; Whether a string that is not empty begins with another.
def expr-leads [lead text]
match (expr-at (split lead text) 0)
case "" true
case _ false
; The digits of the next index: the last digit up by one, and a 9
; carried.
def expr-count-up [digits]
match (top digits)
case "9" (expr-carry digits)
case digit (push (expr-digit-up digit) (pop digits))
def expr-digit-up [digit]
match digit
case "0" "1"
case "1" "2"
case "2" "3"
case "3" "4"
case "4" "5"
case "5" "6"
case "6" "7"
case "7" "8"
case "8" "9"
; The digits after ones that end in a 9: the nines at the end become
; zeros and the digit before them goes up, or, where every digit is a 9, a
; 1 leads zeros one more than there were nines. Written as one string, a
; 9 for each nine and a bar for any other digit, the digits split at the
; bars, and what comes before the last bar is as long as the position of
; the digit that goes up.
def expr-carry [digits]
let [runs (split "|" (string-join "" (map expr-nine-or-bar digits)))]
match (count runs)
case 1 (push "0" (map expr-one-then-zero (indices digits)))
case _
let [up (length (string-join "|" (pop runs)))]
map (fn [i] (expr-carried digits up i)) (indices digits)
def expr-nine-or-bar [digit]
match digit
case "9" "9"
case _ "|"
def expr-one-then-zero [i]
match i
case 0 "1"
case _ "0"
def expr-carried [digits up i]
match (compare i up)
case :less (expr-at digits i)
case :equal (expr-digit-up (expr-at digits i))
case _ "0"
def expr-array-end [s]
match (expr-top s)
case [:pending mode] (expr-close s [:pending mode])
case [:list mode x] (expr-close s [:list mode x])
case _ (fail :protocol-order "the events end an array that is not open, which a tree's never do")
; Where a value is due: at the root, in a list, or after a key.
def expr-due [s event]
match (expr-top s)
case :none (expr-value s event)
case [:pending mode] (expr-first s mode event)
case [:list _ _] (expr-value s event)
case [:map :stream [:value _]] (expr-value s event)
case [:map :hold _ [:value _]] (expr-value s event)
case _ (fail :protocol-order "the events hold a value where a key is due, which a tree's never do")
def expr-step [s event]
match (expr-top s)
case [:done] (fail :protocol-order "the events hold more after the root value, which a tree's never do")
case _
match event
case (key name) (expr-key s name)
case object-end (expr-object-end s)
case array-end (expr-array-end s)
case _ (expr-due s event)
def expr-finish [s]
match (expr-top s)
case [:done] []
case :none (fail :protocol-order "the events hold no value, where a tree's hold one")
case _ (fail :protocol-order "the events ended inside a container, which a tree's never do")
; The render: a tree's events in, the expression document's text out.
def expr-render [input]
join ""
scan-emit [] expr-step expr-finish (events input)