pub enum QueryOp {
Show 131 variants
XPath(Box<XPath>),
Emit(OutFmt, usize),
Match(Regex),
Reject(Regex),
Field(FieldSel),
Fields(Vec<usize>),
Each,
Where(Expr),
Map(Expr),
Via(Arc<ActorDef>, usize),
Count,
Rate,
Tally,
Sum,
Min,
Max,
Avg,
Keys,
JqKeys,
Vals,
NonNull,
JqRawString,
Pick(Vec<String>),
Sort {
numeric: bool,
reverse: bool,
},
Uniq,
Rev,
First,
Last,
Take(usize),
Drop(usize),
Upper,
Lower,
Trim,
Replace(Regex, String),
Join(String),
Nth(usize),
Sel {
css: String,
attr: Option<String>,
},
Find(String),
Attr(String),
Text,
SortBy(String),
UniqueBy(String),
CountBy(String),
GroupBy(String),
MinBy(String),
MaxBy(String),
Has(String),
HasKey(Value),
Entries,
JqEntries,
JqFlatten,
JqMap(Vec<QueryOp>),
JqEach,
JqAdd,
JqLen,
JqProgram(String),
JqSelect(Expr),
JqCalc(Expr),
Flatten,
Add,
Over(f64),
Under(f64),
Between(f64, f64),
Enumerate,
Words,
Dedup,
Tailn(usize),
Pad(usize),
Lpad(usize),
Grepf(String, Regex),
Flip,
Basename,
Dirname,
Commafy,
Bytes,
Duration,
Apply(String),
JsonDocs,
Csv,
B64,
B64d,
Hex,
Unhex,
Urlenc,
Urldec,
Extract(Regex),
Split(String),
Substr(usize, usize),
Chars,
Title,
Repeat(usize),
Set(String, String),
Del(String),
Rename(String, String),
Default(String, String),
Merge,
Floor,
Ceil,
Clamp(f64, f64),
Tsv,
Yaml,
Toml,
Calc(Expr),
Contains(String),
Starts(String),
Ends(String),
Nonempty,
Numeric,
Len,
JsonLen,
JsonSlice(Option<i64>, Option<i64>),
Wc,
Abs,
Round,
Delta,
Cumsum,
Sma(usize),
Ewma(f64),
Prepend(String),
Append(String),
Cut(String, usize),
Median,
Percentile(f64),
Stddev,
Range,
Product,
Distinct,
Sample(usize),
Slice(usize, usize),
Index(usize),
Bins(usize),
}Variants§
XPath(Box<XPath>)
A compiled XPath 1.0 expression (crate::xpath). The expression is
parsed at spec-build time, so this op always holds a well-formed one and
a malformed path is refused before any input is read.
Emit(OutFmt, usize)
Re-serialize the whole stream in another format (out.yaml 4). The
number is the indent, yq’s -I.
Match(Regex)
Keep lines matching the pattern.
Reject(Regex)
Drop lines matching the pattern.
Field(FieldSel)
Replace each line with a selected field (whitespace column or JSON key path).
Fields(Vec<usize>)
Project multiple whitespace columns (1-based), space-joined, keeping the
given order — fields 1 3 on “a b c d” -> “a c”. For columnar input.
Each
Flatten JSON-array lines into one line per element (jq []); non-array
lines pass through unchanged.
Where(Expr)
Keep lines whose numeric value (x = line parsed as a number) satisfies
the predicate — compiled to fusevm and evaluated per line.
Map(Expr)
Replace each line with the value of an expression (field-aware; x =
line-as-number), computed on the fusevm VM.
Via(Arc<ActorDef>, usize)
via NAME [* N] — fan the stream across a supervised pool of N copies
of actor NAME (default: one per hardware thread). Each line’s scalar is
asked as the actor’s first-handler message; the reply is the output line,
order preserved. A parallel actor map (see crate::actor::run_via).
Count
Reduce to the current line count.
Rate
Reduce to lines-per-second over the elapsed window.
Tally
Group identical values and count them, sorted by count desc then key asc.
Sum
Numeric reductions over lines parsed as numbers (non-numeric ignored).
Min
Max
Avg
Keys
Flatten a JSON object’s keys / values into one line each.
JqKeys
The jq front-end’s keys. Same key set as Keys, but emitted as ONE
compact array line, which is what jq returns. The native keys verb keeps
its documented line-per-key shape (SPEC §8) — stdlib/json.arb runs
keys; tally over it — so the two are separate ops and only the jq
literal front-end (. | keys, map(…)) reaches this one.
Vals
Native vals verb: expand a JSON object’s values, one per line.
NonNull
jq values == select(. != null): drop JSON-null lines, pass every other
line through unchanged. (NOT object-value iteration — that is vals/.[].)
JqRawString
Render a PASSTHROUGH line the way jq -r prints it, for the one type where
the raw line and jq’s rendering disagree: a top-level JSON STRING.
Identity ., select(…) and values all emit the INPUT LINE verbatim.
For an object, array, number or boolean that is what jq does too — jq keeps
the source literal of any number it never computed (1.50 stays 1.50,
[1,2] stays [1,2]), so passing the line through matches it. A string is
the exception: jq -r strips the quotes and unescapes, so a line reading
"hello" must print hello. arb printed "hello" — and disagreed with
ITSELF, since .[1:3] on the same input already rendered el raw.
Deliberately string-ONLY. Re-rendering the other types would lose more than
it gained: serde_json::Map is a BTreeMap, so a rebuilt object comes back
key-SORTED ({"b":1,"a":2} -> {"a":2,"b":1}) where jq preserves input
order, and every number would be reprinted through fmt_num, turning jq’s
preserved 1.50 into 1.5. Both would be new divergences in place of one.
Pick(Vec<String>)
Project a JSON object down to the named keys (jq {a,b,c} /
pick(.a,.b,.c)), preserving the listed order. Non-object lines pass
through unchanged; missing keys are dropped.
Sort
Line-list transforms. Sort supports numeric (-n) and reverse (-r).
Uniq
Rev
First
Last
Take(usize)
Drop(usize)
Upper
Per-line string transforms.
Lower
Trim
Replace(Regex, String)
Regex replace-all per line (replace /RE/ TO; TO may use $1 captures).
Join(String)
Collapse all lines into one, joined by a separator.
Nth(usize)
Keep only the Nth line (1-based).
Sel
Parse the accumulated stream as one HTML document and emit, per element
matching the CSS selector, its text (or a named attribute if attr is
set; elements lacking that attribute are dropped).
Find(String)
Recursive descent (xpath //TAG): parse the accumulated stream as one
HTML document and emit the outer HTML of every element matching the
selector, one per line — so attr/text can then pull from each.
Attr(String)
From element fragments (one per line, as emitted by find), emit the
named attribute of each (xpath @NAME, css attr); drop lines whose first
element lacks it.
Text
From element fragments, emit each element’s inner text; non-element lines pass through unchanged.
SortBy(String)
Stable-sort JSON-record lines by FIELD; numeric if every field value parses as a number, else lexicographic on the field’s string value. Non-object lines sink after the sorted records in input order.
UniqueBy(String)
Keep the first record for each distinct value of FIELD, preserving input order. JSON-object lines dedup by that field’s value; other lines dedup by the whole line.
CountBy(String)
Group JSON records by the value of FIELD and count each group, returning value -> count pairs sorted by count desc then value asc; non-object lines are counted under their whole-line text. Reducer (early return).
GroupBy(String)
Group lines by the value of FIELD (jq group_by): emit one JSON-array
line per distinct value, each array holding that group’s members in input
order; groups are ordered by key ascending. Object lines group by the
field’s value, other lines by their whole-line text.
MinBy(String)
Reducer: emit the single JSON record whose numeric FIELD is smallest (records with a missing/non-numeric FIELD are ignored; empty input yields no lines).
MaxBy(String)
Reducer: emit the single record whose numeric FIELD is the largest.
Has(String)
Native has KEY verb: retain only JSON-object lines that contain KEY; all
other lines (missing key, non-object, unparseable) are dropped.
HasKey(Value)
jq has(KEY): emit true/false per input line. The argument is a jq
VALUE, not a name — jq accepts a string key on an object and a numeric
index on an array, and refuses every other pairing by name (“Cannot check
whether array has a string key”) rather than answering false.
Entries
jq to_entries: expand each JSON object line into one {"key":<k>,"value":<v>} line per key (BTreeMap key order); non-object lines pass through.
JqEntries
The jq front-end’s to_entries. Same mapping as Entries, but emits the
whole result as ONE compact array line, which is what jq returns. Native
entries keeps its documented line-per-key shape (SPEC §8) so specs built
on it are unaffected; the two spellings are distinct, so they can differ.
JqFlatten
The jq front-end’s flatten. Same full-depth leaf walk as Flatten, but
emits them as ONE compact array line, which is what jq returns. Native
flatten keeps its line-per-leaf shape — same context gating as
JqKeys/JqEntries.
JqMap(Vec<QueryOp>)
jq map(f) == [.[] | f]: run the inner ops over the ELEMENTS of each
input line and re-wrap the results as one compact array line — jq’s shape.
Scoped per input line, so two input lines stay two output arrays. (Iterate
WITHOUT the re-wrap is jq’s own .[] | f, which arb also accepts.)
JqEach
The jq front-end’s .[]. Same expansion as Each, but a line that parses
to a NON-iterable JSON value (a scalar or null) is an error rather than a
pass-through — jq raises “Cannot iterate over null (null)” and SPEC §8
forbids answering where jq refuses. Native each keeps the pass-through,
because it runs over plain-text streams too.
JqAdd
The jq front-end’s add — reduce .[] as $x (null; . + $x), so an EMPTY
array is null (native add documents [] -> "") and a mixed-type array
raises the same error + would. Iterates an object’s values.
JqLen
The jq front-end’s length. Same JSON-aware measure as JsonLen, except a
boolean is an error (“boolean (true) has no length”) instead of falling back
to the raw line’s character count.
JqProgram(String)
A COMPLETE jq program, run per line by crate::jqlang.
The ops above are arb’s line-stream translations of the jq subset they
cover, and they keep arb’s own promises (identity passes the SOURCE line
through, so its spacing and number literals survive). They cannot express
jq’s generator semantics — .a, .b, reduce, .., try, assignment —
because a Vec<QueryOp> is a stage list, not a language. Anything
outside that subset compiles to a real jq program instead and runs on
arb’s own jq engine, which is why the constructs SPEC §8 used to list as
refusals now answer exactly as jq does.
The payload is the SOURCE, not a compiled program: QueryOp is cloned
into specs, actors and the web target, so it must stay Send-compatible
plain data. Compilation is memoized per thread by jq_program, so a
per-line pipeline still parses the program exactly once.
JqSelect(Expr)
jq select(f) over jq VALUE semantics: keep the line when f is truthy,
where only false and null are falsy. Distinct from native Where,
which evaluates an f64 predicate over a numeric line stream.
JqCalc(Expr)
A jq value expression stage (.a + .b, . * 2, . > 1). Distinct from
native Map, which computes an f64: this yields a jq VALUE, so a compare
renders true/false and a string + concatenates.
Flatten
jq flatten: recursively flatten a JSON-array line to its non-array leaves,
emitting one leaf per line (matching jq’s full-depth flatten, unlike each
which descends a single level). Non-array lines pass through unchanged.
Add
jq add: reduce a JSON array line to a single value — sum numeric
arrays (fmt_num), concatenate non-numeric arrays via their string
values, empty array -> “”. Non-array lines pass through unchanged.
Over(f64)
Keep only lines that parse as a number strictly greater than N.
Lines that do not parse as f64 are dropped.
Under(f64)
Keep only numeric lines whose value is strictly less than N; drop non-numeric lines.
Between(f64, f64)
Keep numeric lines x where lo <= x <= hi (inclusive); non-numeric lines are dropped.
Enumerate
Prefix each line with its 1-based index and a tab: "1\t<line>".
Words
Split each line on whitespace and emit one word per line (flatten); empty lines produce nothing.
Dedup
Collapse runs of adjacent identical lines to a single line (classic uniq), leaving non-adjacent repeats intact.
Tailn(usize)
Keep only the last N lines (complement of take, which keeps the first N). N>=len keeps all.
Pad(usize)
Right-pad each line with spaces to a minimum visible width N (no truncation if the line is already longer).
Lpad(usize)
Left-pad each line with spaces to a minimum width of N (lines already >= N are unchanged).
Grepf(String, Regex)
Retain lines whose FIELD (json key or 1-based whitespace column) matches the regex.
Flip
Reverse the Unicode scalar characters of each line (chars().rev()).
Basename
Path basename: the part after the last / (the whole line if none).
Dirname
Path dirname: the part before the last / (. if none).
Commafy
Group a numeric line’s integer part with thousands separators
(1234567 -> 1,234,567); non-numeric lines pass through.
Bytes
Humanize a byte count (1024-based): 1536 -> 1.5 KB; non-numeric passes through.
Duration
Humanize a duration in seconds: 3661 -> 1h 1m; non-numeric passes through.
Apply(String)
Placeholder for apply .name: at render time it is replaced by the query
pipeline typed into the input .name widget (the megafilter/map binding).
Left in a pipeline unsubstituted it is a no-op.
JsonDocs
in.json: regroup the line stream into one item per JSON document — a
pretty-printed document spanning lines becomes one compact line, and
several values on one line become one each, as jq reads its input. A
line that is no part of a document stays the line it was. See
crate::jsondocs.
Csv
Treat the stream as CSV: the first line is the header; each data row
becomes a JSON object keyed by the header, so field NAME works.
B64
B64d
Base64-decode each line (STANDARD alphabet) into a UTF-8 string; lines that fail to base64-decode or whose bytes aren’t valid UTF-8 pass through unchanged.
Hex
Lowercase hex-encode each line, two hex digits per UTF-8 byte.
Unhex
Decode a hex string to UTF-8 text by parsing byte pairs; on any error (odd length, non-hex digit, invalid UTF-8) the line is left unchanged.
Urlenc
Percent-encode each line, escaping every non-alphanumeric byte (RFC 3986 style).
Urldec
Percent-decode each line to UTF-8 (utf8_percent_decode); lines whose decoded bytes are not valid UTF-8 pass through unchanged.
Extract(Regex)
Emit the first regex match per line (capture group 1 if the pattern captures, else the whole match); drop lines with no match.
Split(String)
Explode each line by the literal string DELIM into multiple lines (one part per line).
One-to-many: unlike cut (one field) or words (whitespace), every split segment becomes its own line.
Substr(usize, usize)
Character substring [A,B) 0-based, clamped to the line length (B may exceed len; A>B yields empty).
Chars
Explode each line into one output line per Unicode scalar (character); one line -> many.
Title
Title-case each line: uppercase the first letter of each whitespace-separated word, lowercase the rest, rejoin with single spaces.
Repeat(usize)
Replace each line with its content repeated N times, concatenated.
Set(String, String)
Set key K to the string value V (Value::String(V)) in each JSON object line; non-object / unparseable lines pass through unchanged.
Del(String)
Remove key K from each JSON object line (jq del(.K)); non-object lines pass through.
Rename(String, String)
Rename JSON object key OLD to NEW in each object, preserving the value; no-op if OLD absent. Non-object lines pass through.
Default(String, String)
Set string key K to V only when K is absent from the JSON object (jq //= for a missing key). Present keys keep their value; non-object / unparseable lines pass through unchanged. Key order is normalized on mutation.
Merge
Reduce all JSON object lines into a single object (later keys overwrite earlier); non-object lines are ignored. Emits one JSON object line, or none if no objects were seen.
Floor
Floor each numeric line to the nearest lower integer (fmt_num(x.floor())); non-numeric lines pass through unchanged.
Ceil
Round each numeric line up to the nearest integer (ceil); non-numeric lines pass through unchanged.
Clamp(f64, f64)
Clamp each numeric line into the inclusive range [LO, HI]; non-numeric lines pass through unchanged.
Tsv
Same as Csv but tab-separated (TSV).
Yaml
Parse the accumulated stream as a YAML document (or ----separated
multi-document) and emit each document as a JSON line, so the JSON verbs
(field/pick/keys/each) work over it (the yq leg).
Toml
Parse the accumulated stream as one TOML document and emit it as a JSON object line.
Calc(Expr)
Reduce to a scalar computed by an arithmetic expression over the current
line count (x), evaluated on the fusevm VM.
Contains(String)
keep lines containing a literal substring.
Starts(String)
keep lines starting with a literal prefix.
Ends(String)
keep lines ending with a literal suffix.
Nonempty
drop empty / whitespace-only lines.
Numeric
keep only lines that parse as a number.
Len
replace each line with its character count.
JsonLen
jq length (JSON-aware): array element count / object key count / string
char count / |number| / null=0; a non-JSON line falls back to char count.
JsonSlice(Option<i64>, Option<i64>)
jq slice .[a:b] over a JSON array line (a/b may be negative — counted
from the end — or omitted); a string line is char-sliced; other lines pass
through unchanged.
Wc
replace each line with its word count.
Abs
absolute value of each numeric line.
Round
round each numeric line to the nearest integer.
Delta
consecutive differences of the numeric series (n values → n-1 deltas) — turns a monotonic counter into a per-step rate-of-change.
Cumsum
running (cumulative) total of the numeric series.
Sma(usize)
simple moving average over a trailing window of N (length-preserving; the first points average a shorter, growing window). Smooths a noisy series.
Ewma(f64)
exponentially-weighted moving average, smoothing factor alpha in (0,1] —
higher alpha tracks faster, lower is smoother. s0 = x0.
Prepend(String)
prefix every line with a literal string.
Append(String)
suffix every line with a literal string.
Cut(String, usize)
split each line by DELIM, keep the Nth (1-based) field.
Median
median of numeric lines.
Percentile(f64)
Nth percentile (0–100) of the numeric values, linear interpolation between
closest ranks (numpy default). percentile 99 / p99 for latency tails.
Stddev
population standard deviation of numeric lines.
Range
max minus min of numeric lines.
Product
product of numeric lines.
Distinct
count of distinct lines.
Sample(usize)
keep every Nth line (1-based).
Slice(usize, usize)
keep lines from index A to B inclusive (1-based).
Index(usize)
keep only the Nth line (1-based); out-of-range yields no lines.
Bins(usize)
bucket numeric lines into N equal-width ranges -> (range, count) pairs.
Trait Implementations§
Auto Trait Implementations§
impl Forget for QueryOp
impl Freeze for QueryOp
impl Move for QueryOp
impl RefUnwindSafe for QueryOp
impl Send for QueryOp
impl Sync for QueryOp
impl Unpin for QueryOp
impl UnsafeUnpin for QueryOp
impl UnwindSafe for QueryOp
Blanket Implementations§
Source§impl<T> ArchivePointee for T
impl<T> ArchivePointee for T
Source§type ArchivedMetadata = ()
type ArchivedMetadata = ()
Source§fn pointer_metadata(
_: &<T as ArchivePointee>::ArchivedMetadata,
) -> <T as Pointee>::Metadata
fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<F, W, T, D> Deserialize<With<T, W>, D> for F
impl<F, W, T, D> Deserialize<With<T, W>, D> for F
Source§impl<T> Downcast for Twhere
T: Any,
impl<T> Downcast for Twhere
T: Any,
Source§fn into_any(self: Box<T>) -> Box<dyn Any>
fn into_any(self: Box<T>) -> Box<dyn Any>
Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>. Box<dyn Any> can
then be further downcast into Box<ConcreteType> where ConcreteType implements Trait.Source§fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
Rc<Trait> (where Trait: Downcast) to Rc<Any>. Rc<Any> can then be
further downcast into Rc<ConcreteType> where ConcreteType implements Trait.Source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
&Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &Any’s vtable from &Trait’s.Source§fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
&mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &mut Any’s vtable from &mut Trait’s.Source§impl<T> DowncastSync for T
impl<T> DowncastSync for T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more