libjay 0.1.0

Independent, modern implementations of the J and APL array languages: parallel and vectorized, embeddable from Rust, Python, and C
Documentation

libjay

Independent, embeddable implementations of the J and APL array languages, as one dependency-light Rust crate. Not a DataFrame library and not a framework: the relationship to your code is the one PCRE has — a small language inside a string literal, compiled once, run many times, with the heavy machinery hidden inside.

The Cargo package is libjay; the library is jay (so the linker artifact is not liblibjay), matching the C -ljay and the Python import jay.

cargo add libjay

Compile once, run many times

use jay::{compile, Array, Data, Dialect, Lang};

let program = compile(Lang::J, "(+/ % #) {x}", &Dialect::default())?;

let x = Array::new(vec![5], Data::F64(vec![3.0, 1.0, 4.0, 1.0, 5.0].into()));
let value = program.run(&[x], &mut |s| print!("{s}"))?;   // Some(2.8) — the mean

{name} holes become parameters. program.params reports them in the order run expects; arguments are positional. run returns Option<Array> — None when the last sentence yields no value (an assignment, or echo/⎕←). The closure is the output sink for echo and ⎕←; stdout is the sandbox default and no other I/O is open.

Dialect carries the host's settings — today APL's ⎕IO (Dialect { index_origin: Some(0) }); J's index origin is 0 and is not configurable. Lang::Apl selects the other frontend, with its own semantics: J reduces along the leading axis, APL along the trailing one.

Errors carry a span into the source. Error::render(source) renders a compile error, Program::render_error a run error, both the way the CLI would:

length error: arguments do not agree: left shape 3, right shape 2
  1 2 3 + 1 2
  ^^^^^^^^^^^
note: frames first differ at axis 0: 3 vs 2

jay::fmt::format_array(&value, &program.fmt) renders a result the way its language displays it.

What is inside

Both frontends lower to one language-agnostic IR: an Expr tree over a Verb combinator tree (Prim / Rank / Reduce / Fork / Hook / Atop / Windowed). One generic rank-and-agreement engine executes everything, so APL's +/ is simply Rank(Reduce(+), 1) and no J assumption reaches the runtime. A compile-time pass fuses chains of elementwise primitives into one blockwise kernel, absorbing a trailing full-rank reduction; anything it will not fuse falls back to the subtree it replaced, so results and error messages cannot change. Dense arrays of bool / i64 / f64 / characters, row-major.

Threads

Execution is parallel by default — part of what a compiled expression is, not a switch the caller flips. Elementwise passes, pure rank cells and leading-axis reductions split above 65,536 element operations.

A Program is immutable, holds no data, and is Send + Sync: share one across threads, or wrap it in an Arc and run it concurrently. The pool is the crate's own, not rayon's global one, so an embedding host that also uses rayon keeps its pool intact; work started inside a rayon worker stays in that worker's pool. LIBJAY_THREADS sets the size (read once, at first use), otherwise it is the machine's available parallelism.

An associative float reduction may be regrouped across chunks, which reorders the rounding; everything else is bit-identical to the sequential path.

Devices

Where a program runs is separate from what it is bound to, and the CPU is one of the answers:

let device = jay::Device::default_gpu();          // None where there is none
let value = match &device {
    Some(d) => program.run_on(d, &args, &mut sink)?,
    None => program.run(&args, &mut sink)?,
};

jay::device::available() lists the adapters. What reaches a GPU is the fused elementwise kernels, generated as WGSL at run time and dispatched through wgpu (Metal, Vulkan, DX12); everything else, and any chain the device will not take, runs on the CPU, and Program::explain_on names each placement and its reason. The backend is in every build and dormant without an adapter — there is no feature flag and no second artifact.

libjay computes floats in f64 and will not lose that quietly: on an adapter without SHADER_F64 — Metal has none — an f64 chain stays on the CPU unless the caller asks for Precision::F32. Device::upload returns an array that carries its own device allocation, so repeated runs over it upload nothing.

More

MIT licensed.