cljrs-value
Core runtime values and persistent collections for clojurust.
Phase: 3 (collections/Value) + 4 (CljxFn, Namespace) + 5 (LazySeq, CljxCons) + 6 (Protocol, ProtocolFn, MultiFn) + 7 (Volatile, Delay, CljxPromise, CljxFuture, Agent) + 6-ext (TypeInstance for defrecord/reify) + B2 (structured-clone boundary) + B3 (shared static arena: intern tables, SharedValue, SharedAtom, ByteBlob) — implemented.
Purpose
Defines Value, the single enum that represents every Clojure runtime value,
plus all persistent (immutable, structurally shared) collection types. The
cljrs-runtime crate's interp and tiered modules operate on Values; its
builtins module and cljrs-stdlib build the standard library on top of them.
File layout
src/
lib.rs — module declarations and re-exports
clone.rs — SerializedValue (Send+Sync wire form), CloneError, serialize/deserialize for cross-isolate copy boundary (Phase B2); SerializedValue::byte_size for boundary metering; SerializedValue::wire_size for the governor's message-queue charge; handles SharedAtom/ByteBlob/Var pass-through (B3 — Var shares its root cell, issue #171)
error.rs — ValueError enum (incl. `MemoryLimitExceeded(cljrs_gc::MemoryLimitExceeded)`, uncatchable like `GasExhausted`), ValueResult<T> alias
hash.rs — ClojureHash trait, Murmur3 helpers, JVM-compatible hash_string
intern.rs — (Phase B3) global keyword/symbol intern tables backed by StaticGcPtr; intern_keyword, intern_symbol
jit_hooks.rs — (Phases 10.2/10.5) var-rebind hooks fired by Var::bind; set_var_rebind_hook registers multiple consumers (the compiler JIT stales superseded native code; cljrs-runtime invalidates cross-defn-specialized lowerings)
keyword.rs — Keyword { namespace, name }
publish.rs — (Phase 10.5, GC builds; identity stub under no-gc) heap-promotion publish barrier: publish_value(Value) -> Value scans for region-allocated boxes, deep-copies them to the GC heap (via clone.rs), or poisons the active regions when the value is opaque to the scan. Called by Var::bind, Atom::new/reset, Volatile::new/reset, CljxPromise::deliver, and cljrs-async channel puts
shared.rs — (Phase B3) SharedValue enum, SharedAtom (Arc<ArcSwap<SharedSlot>>), ByteBlob and SharedSlot (each owns one governor MemoryCharge), promote/demote; PromoteError. Var roots reuse SharedValue via Var::shared_root (issue #171)
regex.rs — Pattern (engine-agnostic compiled regex behind Value::Pattern, selected by the regex-full / small-regex features), PatternError, Captures, Matcher (stateful re-find/re-matches driver), MatchPhase, MatchResult
symbol.rs — Symbol { namespace, name }
type_hint.rs — TypeHint enum (^long/^double/^longs/… primitive type tags) + from_tag/is_array/element
native_object.rs — NativeObject trait, NativeObjectBox wrapper, gc_native_object helper (Phase 9 interop)
types.rs — Var, Atom, Namespace, NativeFn, CljxFn, Thunk, LazySeq, CljxCons, Protocol, ProtocolFn, ProtocolMethod, MultiFn, Volatile, Delay, CljxPromise, CljxFuture, Agent
value.rs — Value enum (incl. SharedAtom, ByteBlob variants), MapValue, SetValue, TypeInstance, pr_str, PartialEq, ClojureHash, std::hash::Hash
collections/
mod.rs — re-exports all collection types
array_map.rs — PersistentArrayMap (≤8 entries, linear scan)
hash_map.rs — PersistentHashMap (32-way HAMT keyed by insertion-sequence index + a red-black tree ordering log, so iteration order matches insertion order deterministically)
hash_set.rs — PersistentHashSet (same index+ordering-log technique as PersistentHashMap)
list.rs — PersistentList (singly-linked cons list)
queue.rs — PersistentQueue (front-list + rear-vector)
share_charge.rs — (crate-private) ShareCharge: per-version gc_size_extra charge for structurally shared collections (full size, or path-copied nodes when derived from a GC-allocated version); vector/HAMT/red-black path-size estimates
vector.rs — PersistentVector (32-way trie + tail buffer)
hamt/
mod.rs — re-exports Node and bitmap helpers
bitmap.rs — BITS, WIDTH, fragment, sparse_index, bit_for
node.rs — Node<V> enum (Leaf, Branch, Collision); HAMT trie operations
Public API
Value
;
PartialEq implements cross-type numeric equality ((= 1 1N), (= 1 1.0))
and sequential collection equality between List and Vector.
Display / pr_str produce Clojure-readable output.
Symbol / Keyword
Both support simple(name), qualified(ns, name), parse(str), and
full_name() -> String. Symbol additionally carries an optional git-commit
version (the @<hash> suffix) with versioned_name() -> String. Free
helpers used by all execution tiers to detect versioned names:
symbol::is_commit_hash(s) -> bool and
symbol::split_version(name) -> (&str, Option<&str>).
regex — Pattern / Captures / Matcher
Value::Pattern holds a GcPtr<Pattern>, never an engine type directly.
Pattern is the only place in the workspace that names a regex engine, so the
engine is a build-time choice (see Features):
;
// the pattern source
// leaf: no GcPtr fields
/// Engine-independent compile failure; carries the engine's message.
;
/// A successful match, borrowing the haystack.
;
Matcher is the stateful driver behind re-find/re-matches/re-seq: it
holds pattern: GcPtr<Pattern> plus a haystack and walks matches left to
right, one per next() call. With match_all set (re-matches) the search is
anchored to the whole haystack — Java's Matcher.matches() semantics — and
yields at most one match: the next next() is Complete either way.
/// Matching's payload is the byte offset the next search resumes from: the end
/// of the match, bumped one character past a zero-width one (Java `find`), so
/// `#"a*"` terminates instead of re-finding the empty match forever.
/// Owned form of a match — outlives the haystack borrow.
Phase B3 — Shared static arena
Intern tables (intern module)
;
;
Global OnceLock<Mutex<HashMap<…>>> tables. First call allocates the
Keyword/Symbol into program-lifetime memory via static_alloc; subsequent
calls return a clone of the same StaticGcPtr (pointer-stable identity across
all isolates).
SharedValue and SharedAtom (shared module)
/// Immutable shared byte buffer. Owns one governor charge
/// (`MemoryClass::SharedValue`) for its bytes, however many `Arc`s exist.
/// `memory` is set when the governor refused the shared allocation.
;
;
promote converts an isolate-local Value to SharedValue (fails for
closures, resources, atoms, …). demote converts back into a fresh
isolate-local Value. compare_and_set is the single lock-free CAS that
backs the Clojure-level compare-and-set! and the swap! retry loop (callers
that must run interpreter code between load and store use it instead of the
closure-based swap). The try_* forms reserve the new slot before they
publish it, so a strict-mode refusal leaves the atom's value unchanged; the
runtime's shared-atom, reset!, swap!, and compare-and-set! use them.
Promoting a byte array reserves its blob before copying the bytes.
Var roots — two-tier, promote-on-def (issue #171)
A var's root binding uses the same cross-isolate mechanism as
shared-atom. Var carries two slots:
- Reads stay local. Every var deref, the IR tier, and the JIT/AOT
rt_*ABI readvalue— promotion never touches it, so inline caches and pointer-identity assumptions in compiled code remain valid (no JIT regression). bindpromotes-on-write.def/alter-var-root/set!all funnel throughVar::bind, which mirrors the new root intoshared_rootwhen it is promotable, clears it toNoneotherwise, and then advancesbinding_generation. Dependent caches can therefore track one Var without process-global invalidation.defis rare, so this write-path cost is acceptable.- Crossing isolates.
clone::serializepasses theshared_rootArcthrough (both isolates share the same cell); the receiver rebuilds the var withfrom_shared_root, seeding its local slot from the demoted snapshot. A var bound to a non-promotable root (closure / native resource) is explicitly isolate-local (ADR option (b)):serializereturnsCloneError::NotShareable { type_name: "var" }— a non-silent boundary error. Var-root watches stay isolate-local (the shared cell carries no watch callbacks), matchingshared-atom.
ClojureHash
Implemented for Value using Murmur3 + JVM String.hashCode semantics.
Whole-number doubles hash like their Long equivalent.
Collections
| Type | Description | Key operations |
|---|---|---|
PersistentList |
Singly-linked cons list | cons, first, rest, count (O(1)) |
PersistentVector |
32-way trie + tail buffer | conj, nth, assoc_nth, pop, iter, map_entry, is_map_entry |
PersistentArrayMap |
Flat key/value vec, ≤8 entries | assoc (returns AssocResult), get, dissoc, iter |
PersistentHashMap |
32-way HAMT index (key→seq) + red-black ordering log (seq→(key,val)); iterates in insertion order | assoc, get, dissoc, merge, iter, keys, vals |
PersistentHashSet |
Same index+ordering-log technique; iterates in insertion order | conj, disj, contains, iter |
PersistentQueue |
Front-list + rear-vector | enqueue, dequeue, peek |
PersistentArrayMap::assoc returns AssocResult::Array(Self) while under the
threshold, or AssocResult::Promote(Vec<(Value, Value)>) when the map is full.
MapValue::assoc handles the transparent promotion to PersistentHashMap.
Keyword-argument rest normalization
from_kwargs_rest ;
Clojure's seq-to-map-for-destructuring: the value a map-shaped variadic
rest pattern ([& {:keys [a b]}]) destructures against, given the trailing
arguments as a flat vector. Zero or one argument returns that argument
verbatim — so (f {:a 1}) binds the caller's own map (sortedness and metadata
intact) and (f) / (f nil) bind nil; two or more are alternating
key/value pairs, optionally closed by a trailing map whose entries merge over
and beat them. A dangling key is a ValueError, not a dropped argument.
This lives here rather than in a tier because every tier has to agree on it:
the tree-walker calls it from bind_fn_params, and the IR interpreter and
compiled code reach it through KnownFn::KwargsMap / rt_kwargs_map. Issue
#368 was those paths disagreeing.
All collections implement PartialEq, Debug, Clone, and cljrs_gc::Trace.
PersistentList, PersistentVector, and PersistentHashSet implement
std::iter::FromIterator<Value>.
A PersistentVector may be tagged as a map entry — the [key val] pairs
produced by seq'ing a map, find, or the map-entry builtin.
PersistentVector::map_entry(key, val) builds one; is_map_entry() reads the
tag (there is also a Value::map_entry(k, v) / Value::is_map_entry()
convenience pair). The tag is invisible to equality, hashing, and printing —
(= (first {:a 1}) [:a 1]) still holds — and it exists only so map-entry?
can distinguish real entries from plain 2-element vectors. As in Clojure, any
derived vector (conj, assoc_nth, pop, from_iter, ...) is a plain
vector again.
Transient collections
TransientVector, TransientMap, and TransientSet provide the mutable
construction phase used by transient, the bang operations, and
persistent!. Their read methods support the same indexed/associative lookup
interface as their persistent counterparts while the transient is active.
All three implement Default, Clone, ClojureHash, and Trace.
All collection Trace impls also override gc_size_extra to report the heap
bytes owned by each collection beyond the GcBox struct. The table gives the
full charge. PersistentVector, PersistentHashMap, PersistentHashSet,
SortedMap, and SortedSet instead charge only their path-copied nodes —
O(log n), see vector_path_bytes, hamt_path_bytes, and rb_path_bytes in
share_charge.rs — when the version was derived by one update (conj,
assoc, assoc_nth, dissoc, disj, pop, or a no-op of these) from a
version already allocated behind a GcPtr, because that parent's box already
accounts for the shared nodes. They charge the full amount when:
- built wholesale (
empty,from_iter,from_pairs,from_vector,map_entry, an in-placeconj_mut) or cloned; - derived from a version that was never GC-allocated (a Rust-side builder loop), since the result then owns every node;
- the count reaches twice that of the nearest fully charged ancestor, so a
chain of
nupdates charges O(n) element bytes in total plus O(log n) per step, and a collection cannot grow past twice the size last admitted in full.
When a fully charged ancestor is collected, a surviving derived version is under-counted by less than twice that ancestor's full charge. Each box still returns exactly its own stored header size when swept. Approximations used:
| Type | Formula |
|---|---|
PersistentArrayMap |
16 + capacity × size_of::<Value>() |
PersistentHashMap |
n × (88 + 2×size_of::<Value>()) |
PersistentHashSet |
n × (88 + size_of::<Value>()) |
PersistentVector |
n × (24 + size_of::<Value>()) |
SortedMap |
n × (40 + 2×size_of::<Value>()) |
SortedSet |
n × (40 + size_of::<Value>()) |
TransientMap/Set |
full HashMap/Set charge of the inner map (locked at alloc) |
TransientVector |
same as Vector (locked at alloc) |
ObjectArray |
capacity × size_of::<Value>() |
| Primitive arrays | capacity × size_of::<T>() |
BoundFn |
capacity × (1 + size_of::<usize>() + size_of::<Value>()) |
ExceptionInfo |
message.capacity() |
The 40-byte per-entry overhead for HAMT/RBTree is: 16 bytes Arc ref-counts +
16 bytes EntryWithHash/left-right pointers + 8 bytes tree-node sharing. The
24-byte overhead for trie vector elements is: 16 bytes Arc overhead + 8 bytes
thin pointer in the leaf-node Vec.
CljxFn / CljxFnArity (Phase 4)
// Requires cljrs-reader (for Vec<Form> body).
is_async is set by the interpreter when a fn/defn carries ^:async (or an
{:async true} attr-map). CljxFn::new defaults it to false;
cljrs_runtime::env's dispatch_if_async checks it at call time.
self_ptr is set immediately after GcPtr::new(cljrs_fn) in eval_fn (for
named anonymous functions) so that the self-reference returned from the function
body is the same GcPtr as the outer binding, preserving pointer-equality
semantics ((= f (f)) → true). CljxFn::new defaults it to None.
Var-rebind hooks (jit_hooks, Phases 10.2/10.5)
/// Register a rebind hook. Multiple hooks may be registered; each is called
/// with (old_value, new_value) in registration order.
;
Var::bind invokes every registered hook (via notify_var_rebind) whenever
it overwrites an existing binding. Two consumers exist: the JIT stales and
reclaims native code compiled for the superseded definition (10.2), and
cljrs_runtime::tiered's defn registry invalidates lowerings of other functions that
specialized against it (10.5). When no hook is registered the cost is a
single atomic flag load.
Heap-promotion publish barrier (publish, Phase 10.5 — GC builds)
/// Prepare a value for publication into a program-lifetime cell (or another
/// thread): returns the value to store — the original when no region-
/// allocated box is reachable, or a heap deep-copy when one is. Values
/// opaque to the scan (closures, unrealized lazy seqs, native objects)
/// poison the thread's active regions instead
/// (cljrs_gc::region::poison_active_regions), retiring them at scope close.
/// One thread-local depth check when no region is open.
;
The runtime safety net for bump regions coexisting with the tracing GC:
correctness never depends on escape analysis being perfect. Invoked by
Var::bind, Atom::new/Atom::reset, Volatile::new/Volatile::reset,
CljxPromise::deliver, and cljrs-async's channel puts. Under no-gc the
module is an identity stub (that build keeps its StaticCtxGuard discipline).
Namespace (Phase 4)
ReferClojureFilter (Phase 4)
How much of clojure.core a namespace auto-refers, as narrowed by an
(:refer-clojure ...) clause in ns. None on the namespace means the
default: every public core name is referred. Applied by
GlobalEnv::refer_core, and only there — an explicit refer (refer_all /
refer_named) bypasses it.
A renamed name is not also referred under its original name, matching
clojure.core/refer. local_name is a pure lookup: validating the filter
against what core actually defines, and rejecting two names that collide on
one local name, happens once in GlobalEnv::set_refer_clojure_filter.
Thunk / LazySeq / CljxCons (Phase 5)
Thunk implementations live in cljrs-runtime (e.g. ClosureThunk) so that
cljrs-value stays free of evaluator dependencies while LazySeq can still
call back through the trait object.
TypeInstance (Phase 6-ext — defrecord/deftype/reify)
Used by defrecord (named type_tag, generates ->Name/map->Name constructors),
deftype (named type_tag, ->Name only), and reify (gensym'd type_tag, no
constructors). Supports keyword field access (:field rec), get, assoc
(returns new TypeInstance), and count.
mutable holds a deftype's ^:unsynchronized-mutable / ^:volatile-mutable
fields in one interior-mutable cell — an Atom over a keyword→value map — so
set! updates them in place and every clone of the instance sees the write.
It is None for defrecord, reify, and an all-immutable deftype.
Instances are !Send, so one shared cell needs no stronger volatility than an
Atom, and the two mutability markers behave identically. assoc/assoc-in
and with-meta carry the cell through; serialize folds the slot values
into the field map and deserialize restores mutable: None, since mutability
itself is runtime state that does not cross a clone boundary.
Volatile / Delay / CljxPromise / CljxFuture / Agent (Phase 7)
// Pending(Box<dyn Thunk>) | Forced(Value)
/// Message carried by `CljxFuture::cancelled_error()`.
pub const FUTURE_CANCELLED_MSG: &str = "future was cancelled";
pub type AgentFn = ;
Protocol / ProtocolFn / MultiFn (Phase 6)
/// Phase 10.6 — protocol-dispatch inline-cache invalidation.
/// `bump_protocol_generation()` must follow every mutation of any
/// `Protocol::impls` map (the `extend` builtin / `deftype*` inline impls);
/// `rt_call_ic` (cljrs-compiler) tags each cached dispatch with the
/// generation observed at fill time and re-resolves on mismatch.
;
;
Each method-cache entry is tagged with Var::binding_generation() for the
specific hierarchy root and MultiFn::method_generation() for the specific
multimethod. A concurrent mutation can leave an entry under an old tag, but
the next lookup clears it before use.
When Protocol::extend_via_metadata is set, apply_value's ProtocolFn arm
(cljrs-runtime/src/env/apply.rs) checks the dispatch value's metadata for an entry
keyed by the exact ProtocolFn before falling back to the impls type-tag
lookup — see that crate's README for the dispatch order.
clone — isolate copy boundary (Phase B2)
/// A Send + Sync intermediate representation for cross-isolate transfer.
/// All heap data is owned (no GcPtr); safe to move across thread boundaries.
/// Reason a value cannot cross an isolate boundary.
/// Convert a Value to SerializedValue. Returns CloneError for mutable state,
/// closures, native resources, and other non-shareable types.
;
/// Allocate a fresh Value in the *current* GC heap from a SerializedValue.
/// Infallible — non-shareable types are rejected at serialize time.
;
Shareable types: all scalars, strings, BigInt/BigDecimal/Ratio, Symbol/Keyword, all persistent collections, TypeInstance records, Error chains, primitive and object arrays, lazy sequences (realized first), WithMeta/Reduced wrappers.
Cross-isolate shared references (Arc passed through, not deep-copied): SharedAtom,
ByteBlob, and Var — a var crosses by sharing its shared_root cell, so a value
def'd in one isolate is observable by value in another (issue #171). A var
whose current root is non-promotable (closure/resource) is the exception and
returns CloneError.
Non-shareable (returns CloneError): Atom, Volatile, Promise, Future, Agent
(mutable state); Fn, BoundFn, NativeFn, Macro, ProtocolFn, MultiFn (closures
with isolate-local captures); Namespace, Protocol (global singletons); Resource,
NativeObject (isolate-bound handles); TransientMap/Set/Vector; unforced Delay;
Matcher; Var whose root holds a non-promotable value.
Dependencies
cljrs-value depends on cljrs-reader so that CljxFnArity::body can store
Vec<Form> (unevaluated source bodies for interpreter evaluation and closure
capture).
Features
| Feature | Default | Effect |
|---|---|---|
regex-full |
yes | Value::Pattern uses the regex crate: linear-time matching, full Unicode character classes. |
small-regex |
no | Value::Pattern uses regex-lite instead — roughly a tenth of the code size; regex-automata, regex-syntax and aho-corasick drop out of the build. |
no-gc |
no | Forwards cljrs-gc/no-gc (region allocation, no tracing GC). |
Exactly one regex engine is compiled in, and regex-full wins when both
features are enabled. Cargo features are additive, so that ordering is
deliberate: a build where one dependent asks for small-regex and another takes
the default gets the more capable engine rather than a silent semantic
downgrade.
Measured on a stripped release build of the interpreter plus
clojure.core/clojure.string with deps off (cljrs-stdlib with
default-features = false), swapping the engine takes .text from 3.68 MB to
2.89 MB and the binary from 5.77 MB to 4.10 MB.
small-regex is a behaviour change as well as a size one — regex-lite has
no Unicode character classes (\w, \d, [[:alpha:]] and friends are ASCII
only) and is materially slower on pathological patterns. re-find/re-matches/
re-seq over short strings, which is what most Clojure code does with regexes,
are unaffected in behaviour.
Because regex-full wins ties, selecting the small engine means turning default
features off on every workspace crate you depend on directly. Cargo unions
features across all edges to a package, so an edge left at its defaults re-enables
regex-full for the whole graph — and a second, direct dependency declaration
cannot switch it off again. Every workspace crate therefore takes its own internal
dependencies with default features off and re-exports both features, so one
default-features = false at your edge is enough:
[]
# interpreter + clojure.core/clojure.string, no git dependency support
= { = "0.1", = false, = ["small-regex"] }
One crate left at its defaults anywhere in the graph puts regex back — the same
unification caveat that applies to cljrs-runtime's deps feature. And deps
itself has to be off for the swap to pay: cljrs-project/vcs pulls regex in
through pgp, so an embedding that resolves git-hosted dependencies links the
full engine no matter what this feature says. Every crate that re-exports the regex pair also
re-exports deps for that reason.
The one crate where the swap cannot pay is the CLI: cljrs depends on
cljrs-project with vcs/vcs-net/ssh unconditionally, so pgp — and with it
regex — is always in that binary.