pub struct CompiledTrace {Show 26 fields
pub head_pc: u32,
pub entry: unsafe extern "C" fn(*mut i64) -> i64,
pub n_ops: u32,
pub dispatchable: bool,
pub window_size: u32,
pub exit_tags: Rc<[ExitTag]>,
pub global_tag_res_kind: TagResKind,
pub entry_tags: Rc<[u8]>,
pub per_exit_tags: Rc<[(u32, Rc<[ExitTag]>)]>,
pub per_exit_inline: Rc<[InlineSideExit]>,
pub exit_hit_counts: Rc<[TCellU32]>,
pub exit_side_trace_ptrs: Rc<[TCellPtr]>,
pub tags_side_trace_ptrs: Rc<[Box<TCellPtr>]>,
pub global_side_trace_ptr: Box<TCellPtr>,
pub side_trace_cache: TRefLock<HashMap<u32, u32>>,
pub has_any_side_wired: TCellBool,
pub is_inline_abort_close: bool,
pub dispatch_off_reason: Option<&'static str>,
pub sinkable_sites_seen: u32,
pub accum_bufferable_seen: u32,
pub sunk_alloc_seen: u32,
pub materialize_emit_count: u32,
pub closure_seen: u32,
pub body_writes: Box<[u32]>,
pub downrec_link: Option<(u32, u32)>,
pub downrec_multi_way_count: u8,
}Expand description
A trace compiled by S2’s lowerer and ready to be dispatched into
at its head PC. Owned by Proto.traces; the underlying mmap is
kept alive by the Vm.jit_handles Vec for the Vm’s lifetime,
just like the method JIT’s compiled functions.
Fields§
§head_pc: u32Pc the trace dispatches at (matches the recorder’s head_pc).
entry: unsafe extern "C" fn(*mut i64) -> i64Native entry function (mmap’d machine code, valid for the Vm’s lifetime as long as the backing handle is kept).
n_ops: u32Number of ops in the source TraceRecord. Diagnostic only;
tuning will gate re-record vs. recompile based on this.
dispatchable: booltrue iff the dispatcher can safely invoke this trace.
False when the trace has ops the lowerer can’t predict the
exit type for (today: Op::GetI — the helper returns a
raw payload that may be Int or Table or Float; without
runtime tag info the dispatcher can’t repack the slot).
Non-dispatchable traces still compile and stay cached so
a future dispatcher with richer marshalling can pick them
up — they just don’t run today.
window_size: u32P12-S4-step3a — size of the reg_state buffer the dispatcher
must allocate when calling entry. Today always equals
head_proto.max_stack (the trace covers only the head
frame). S4-step3b’s inline emit pushes this past max_stack
to fit additional inlined frames whose register windows sit
at offsets[i]..offsets[i] + max_stack within the buffer.
The dispatcher’s marshal-in still only writes [0..max_stack)
— depth>0 slots start initialized to zero, and the trace’s
own GetUpval / arith fills them as it runs.
Per-register exit tag of length window_size. Indexed by
position within the trace’s reg_state_buf. The dispatcher
consults this to pack reg_state[i] back into a Value
after the trace returns at the clean tail (head_pc or
call-truncation pc). See ExitTag for the semantics.
Rc<[]> so the dispatcher’s per-dispatch lookup is a cheap
refcount bump, not a Vec heap clone (fib_28 dispatches 1M×
— clone cost dominates without this).
global_tag_res_kind: TagResKindP13-S13-E — classification of the global exit_tags for
the dispatcher’s restore-loop fast path. The dispatcher
dispatches on this when site_id == 0 AND
per_exit_tags.find(cont_pc) misses (the common
back-edge / clean-tail exit shape):
AllUntouched→ skip the restore loop entirely (trace touched no slots; vm.stack already holds the right values from entry, possibly modified by spill helpers)AllInt→vm.stack[base+i] = Value::Int(reg_state[i])per slot, no per-iter matchMixed→ original match-arm loop
P12-S12-C v3 — compile-time snapshot of entry_tags from the
TraceRecord. The trace’s IR + current_kinds propagation
are specialised to these tags; if the runtime entry tags
differ, the dispatcher must skip dispatch (fall back to
interp) — otherwise the trace would treat e.g. a Str ptr
slot as Int and produce garbage. Rc<[]> to match the
other tag arrays’ cheap-clone idiom.
P12-S4-step2c — per side-exit exit_tags. Each entry is
(continuation_pc, exit_tags); when the trace returns a PC
matching an entry, the dispatcher uses that vector instead of
the clean-tail exit_tags. This makes side-exits that fire
before later writers (GetUpval is the today motivator)
restore the affected slot as Untouched (carry entry tag)
rather than pack with a tag the slot hasn’t actually become.
Empty when no side-exit needs a different vector than the
clean tail (e.g. plain numeric loops with no GetUpval).
per_exit_inline: Rc<[InlineSideExit]>P12-S4-step4b-C-2 — per inline side-exit metadata, indexed by
site_idx. Each entry carries the side-exit’s cont_pc,
the per-slot exit_tags snapshot (sized to window_size so
every materialised frame’s window is restored), and the
frame-materialise chain to push.
fib has SIBLING self-recursive Calls (pc7, pc11) and EVERY
depth’s cmp lands at the same cont_pc — keying the lookup
by cont_pc alone (the v2 attempt) collapsed all those
distinct chains onto one entry. The trace IR encodes the
firing site’s (site_idx + 1) in the upper 32 bits of the
returned i64 so the dispatcher disambiguates O(1).
Empty when no cmp@d>0 fires in the trace. The IR pre-bakes
each chain’s raw pointer (Rc::as_ptr) at compile time;
the Rc clones in this field keep the slice alive for the
trace’s mmap lifetime (Proto.traces owns the CompiledTrace).
exit_hit_counts: Rc<[TCellU32]>P15-prep — per-exit hit counter (LuaJIT-study foundation for future side trace work). Length and layout:
[0..per_exit_inline.len()): parallel to per_exit_inline (indexed bysite_id - 1in the dispatcher).[per_exit_inline.len()..per_exit_inline.len()+per_exit_tags.len()): parallel to per_exit_tags (indexed by find-by-cont_pc order).- Last slot: global / clean-tail exit (when site_id == 0 AND per_exit_tags.find misses).
Rc<[Cell<u32>]> so the dispatcher can increment without a
mutable borrow on the CompiledTrace. Vm’s
trace_exit_hit_distribution() aggregates this for probe use.
exit_side_trace_ptrs: Rc<[TCellPtr]>P15-A v2-A — per-exit raw side-trace function pointer. Same
length / layout as Self::exit_hit_counts. null means
“no side trace compiled for this exit yet”; non-null means a
child side trace’s entry fn lives at this pointer and v2-B/C
will wire the parent’s IR at each exit site to read this Cell
and indirect-call when non-null.
Cell<*const u8> (not Atomic) since the Vm is single-
threaded — see RFC Q2. The pointer’s stability is owned by
the child side trace’s TraceHandle in TRACE_JIT_HANDLES
(thread-local Vec), which persists for the thread lifetime.
Send/Sync invariant: Cell<*const u8> is not Sync, but
CompiledTrace was never required to be Sync (it lives in
Proto.traces: RefCell<Vec<CompiledTrace>> on the runtime
path). Adding this field doesn’t tighten that.
P15-A v2-C-A2 — per-per_exit_tags-entry side-trace cell.
Same length as per_exit_tags; the IR at the corresponding
emit_store_back_and_return_pc callsite (immediately after
per_exit_kinds.push) bakes this cell’s heap address. Same
semantics as InlineSideExit::side_trace_ptr but with
kind = SIDE_SENT_KIND_TAG and local = tag_idx.
global_side_trace_ptr: Box<TCellPtr>P15-A v2-C-A2 — singleton cell shared by every GLOBAL-kind
callsite (clean-tail return, Call truncation, ForLoop /
TForLoop exits, generic err deopts, etc.). All such sites’
IR bakes the same heap address; the close handler writes
the child entry ptr here for parent_exit_idx == per_exit_inline.len() + per_exit_tags.len() (the
exit_hit_counts layout’s last slot).
side_trace_cache: TRefLock<HashMap<u32, u32>>P15-A v2-C-A1 — when a child side trace compiles for any
of this trace’s hot exits, the close handler inserts
(child.head_pc, child_traces_idx) here. v2-C-A3’s
dispatcher uses this for an O(1) lookup of the side trace’s
own CompiledTrace when the sentinel bit on raw_ret
(introduced by v2-C-A2) flags a side-trace return — so
decode_exit_shape can be called with the SIDE TRACE’s
per_exit_inline / per_exit_tags / exit_tags instead
of the parent’s.
Value is an index into head_proto.traces (the same
proto this CompiledTrace lives in — trace JIT only fires
side traces from self-recursive parents today, so child +
parent share head_proto). Storing an index instead of a
raw pointer dodges the Vec<CompiledTrace> realloc-
invalidation pitfall: proto.traces.push doesn’t reorder,
only appends, so an index assigned at compile time stays
valid for the trace’s lifetime.
RefCell<HashMap<u32, u32>> because the close handler
holds only &CompiledTrace (the parent’s traces borrow is
immutable while we’re walking it to find the parent_ct).
has_any_side_wired: TCellBoolP15-A v2-D-A8 — fast-path short-circuit hint for the
dispatcher’s tentative-decode + cell-load + check path. Set
to true by the close handler when ANY of this trace’s
exit_side_trace_ptrs cells gets wired (i.e., the first
time a child side trace compiles + the A5-C shape gate
passes). Stays true for the trace’s lifetime — once any
side trace exists, the dispatcher must perform the per-
exit check on every dispatch.
When false, the dispatcher skips the tentative decode +
cell load + child lookup entirely, falling straight through
to the cheap parent decode + writeback. Trims fib_10_x10k-
class tight-trace workloads’ per-dispatch overhead from the
double-decode pattern to a single Cell::get().
Cell<bool> so the close handler can flip the flag through
only an &CompiledTrace borrow (the parent’s traces
borrow is immutable while the close handler walks).
is_inline_abort_close: boolP13-S13-G v2 — true iff this trace closes at a
TraceEnd::InlineAbort (depth>0 op the lowerer can’t
continue past: depth past MAX_INLINE_DEPTH, non-self
Call@d>0, ForLoop@d>0, TForLoop@d>0, or proto mismatch).
Such traces compile but pin dispatchable=false —
dispatching them would resume interp at a depth>0 PC
without the matching CallFrames the trace inlined past
(S4-step4b’s frame mat helper can synthesise these but
isn’t wired up for InlineAbort exits yet — that’s the
S13-G v2 follow-up). Vm’s trace_inline_abort_count
tallies these so future-tuning sees what bench cells
would benefit from the frame-mat unlock.
dispatch_off_reason: Option<&'static str>P13-S13-G v2.5 — if dispatchable == false, the static
label of the emit-pass site that flipped it. Lets a probe
distinguish among the six places trace.rs pins dispatch
off (GetI / GetTable / GetUpval inference fail, TForCall
slow-path, length gate, InlineAbort gate). None if the
trace IS dispatchable, the first label otherwise.
sinkable_sites_seen: u32P12-S5-A — number of NewTable sites in this trace whose
final EscapeState is EscapeState::Sinkable after
S5-B’s pre-emit demotion pass. Vm’s
trace_sinkable_seen_count tallies these for telemetry.
accum_bufferable_seen: u32P14-S14-B v1 — number of AccumSites with BufferState::Bufferable
detected by detect_accumulators. v1 only counts; v2+ will use
the sites for buffered emit. Vm’s trace_accum_bufferable_seen_count
tallies these for probe visibility.
sunk_alloc_seen: u32P12-S5-B — number of Sinkable sites this trace’s emit
actually allocated virt slot Variables for (i.e., took the
no-heap-alloc path). Always <= sinkable_sites_seen. Bumps
Vm::trace_sunk_alloc_count on compile success.
materialize_emit_count: u32P12-S5-C — number of (site × cmp side-exit) pairs in this trace’s IR that emit the materialise helper. Each pair is “this cmp’s side-exit reconstructs site X’s heap Table”. Static count; the runtime number of helper calls depends on dispatch shape (which side-exits actually fire).
closure_seen: u32P12-S7-A — number of Op::Closure ops this trace’s emit
lowered to a luna_jit_op_closure helper call. Each
closure-creating op replaces a Heap::new_closure_inline
allocation, which dwarfs the dispatcher’s marshal overhead;
the length-gate skip below treats closure_seen > 0 the
same as sunk_alloc_seen > 0 (don’t gate short traces).
body_writes: Box<[u32]>P15-A v2-E — sorted unique list of slot indices that ANY
op in this trace’s body WRITES (post inline_depth offset).
Computed at compile via compute_body_writes; consumed
by the v2-E smart side-trace gate at child compile to
detect read-before-write live-in registers that would
re-read the parent’s stale exit value across the child’s
internal-loop iters (see s12_step_b bug analysis).
downrec_link: Option<(u32, u32)>v2.0 Track-R R3b — down-recursion stitch link populated by
the lowerer’s downrec_idx_opt arm
(crates/luna-jit/src/jit_backend/trace.rs:7129+) when a
trace closes via TraceEnd::DownRec. Layout:
Some((trace_id_placeholder, target_head_pc)). R3b emits a
caller-pc guard at the close site that, on guard hit, returns
the SIDE_SENT_DOWNREC_CODE sentinel — and on guard miss,
falls back to R1’s safe deopt-tail (store back caller window +
return head_pc via GLOBAL sentinel, Self::dispatchable
pinned to false). R3b deliberately keeps
dispatchable = false even when this field is Some(_);
R3d will lift dispatchable = true after R3c wires the
dispatcher consumer.
Field semantics agreed for R3b -> R3c handoff:
.0= placeholder trace id. At compile time the trace doesn’t know its own index inhead_proto.tracesyet (the index is assigned at the close handler’straces.pushsite after this function returns). R3b writes0here as a “this trace, self-stitch” sentinel; R3c interprets a non-Nonevalue with.0 == 0as “stitch target = the trace currently dispatching” and usesSelf::head_pcfor resolution. A future R3.2 may carry an explicithead_proto.traces-index when mutual-recursion stitch lands..1=target_head_pc, copied fromrecord.head_pcat compile time. R3c’s stitch dispatcher tail-calls into the target trace at this PC (which today = self, the trace currently dispatching).
None for every trace that doesn’t close via TraceEnd:: DownRec. Tested via R3b’s r3b_lowerer_stitch_sentinel.rs
regression: at least 1 trace recorded on a fib(3) hot-loop
has downrec_link == Some(_) after the R3a recorder pushes
the threshold-tripping retfs.
downrec_multi_way_count: u8v2.0 Track-R R3d — number of distinct caller_pc candidates the
lowerer baked into the multi-way guard at the
TraceEnd::DownRec close. 0 for every trace that doesn’t
close via DownRec; 1 for R3c-shape single-CMP guards (the
dr_return_pc alone, no additional retfs matched the close
marker’s target_proto); >= 2 for R3d-shape multi-way
guards that triggered the dispatchable = true lift.
Capped at DOWNREC_MULTI_WAY_GUARD_MAX.
Read by the close handler in crates/luna-core/src/vm/exec.rs
to bump JitCounters.multi_way_guard_emitted (the probe
surface for R3d’s regression test
r3d_multi_way_guard_dispatch.rs).
Implementations§
Source§impl CompiledTrace
impl CompiledTrace
Sourcepub fn from_aot_meta(
entry: unsafe extern "C" fn(*mut i64) -> i64,
head_pc: u32,
n_ops: u32,
dispatchable: bool,
window_size: u32,
entry_tags: Rc<[u8]>,
exit_tags: Rc<[ExitTag]>,
global_tag_res_kind: TagResKind,
per_exit_tags: Vec<(u32, Rc<[ExitTag]>)>,
per_exit_inline: Vec<InlineSideExit>,
) -> CompiledTrace
pub fn from_aot_meta( entry: unsafe extern "C" fn(*mut i64) -> i64, head_pc: u32, n_ops: u32, dispatchable: bool, window_size: u32, entry_tags: Rc<[u8]>, exit_tags: Rc<[ExitTag]>, global_tag_res_kind: TagResKind, per_exit_tags: Vec<(u32, Rc<[ExitTag]>)>, per_exit_inline: Vec<InlineSideExit>, ) -> CompiledTrace
v1.3 Phase AOT Stage 7 sub-piece 4 — minimal AOT install
constructor; Stage 7 follow-up — extended to accept
per_exit_tags (typed-register side-exits) so GetUpval-heavy
traces install correctly.
Builds a CompiledTrace from the fields the AOT meta blob
carries plus a deploy-resolved trace fn pointer.
per_exit_inline, tags_side_trace_ptrs,
side_trace_cache, body_writes default to empty / null;
exit_hit_counts / exit_side_trace_ptrs are sized to
per_exit_tags.len() + 1 (each typed-exit slot + the global
clean-tail). The deploy Vm never records side traces against
an AOT-installed parent (the recorder is invoked from the
dispatch path; AOT traces install before any record can fire),
so the side-trace bookkeeping defaults are sound.
Traces whose runtime shape requires non-empty
per_exit_inline (depth>0 inlined cmp side-exits with frame
materialization chains) must not be emitted by the AOT
pipeline — this constructor produces a CompiledTrace that the
dispatcher would mis-restore for those traces. The AOT
recorder driver (luna-aot side) filters to traces whose
per_exit_inline.is_empty() before serializing.
§Safety
entry must be a valid unsafe extern "C" fn(*mut i64) -> i64
living in the binary’s text segment (linker-resolved from the
AOT-emitted trace .o). The constructor doesn’t validate this
— install_aot_trace is the next gate.