Crate polydat_core 

polydat-core

The Polydat runtime: the value model, the graph compiler, the execution engines, the kernels, the comprehension runtime, the node macro’s support surface, the nodes the compiler synthesizes itself, and the numeric bodies the native lowerings share with the node library.

A program declares typed inputs and a graph of named functions; the compiler produces a kernel whose named outputs are pulled on demand. The same inputs always yield the same outputs, on any thread, any host, and any engine, with no state carried between evaluations.

Most programs depend on the polydat facade, which re-exports this crate together with the node library (polydat-nodes) and the language (polydat-grammar) at the paths they always had. Depend on polydat-core directly to assemble your own node set without the standard library linked, or to build a tool that needs the compiler and engines alone.

Quick start

The runtime compiles any program whose functions are linked. The standard functions such as hash live in polydat-nodes and register at link time, so a program that calls them needs that crate linked as well:

use polydat_core::dsl::compile_polydat_with;
use polydat_core::{Engine, Provenance};

let mut kernel = compile_polydat_with(
    r#"
        input cycle: u64
        id := mod(hash(cycle), 1000)
    "#,
    Engine::Closures(Provenance::PushPull),
)?;

kernel.set_inputs(&[7]);
assert!(kernel.pull("id").as_u64() < 1000);

For programmatic construction, compile::assembly::PolydatAssembler wires boxed nodes by name and compiles the result the same way.

Engines

One program compiles to any of three engines and gives the same values on each; the host names one with Engine, and Engine::default is the fastest the build has.

• Engine::Interpreter: boxed nodes over typed value buffers, with as much of the graph fused into native cones as its JitMode allows.
• Engine::Closures: one generated closure per node over a flat slot buffer.
• Engine::Native: Cranelift machine code where a node has a lowering and the node’s closure elsewhere. Needs the jit feature.

Provenance chooses how much re-evaluation a changed input triggers; it is an optimization and never changes a result. Every engine accepts every program the interpreter accepts and drives it through the one Kernel trait.

Program and state

inputs (u64 tuple, cursors, externs)
    │
    ▼
┌──────────────────────────────────┐
│ KernelProgram   immutable, Arc   │  shared by every thread
│  nodes · wiring · outputs · consts│
└───────────────┬──────────────────┘
                │ create_kernel()
                ▼
┌──────────────────────────────────┐
│ Kernel          one per thread   │  no locks, no shared writes
│  slot buffers · provenance masks │
└───────────────┬──────────────────┘
                ▼
         pull("id") → Value

A KernelProgram is the compiled, immutable half, shared by reference; a Kernel is one thread’s private state over it. Outputs are owned by their provenance: a value stands until an input that reaches it is written.

Cargo features

• jit (default): the native engine, on Cranelift.
• vectordata: vector-dataset access nodes for ML/AI-oriented workloads.

Modules

• ast: the value model and node contract: ast::Value, the ast::PolydatNode trait, ast::Port.
• dsl: compiling Polydat source: dsl::compile_polydat_with for a chosen engine, dsl::compile_polydat_kernel for the default, and dsl::compile_polydat for the interpreter kernel; the node registry, factories, and compile events.
• compile: graph construction and the engines: compile::assembly (the assembler and adapter insertion), compile::fusion, compile::closures, compile::hybrid (the native engine’s kernel), compile::jit (Cranelift lowering, feature-gated), compile::select (engine and provenance selection).
• kernel: the runtime: the Kernel and KernelProgram traits, the interpreter’s kernel::PolydatProgram and kernel::PolydatState, shared cells, scopes, subcontexts, traversal activation.
• iteration: comprehensions, cursors, partitions, and the coordinate algebra.
• library: the nodes the compiler keeps: adapters (library::polyfill), assertions, constants, identity, formatting, tile rendering (library::tile_render), and the library-internal support (library::support). The node library proper is polydat-nodes.
• numeric: the numeric bodies shared by the node library and the native lowerings.
• tile: Polytile at the host boundary.
• binder, derive_support, resource, audit: the typed binding contracts, the #[polydat_node] macro’s support surface, the host resource bridge, and the log sink.
• viz: AST and graph visualization, re-exported from the grammar.

The narrative documentation lives in the repository under crates/polydat/docs/, organized by the documentation index; the runtime model, the graph compiler, and the engines design documents are the ones to read first.

Re-exports

pub use derive_support::Const;

pub use compile::cone::JitMode;

pub use compile::select::Engine;

pub use compile::select::EnginePlan;

pub use compile::select::KernelError;

pub use compile::select::Provenance;

pub use kernel::Kernel;

pub use kernel::KernelProgram;

pub use kernel::set_panic_reporting_downstream;

pub use resource::RESOURCE_ACCESSOR;

pub use resource::ResourceAccessor;

pub use resource::resource_lookup;

pub use library::support::audit;

pub use inventory;

pub use half;

Modules

ast

Core types for Polydat nodes: values, ports, metadata, and the evaluation trait.

binder

Typed-binding contracts between adapters and the polydat runtime.

compile

Kernel compilation: assembled DAG → fast executable kernel.

derive_support

Trait surface that the #[polydat_node] proc-macro (polydat-derive) calls into for boxing / unboxing wire values.

dsl

Polydat DSL: lexer, parser, and AST for .polydat kernel definition files.

iteration

Iteration & coordinate algebra.

kernel

Polydat runtime kernel: compiled DAG with pull-through evaluation.

library

Standard Polydat function-node library + sampling primitives + library-internal support.

numeric

The numeric bodies the node library and the native lowerings share. A node in polydat-nodes is a thin wrapper over a function here, and the JIT helper for that node calls the same function, so the interpreter and native code compute the same bits by construction (compiled_handles.md §6, the corner suite).

resource

Dependency-inverted resource-accessor bridge (SRD-104, Phase 0).

tile

Polytile at the host boundary (SRD 114 §5.6): build a tile from template text, from structural JSON text, or from a parsed JSON value, then compile it with a program via tile::compile_polydat_with_tiles.

viz

DAG visualization for Polydat Kernels.

Macros

register_nodes

Register a node module’s signatures and builder with the Polydat runtime.

Attribute Macros

polydat_node

#[polydat_node] — derive a polydat node from a typed Rust function signature.