Alef

Alef is the polyglot binding generator behind the xberg.io ecosystem. It extracts a Rust API surface and emits language-native bindings, package scaffolding, type stubs, README files, API docs, e2e tests, and release metadata from one alef.toml.

Installation | Quick Start | Supported Targets | CLI Reference

Key Features

One source of truth - Configure a Rust workspace once and generate every enabled language target from it.
Language-native bindings - Emit host-language types, docs, errors, async wrappers, callbacks, and package files.
Multi-crate workspaces - Drive multiple independently published binding packages from a shared workspace config.
End-to-end fixtures - Generate cross-language test suites and registry-mode test apps from shared JSON fixtures.
Release-aware packaging - Sync versions, generate registry metadata, build artifacts, and validate publication state.
Configurable pipelines - Run setup, update, format, lint, test, clean, build, and publish commands per language.
Pluggable extension surface - Author domain-specific codegen logic via the Extension trait; ship as linked binaries, dynamic libraries, or template-only declarations.
Staleness checks - Cache inputs, embed generation hashes, and verify whether generated files are up to date.

Installation

Alef requires Rust 1.89 or newer. To build from source:

cargo install alef --locked

cargo-binstall avoids compiling Alef when a matching published artifact is available. It still requires Cargo and falls back to cargo install from source when no compatible artifact is available:

cargo binstall alef

Homebrew installs a bottle when one is available and otherwise may build the formula from source:

brew install xberg-io/tap/alef

Scoop installs the published Windows binary:

scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install alef

Quick Start

Create or edit alef.toml in your Rust workspace:

[workspace]
languages = ["python", "node", "ffi", "go"]
alef_version = "0.79.2"

[[crates]]
name = "sample_core"
sources = ["src/lib.rs"]
version_from = "Cargo.toml"

Then generate the language packages:

alef generate --format
alef scaffold
alef readme
alef docs --output docs/reference
alef verify

For a new project, Alef can create the initial config and first generated files:

alef init --lang python,node,ffi

For the full local generation pass, use:

alef all --format

Use --lang python,node to restrict commands to selected targets and --crate <name> to restrict commands to one configured crate.

Supported Targets

Target	Backend / package style
Python	PyO3 bindings with Python type stubs
TypeScript / Node.js	NAPI-RS native addon with `.d.ts` output
WebAssembly	wasm-bindgen package for browser and JS runtimes
Ruby	Magnus native extension
PHP	Native PHP extension
Elixir	Rustler NIF package
R	extendr package
Go	cgo package over the generated C FFI layer
Java	JVM package over the generated native library
Kotlin	Kotlin/JVM package over generated native bindings
Kotlin Android	Android package with generated JNI shims
C#	.NET package using P/Invoke
Dart / Flutter	flutter_rust_bridge package
Swift	Swift package with Rust bridge support
Zig	Zig package over the generated C ABI
Gleam	Gleam package backed by Rustler
C FFI	C ABI, header, and shared-library glue
JNI	Rust JNI shim crate exercised by both kotlin_android (Android AAR) and host-JVM tests

Canonical language slugs are python, node, wasm, ruby, php, elixir, r, go, java, csharp, kotlin, kotlin_android, swift, dart, gleam, zig, ffi, and jni.

Configuration Model

Alef uses the current multi-crate schema:

[workspace] stores shared target languages, tool preferences, and pipeline defaults.
[[crates]] describes each Rust API surface that should become one or more published packages.
[crates.<language>] sections customize module names, package names, feature flags, output paths, field naming, dependency extras, and language-specific generation behavior.
[[crates.adapters]], trait bridge config, service API config, and e2e config opt into higher-level generated wrappers when a target supports them.

Generated binding files carry Alef hashes and are overwritten by generation commands. Scaffolded package files are generated once unless the command explicitly opts into overwrite behavior; generated README and API doc files are owned by alef readme and alef docs.

Cargo Package Archives

Generated Rust crates can use Cargo's native package allowlist or denylist:

[workspace.package_metadata.cargo]
include = ["src/**", "build.rs", "!src/**/tests.rs"]

Use exclude instead of include for a denylist. Alef rejects configuring both because Cargo treats include as overriding exclude. Patterns remain ordered and crate-relative. A [crates.package_metadata.cargo] table replaces the workspace table for that crate, and an empty crate table clears the inherited filters.

Transparent String Wrappers

A core type that deliberately hides its string value from Display can still be exposed as a plain string in every binding. Mark the wrapper with explicit conversion operations:

#[derive(Clone)]
#[cfg_attr(alef, alef(transparent_string(from = "from", into = "into_inner")))]
pub struct SecretString(String);

impl SecretString {
    pub fn from(value: String) -> Self { Self(value) }
    pub fn into_inner(self) -> String { self.0 }
}

The wrapper is omitted from generated binding APIs. Alef calls SecretString::from(String) when a binding value enters Rust and consumes the wrapper with SecretString::into_inner() when a core value leaves Rust. Both operations must be public synchronous inherent methods with the exact signatures shown above. The wrapper must implement Clone: generated FFI getters clone an owned wrapper before calling the consuming into method. Alef reports invalid annotations, shapes, or methods as unsupported public-item diagnostics and never falls back to Display or ToString. Binding-visible wrappers must also have unique short type names; Alef reports same-named wrappers from different modules as ambiguous instead of attaching one module's conversion to the other.

The conversion metadata is preserved through struct and enum fields, parameters and returns, including nested Option, Vec, and map keys and values. Rust-backed shared generators, Dart's bridge crate, and the C FFI layer apply the conversion; Zig and Gleam inherit it through that FFI layer. The emitted-tree compile gate covers FFI, Python, Node, Wasm, and JNI. PHP is text-gated because its Rust crate needs PHP headers. Dart has direct conversion-generator tests; R, Zig, and Gleam do not yet have an annotation-specific downstream toolchain build.

Sensitive Fields

Mark a struct field, an enum variant, or a field of an enum variant that holds a credential so generated representations never print it:

pub struct Credentials {
    pub label: String,
    #[cfg_attr(alef, alef(sensitive))]
    pub token: String,
}

The flag only changes human-readable representations. Serialization and deserialization stay lossless: serde_json and every conversion between the binding and the core type still carry the real value.

Backend	Effect
Python (PyO3)	Dataclass fields are generated with `field(repr=False)`, so `repr()` and `str()` omit them while `dataclasses.asdict()` keeps them. Data enums with a sensitive field or variant return `Name(<redacted>)` from `__str__` and `__repr__` instead of serializing the payload.
Ruby (Magnus), Elixir (Rustler)	The Rust-side binding type drops `derive(Debug)` and gets a `Debug` impl that prints `<redacted>` for sensitive fields (the whole value for a sensitive enum variant).
Kotlin	A sensitive positional field of an error variant is rendered as `<redacted>` in the generated exception message.

The remaining backends (NAPI, WASM, extendr, Swift, Go, Java, C#, and the others) do not yet honour the flag; their generated types and toString/Debug-style output still print every field. Error messages produced by the core crate's own Display implementation are never rewritten, so a #[error("... {0}")] template that prints a sensitive field is the core crate's responsibility.

Elixir protection is incomplete: complete protection for ordinary tuple representations would need a later, breaking change to the Elixir representation.

Downloadable Native Components

Alef can split fixed Cargo feature profiles into signed native components instead of putting every feature combination in every wheel. A component is the deployable unit -- one Cargo feature set built into one signed prebuilt artifact per target -- modelled on Python extras but language-neutral; it provides one or more contracts. A contract is an explicitly selected Rust trait; Alef generates a versioned C function table for it, one producer entrypoint per contract in the component's cdylib, and a host runtime. Component implementation types currently construct through Default. Only C-compatible scalar, UTF-8, byte-buffer, and opaque-handle representations cross the dynamic-library boundary—never Rust trait objects or Rust-owned layouts. A component's version always tracks the core crate's version; components are not versioned independently.

[[crates.component_contracts]]
name = "engine"
trait_path = "my_core::Engine"
interface_version = 1

[[crates.components]]
name = "cuda"
provides = [{ contract = "engine", implementation = "my_core::CudaEngine" }]
features = ["cuda"]
default_features = false
# Optional: defaults to every supported target enabled by [targets], minus bundled_on.
targets = ["x86_64-unknown-linux-gnu", "aarch64-apple-darwin"]
# Optional: targets/platforms where downloading is unsupported and the component must
# instead be linked into the binding (not yet implemented by any backend).
bundled_on = []

[component_distribution]
url_template = "https://downloads.example.com/{component}/{version}/{target}/{artifact}"

[component_distribution.public_keys]
release-2026 = "BASE64_ENCODED_32_BYTE_ED25519_PUBLIC_KEY"

[component_distribution] may also be set per crate as [crates.component_distribution], which overrides the workspace default field by field (a crate-level url_template replaces the workspace one; a crate-level public key overrides a workspace key with the same ID, and workspace-only keys are kept). A component may provide more than one contract; a component provides each contract at most once, though the same contract may be provided by more than one component under different feature sets.

Generate/scaffold first, then build and sign each configured target in CI:

alef scaffold
alef component build
alef component package --signing-key "$CI_ED25519_PRIVATE_KEY_FILE" --key-id release-2026
alef component verify
alef component lock

The private Ed25519 key stays in protected CI secret storage. It signs canonical component manifests; it does not encrypt the binary. Generated native binding packages compile in the contents of components.lock.json, including the public verification keys, but do not contain the component libraries. Each supported binding exposes idiomatic load, prefetch, status, and cache path operations. The runtime selects the exact target, downloads on demand, verifies the pinned size, SHA-256 digest, and Ed25519 signature, installs under a cache lock, validates the ABI/contract/feature identity, and keeps the library loaded for the process lifetime.

Component host surface	Bindings
Direct native runtime	Python, Node.js, Ruby, PHP, Elixir, R, Dart/FRB, Swift
Shared generated C manager	C FFI, Go, Java, Kotlin/JVM, C#, Zig, Dart/FFI
Rustler manager	Gleam
Explicitly unsupported	WebAssembly, Kotlin Android, Android JNI, Apple mobile targets

Unsupported bindings still generate the four management operations, but each operation returns a clear platform error instead of silently omitting the API. Rust callers can use alef-component-runtime::ComponentManager directly.

The initial loader supports x86-64 and ARM64 Linux GNU, x86-64 and ARM64 macOS, and x86-64 Windows MSVC. Mobile and WebAssembly targets are intentionally excluded because they do not provide the same dynamic-loading model.

Extending Alef

Alef is opinionated about codegen and neutral about domain. The Extension trait lets you ship domain-specific generation logic (HTTP service APIs, plugin registries, custom bindings) without bloat in alef.

Linked Extension

Consumer crate implements alef::Extension, ships a thin CLI binary:

fn main() {
    alef::run_with_extensions(vec![Box::new(MyDomainExtension)])
}

Full type safety. Recommended for frameworks that generate an HTTP service API.

Dynamic Extension

Load a compiled .so/.dylib/.dll declaring a C-ABI factory function. Works when you can't ship a Rust binary.

[[extensions.dylib]]
path = "target/release/libmy_extension.dylib"

extern "C" fn alef_extension_factory() -> Box<dyn alef::Extension> {
    Box::new(MyExtension)
}

Template-only Extension

Declare [[extensions.template]] blocks in alef.toml pointing to Jinja templates. Alef's built-in TemplateExtension emits them — no Rust required.

The full extension walkthrough covers trait references and per-language emission patterns.

CLI Reference

Command	Purpose
`alef init`	Create `alef.toml`, generate initial bindings, and scaffold package files.
`alef extract`	Extract Rust source into Alef IR JSON.
`alef generate`	Generate bindings, service API wrappers, public API wrappers, and type stubs.
`alef stubs`	Generate type stubs only.
`alef scaffold`	Generate package manifests, native build files, and package scaffolding.
`alef readme`	Generate per-language README files.
`alef docs`	Generate Markdown API reference pages.
`alef setup`	Install per-language development dependencies.
`alef fmt` / `alef lint`	Run configured formatters, linters, and type checks.
`alef test`	Run configured unit, integration, e2e, or coverage test commands.
`alef build`	Build language bindings using native tools.
`alef verify`	Check generated files and optional compile/lint state for CI.
`alef diff`	Show what generation would change without writing files.
`alef schema`	Write or `--check` a vendored copy of the `alef.toml` JSON Schema.
`alef e2e`	Initialize, scaffold, validate, list, or generate local e2e suites.
`alef test-apps`	Generate and run standalone registry-mode test applications.
`alef publish`	Prepare, build, package, and validate release artifacts.
`alef component`	Build, package, verify, and lock downloadable runtime components.
`alef all`	Run the full generation workflow in one command.

Run alef --help or alef <command> --help for the full option set.

Vendoring the `alef.toml` JSON Schema

schemas/alef.schema.json is Alef's own config schema — the JSON Schema for alef.toml — which you may vendor so an editor can validate alef.toml offline. It is not a generated binding: nothing in alef generate, alef build, or alef all writes or refreshes it, and no build step reads it. Alef never creates the file on its own; run alef schema explicitly if you want a copy:

alef schema --output schemas/alef.schema.json   # write or refresh
alef schema --check                             # byte-exact staleness check

If a copy exists at the default path, alef verify reports it. A copy that describes a different alef.toml surface than the running Alef fails verification, because an editor validating against it is answering for a different release. A copy whose only difference is the embedded version stamp is reported as informational and does not fail — the described config surface is unchanged, so editor validation is still correct.

Development

This repository uses task for common workflows:

task setup
task build
task test
task lint

The most useful targeted commands while working on Alef itself are:

cargo test <module_or_test_name>
cargo insta review
prek run --all-files

Part of Xberg.io

Xberg — the open-source content-intelligence engine: text, tables, and metadata from 101 formats (115 file extensions), with OCR, transcription, and code intelligence. MIT.
Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
html-to-markdown — fast, lossless HTML→Markdown engine.
liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
Discord — community, roadmap, and release discussion.

License

MIT - see LICENSE for details.

alef 0.110.2