Apache Fory™ Rust

Apache Fory™ is a blazing fast multi-language serialization framework powered by JIT compilation and zero-copy techniques, providing up to ultra-fast performance while maintaining ease of use and safety.

The Rust implementation provides versatile and high-performance serialization with automatic memory management and compile-time type safety. It defaults to xlang mode for cross-language payloads; use native mode with .xlang(false) for Rust-only traffic and native-only data shapes. Trait objects and dyn Any also work in xlang mode when every selected concrete target has an xlang-compatible structural or EXT identity.

Why Apache Fory™ Rust?

Blazingly Fast: Zero-copy deserialization and optimized binary protocols
Cross-Language: Seamlessly serialize/deserialize data across Java, Python, C++, Go, JavaScript, and Rust
Type-Safe: Compile-time type checking with derive macros
Circular References: Automatic tracking of shared and circular references with Rc/Arc and weak pointers
Polymorphic: Serialize trait objects with Box<dyn Trait>, Rc<dyn Trait>, and Arc<dyn Trait>
Schema Evolution: Compatible mode for independent schema changes
Reduced-Precision Types: Float16 and BFloat16 scalars with Vec<Float16> / Vec<BFloat16> arrays
Two Formats: Object graph serialization and the Standard Row Format shared with Java, C++, and Python

Crates

Crate	Description	Version
`fory`	User-facing API, runtime types, and derive macros	1.6.0
`fory-core`	Lower-level runtime crate for advanced integrations	1.6.0
`fory-derive`	Lower-level procedural macro crate for direct runtime use	1.6.0

Most applications should depend on fory only. It re-exports the derive macros and the public runtime types needed by generated code. Use fory-core or fory-derive directly only when intentionally building on the lower-level runtime crates.

Quick Start

Add Apache Fory™ to your Cargo.toml:

[dependencies]
fory = "1.6.0"

Basic Example

use fory::{Fory, Error, Reader};
use fory::{ForyEnum, ForyStruct, ForyUnion};

#[derive(ForyStruct, Debug, PartialEq)]
struct User {
    name: String,
    age: i32,
    email: String,
}

fn main() -> Result<(), Error> {
    let mut fory = Fory::builder().xlang(true).build();
    fory.register::<User>(1)?;

    let user = User {
        name: "Alice".to_string(),
        age: 30,
        email: "alice@example.com".to_string(),
    };

    // Serialize
    let bytes = fory.serialize(&user)?;
    // Deserialize
    let decoded: User = fory.deserialize(&bytes)?;
    assert_eq!(user, decoded);

    // Serialize to specified buffer
    let mut buf: Vec<u8> = vec![];
    fory.serialize_to(&mut buf, &user)?;
    // Deserialize from specified buffer
    let mut reader = Reader::new(&buf);
    let decoded: User = fory.deserialize_from(&mut reader)?;
    assert_eq!(user, decoded);
    Ok(())
}

Core Features

1. Object Graph Serialization

Apache Fory™ provides automatic serialization of complex object graphs, preserving the structure and relationships between objects. The #[derive(ForyStruct)] macro generates efficient serialization code at compile time, eliminating runtime overhead.

Key capabilities:

Nested struct serialization with arbitrary depth
Collection types (Vec, HashMap, HashSet, BTreeMap)
Optional fields with Option<T>
Automatic handling of primitive types and strings
Efficient binary encoding with variable-length integers

use fory::{Fory, Error};
use fory::{ForyEnum, ForyStruct, ForyUnion};
use std::collections::HashMap;

#[derive(ForyStruct, Debug, PartialEq)]
struct Person {
    name: String,
    age: i32,
    address: Address,
    hobbies: Vec<String>,
    metadata: HashMap<String, String>,
}

#[derive(ForyStruct, Debug, PartialEq)]
struct Address {
    street: String,
    city: String,
    country: String,
}

let mut fory = Fory::builder().xlang(true).build();
fory.register_by_name::<Address>("example.Address").unwrap();
fory.register_by_name::<Person>("example.Person").unwrap();

let person = Person {
    name: "John Doe".to_string(),
    age: 30,
    address: Address {
        street: "123 Main St".to_string(),
        city: "New York".to_string(),
        country: "USA".to_string(),
    },
    hobbies: vec!["reading".to_string(), "coding".to_string()],
    metadata: HashMap::from([
        ("role".to_string(), "developer".to_string()),
    ]),
};

let bytes = fory.serialize(&person).unwrap();
let decoded: Person = fory.deserialize(&bytes)?;
assert_eq!(person, decoded);

2. Native-Mode Shared and Circular References

Apache Fory™ automatically tracks and preserves reference identity for shared objects using Rc<T> and Arc<T>. When the same object is referenced multiple times, Fory serializes it only once and uses reference IDs for subsequent occurrences. This ensures:

Space efficiency: No data duplication in serialized output
Reference identity preservation: Deserialized objects maintain the same sharing relationships
Circular reference support: Use RcWeak<T> and ArcWeak<T> to break cycles

The examples in this section use native mode because Rc, Arc, and weak-pointer identity are Rust object-graph features. Native mode stays on Rust's native type system instead of limiting the payload to portable xlang mappings.

Shared References with Rc/Arc

use fory::Fory;
use std::rc::Rc;

let fory = Fory::builder().xlang(false).build();

// Create a shared value
let shared = Rc::new(String::from("shared_value"));

// Reference it multiple times
let data = vec![shared.clone(), shared.clone(), shared.clone()];

// The shared value is serialized only once
let bytes = fory.serialize(&data)?;
let decoded: Vec<Rc<String>> = fory.deserialize(&bytes)?;

// Verify reference identity is preserved
assert_eq!(decoded.len(), 3);
assert_eq!(*decoded[0], "shared_value");

// All three Rc pointers point to the same object
assert!(Rc::ptr_eq(&decoded[0], &decoded[1]));
assert!(Rc::ptr_eq(&decoded[1], &decoded[2]));

For thread-safe shared references, use Arc<T>.

Circular References with Weak Pointers

To serialize circular references like parent-child relationships or doubly-linked structures, use RcWeak<T> or ArcWeak<T> to break the cycle. These weak pointers are serialized as references to their strong counterparts, preserving the graph structure without causing memory leaks or infinite recursion.

How it works:

Weak pointers serialize as references to their target objects
If the strong pointer has been dropped, weak serializes as Null
Forward references (weak appearing before target) are resolved via callbacks
All clones of a weak pointer share the same internal cell for automatic updates

use fory::{Fory, Error};
use fory::{ForyEnum, ForyStruct, ForyUnion};
use fory::RcWeak;
use std::rc::Rc;
use std::cell::RefCell;

#[derive(ForyStruct, Debug)]
struct Node {
    value: i32,
    parent: RcWeak<RefCell<Node>>,
    children: Vec<Rc<RefCell<Node>>>,
}

let mut fory = Fory::builder().xlang(false).build();
fory.register::<Node>(2000)?;

// Build a parent-child tree
let parent = Rc::new(RefCell::new(Node {
    value: 1,
    parent: RcWeak::new(),
    children: vec![],
}));

let child1 = Rc::new(RefCell::new(Node {
    value: 2,
    parent: RcWeak::from(&parent),
    children: vec![],
}));

let child2 = Rc::new(RefCell::new(Node {
    value: 3,
    parent: RcWeak::from(&parent),
    children: vec![],
}));

parent.borrow_mut().children.push(child1.clone());
parent.borrow_mut().children.push(child2.clone());

// Serialize and deserialize the circular structure
let bytes = fory.serialize(&parent)?;
let decoded: Rc<RefCell<Node>> = fory.deserialize(&bytes)?;

// Verify the circular relationship
assert_eq!(decoded.borrow().children.len(), 2);
for child in &decoded.borrow().children {
    let upgraded_parent = child.borrow().parent.upgrade().unwrap();
    assert!(Rc::ptr_eq(&decoded, &upgraded_parent));
}

3. Trait Object Serialization

Apache Fory™ supports polymorphic serialization through trait objects, enabling dynamic dispatch and type flexibility. This is essential for plugin systems, heterogeneous collections, and extensible architectures.

The examples in this section use native mode. The same trait-object and dyn Any carriers work in xlang mode when every selected concrete target has an xlang-compatible structural or EXT identity; the Rust trait or erased-carrier identity is not written to the wire.

Supported trait object types:

Box<dyn Trait> - Owned trait objects
Rc<dyn Trait> - Reference-counted trait objects
Arc<dyn Trait> - Thread-safe reference-counted trait objects
Box<dyn Any>/Rc<dyn Any>/Arc<dyn Any + Send + Sync> - Any trait type objects
Vec<Box<dyn Trait>>, HashMap<K, Box<dyn Trait>> - Collections of trait objects

Box<dyn Any>, Rc<dyn Any>, and Arc<dyn Any + Send + Sync> are supported erased Any carriers for registered concrete non-container payloads. Use Arc<dyn Any + Send + Sync> when the erased payload must be shareable across threads; the concrete payload type must also satisfy Send + Sync. Registered structs, enums, and unions that satisfy those bounds can be used as the erased payload. Generic containers such as Vec<T>, HashMap<K, V>, HashSet<T>, and LinkedList<T> are not supported directly as top-level erased Any payloads behind any of those carriers. This also includes primitive vector encodings such as Vec<u8>. Wrap the container in a registered derived type, or register an exact-target custom serializer when an opaque EXT/NAMED_EXT representation is appropriate.

Basic Trait Object Serialization Example:

use fory::{register_trait_type, Fory, ForyObject, ForyStruct};

trait Animal: ForyObject {
    fn speak(&self) -> String;
    fn name(&self) -> &str;
}

#[derive(ForyStruct)]
struct Dog { name: String, breed: String }

impl Animal for Dog {
    fn speak(&self) -> String { "Woof!".to_string() }
    fn name(&self) -> &str { &self.name }
}

#[derive(ForyStruct)]
struct Cat { name: String, color: String }

impl Animal for Cat {
    fn speak(&self) -> String { "Meow!".to_string() }
    fn name(&self) -> &str { &self.name }
}

// Register trait implementations
register_trait_type!(Animal, Dog, Cat);

#[derive(ForyStruct)]
struct Zoo {
    star_animal: Box<dyn Animal>,
}

let mut fory = Fory::builder().xlang(false).build();
fory.register::<Dog>(100)?;
fory.register::<Cat>(101)?;
fory.register::<Zoo>(102)?;

let zoo = Zoo {
    star_animal: Box::new(Dog {
        name: "Buddy".to_string(),
        breed: "Labrador".to_string(),
    }),
};

let bytes = fory.serialize(&zoo)?;
let decoded: Zoo = fory.deserialize(&bytes)?;

assert_eq!(decoded.star_animal.name(), "Buddy");
assert_eq!(decoded.star_animal.speak(), "Woof!");

4. Schema Evolution

Apache Fory™ supports schema evolution in Compatible mode, allowing serialization and deserialization peers to have different type definitions. Compatible mode is the default for both xlang and native mode. Set .compatible(false) only when every reader and writer always uses the same schema and you want faster serialization and smaller size. For xlang payloads, use .compatible(false) only after verifying that every language uses the same schema, or when native types are generated from Fory schema IDL.

Features:

Add new fields with default values
Remove obsolete fields (skipped during deserialization)
Change field nullability (T ↔ Option<T>)
Reorder fields (matched by name, not position)
Type-safe fallback to default values for missing fields

Compatibility rules:

Field names must match (case-sensitive)
Type changes are not supported (except nullable/non-nullable)
Nested struct types must be registered on both sides

use fory::Fory;
use fory::{ForyEnum, ForyStruct, ForyUnion};
use std::collections::HashMap;

#[derive(ForyStruct, Debug)]
struct PersonV1 {
    name: String,
    age: i32,
    address: String,
}

#[derive(ForyStruct, Debug)]
struct PersonV2 {
    name: String,
    age: i32,
    // address removed
    // phone added
    phone: Option<String>,
    metadata: HashMap<String, String>,
}

let mut fory1 = Fory::builder().xlang(true).build();
fory1.register_by_name::<PersonV1>("example.Person").unwrap();

let mut fory2 = Fory::builder().xlang(true).build();
fory2.register_by_name::<PersonV2>("example.Person").unwrap();

let person_v1 = PersonV1 {
    name: "Alice".to_string(),
    age: 30,
    address: "123 Main St".to_string(),
};

// Serialize with V1
let bytes = fory1.serialize(&person_v1).unwrap();

// Deserialize with V2 - missing fields get default values
let person_v2: PersonV2 = fory2.deserialize(&bytes)?;
assert_eq!(person_v2.name, "Alice");
assert_eq!(person_v2.age, 30);
assert_eq!(person_v2.phone, None);

5. Native-Mode Enum Support

Apache Fory™ supports three types of enum variants with full schema evolution in Compatible mode:

Variant Types:

Unit: C-style enums (Status::Active)
Unnamed: Tuple-like variants (Message::Pair(String, i32))
Named: Struct-like variants (Event::Click { x: i32, y: i32 })

Features:

Efficient varint encoding for variant ordinals
Schema evolution support (add/remove variants, add/remove fields)
Default variant support with #[fory(default)]
Automatic type mismatch handling

use fory::{Fory, ForyUnion};

#[derive(ForyUnion, Debug, PartialEq)]
enum Value {
    #[fory(default)]
    Null,
    Bool(bool),
    Number(f64),
    Text(String),
    Object { name: String, value: i32 },
}

let mut fory = Fory::builder().xlang(false).build();
fory.register::<Value>(1)?;

let value = Value::Object { name: "score".to_string(), value: 100 };
let bytes = fory.serialize(&value)?;
let decoded: Value = fory.deserialize(&bytes)?;
assert_eq!(value, decoded);

Evolution capabilities:

Unknown variants → Falls back to default variant
Named variant fields → Add/remove fields (missing fields use defaults)
Unnamed variant elements → Add/remove elements (extras skipped, missing use defaults)
Variant type mismatches → Automatically uses default value for current variant

Best practices:

Always mark a default variant with #[fory(default)]
Named variants provide better evolution than unnamed
Use compatible mode for cross-version communication

6. Native-Mode Tuple Support

Apache Fory™ supports tuples up to 22 elements out of the box with efficient serialization in both compatible mode and same-schema mode.

Features:

Automatic serialization for tuples from 1 to 22 elements
Heterogeneous type support (each element can be a different type)
Schema evolution in Compatible mode (handles missing/extra elements)

Schema modes:

Same-schema mode: Serializes elements sequentially without collection headers for minimal overhead
Compatible mode: Uses collection protocol with type metadata for schema evolution

use fory::{Fory, Error};

let mut fory = Fory::builder().xlang(false).build();

// Tuple with heterogeneous types
let data: (i32, String, bool, Vec<i32>) = (
    42,
    "hello".to_string(),
    true,
    vec![1, 2, 3],
);

let bytes = fory.serialize(&data)?;
let decoded: (i32, String, bool, Vec<i32>) = fory.deserialize(&bytes)?;
assert_eq!(data, decoded);

7. Custom Serializers

For a type that needs an opaque encoding, implement a custom Serializer. A separate serializer type can also target a type from another crate; see the external-type serialization guide. Custom serializers work in native and xlang modes when the chosen EXT identity and opaque body format are supported by every peer. The example below uses native mode.

Types with special serialization requirements
Existing data format compatibility
Performance-critical opaque encoding

use fory::{Error, Fory, ReadContext, Serializer, WriteContext};

#[derive(Debug, PartialEq)]
struct Point {
    value: i32,
}

impl Serializer for Point {
    type Target = Self;

    fn write_data(value: &Self, context: &mut WriteContext) -> Result<(), Error> {
        context.writer.write_i32(value.value);
        Ok(())
    }

    fn read_data(context: &mut ReadContext) -> Result<Self, Error> {
        Ok(Self {
            value: context.reader.read_i32()?,
        })
    }
}

let mut fory = Fory::builder().xlang(false).build();
fory.register_serializer::<Point>(100)?;

let point = Point {
    value: 42,
};
let bytes = fory.serialize(&point)?;
let decoded: Point = fory.deserialize(&bytes)?;
assert_eq!(point, decoded);

Custom serializers implement the body-only write_data and read_data operations. Fory's complete-value write and read operations add reference and type-information framing.

8. Standard Row Format

Apache Fory™ Rust implements the Standard Row Format shared with Java, C++, and Python. from_row returns a borrowed view, so applications can validate and access selected fields or collection elements without reconstructing the complete value.

Key benefits:

Zero-copy access: Read strings, binary values, and nested structures through borrowed views
Selective access: Read only the fields or collection elements the application needs
Cross-language rows: Exchange the same binary layout with Java, C++, and Python
Bounds-checked reads: Malformed offsets, sizes, counts, and UTF-8 return Error

When to use row format:

Analytics workloads with selective field access
Large datasets where only a subset of fields is needed
Memory-constrained environments
High-throughput data pipelines
Standard Row Format interchange with other Fory implementations

use fory::{from_row, to_row, Error, ForyRow, RowView};

#[derive(ForyRow)]
struct UserProfile {
    id: i64,
    username: String,
    email: Option<String>,
    scores: Vec<i32>,
    is_active: bool,
}

fn main() -> Result<(), Error> {
    let bytes = to_row(&UserProfile {
        id: 12345,
        username: "alice".to_string(),
        email: None,
        scores: vec![95, 87, 92, 88],
        is_active: true,
    })?;

    let row = from_row::<UserProfile>(&bytes)?;
    assert_eq!(row.id()?, 12345);
    assert_eq!(row.username()?, "alice");
    assert_eq!(row.email()?, None);
    assert!(row.is_active()?);

    let scores = row.scores()?;
    assert_eq!(scores.len(), 4);
    assert_eq!(scores.get(1)?, 87);
    assert_eq!(
        scores.iter().collect::<Result<Vec<_>, _>>()?,
        [95, 87, 92, 88]
    );
    assert_eq!(row.as_bytes(), bytes);
    Ok(())
}

#[derive(ForyRow)] supports named structs, including generic structs, and encodes fields in source declaration order. Option<T> supplies field or array-element nullability without changing T's slot width. Generated field methods, array access and iteration, and map indexed access return Result, validating variable ranges and UTF-8 when accessed. Immutable views are cheap Copy values, and the RowView trait exposes their exact encoded slice through as_bytes.

Supported fixed-width values are bool, i8, i16, i32, i64, f32, f64, Date, Timestamp, and Duration. Supported variable-width values are UTF-8 String/&str, binary Vec<u8>/&[u8], fixed and dynamic arrays over supported element types, BTreeMap, nested derived structs, and Option<T>. Vec<u8> uses the binary encoding rather than the Standard Array encoding. Float16 and Decimal are not supported by Row Format because the standard specification does not define complete interoperable encodings for them.

Standard rows use an 8-byte-aligned null bitmap and one 8-byte slot per struct field. Fixed-width fields are stored little-endian in their slots. Variable-width slots encode the little-endian u64 value (relative_offset << 32) | size; variable bodies and array slot regions have zero padding to 8-byte alignment. Standard arrays use natural-width storage for fixed elements, and maps contain complete key and value arrays.

to_row accepts derived structs, supported arrays, and BTreeMap roots. to_row_into writes the same bytes into a reusable caller-owned Vec<u8> and clears partial output on error. Scalar, string, binary, and Option<T> values are field or element values rather than standalone roots. See the Rust Row Format guide and Row Format specification for details.

Cross-Language Serialization

Apache Fory™ supports seamless data exchange across multiple languages:

use fory::Fory;

// Use xlang mode, the Rust default.
let mut fory = Fory::builder().xlang(true).build();

// Register types with consistent IDs across languages
fory.register::<MyStruct>(100)?;

// Or use name-based registration
fory.register_by_name::<MyStruct>("com.example.MyStruct")?;

See xlang_type_mapping.md for type mapping across languages.

Performance

Apache Fory™ Rust is designed for maximum performance:

Selective Zero-Copy Access: Row Format returns borrowed views for direct field and element access
Buffer Pre-allocation: Minimizes memory allocations during serialization
Compact Encoding: Variable-length encoding for space efficiency
Little-Endian: Optimized for modern CPU architectures
Reference Deduplication: Shared objects serialized only once

Run benchmarks:

cd benchmarks/rust
./run.sh

Documentation

User Guide - Comprehensive user documentation
API Documentation - Complete API reference
Protocol Specification - Serialization protocol details
Type Mapping - Cross-language type mappings
Source - Source code for doc

Use Cases

Object Serialization

Complex data structures with nested objects and references
Cross-language communication in microservices
General-purpose serialization with full type safety
Schema evolution with compatible mode
Graph-like data structures with circular references

Standard Row Format

High-throughput data processing
Analytics workloads requiring fast field access
Memory-constrained environments
Real-time data streaming applications
Zero-copy scenarios

Development

Building

cd rust
cargo build

Testing

# Run all tests
cargo test --workspace

# Run specific test
cargo test -p tests --test test_complex_struct

Code Quality

# Format code
cargo fmt

# Check formatting
cargo fmt --check

# Run linter
cargo clippy --all-targets --all-features -- -D warnings

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Support

Documentation: docs.rs/fory
Issues: GitHub Issues
Discussions: GitHub Discussions
Slack: Apache Fory Slack

Apache Fory™ - Blazingly fast multi-language serialization framework.

fory-core 1.6.0-rc.1

Apache Fory™ Rust

Why Apache Fory™ Rust?

Crates

Quick Start

Basic Example

Core Features

1. Object Graph Serialization

2. Native-Mode Shared and Circular References

Shared References with Rc/Arc

Circular References with Weak Pointers

3. Trait Object Serialization

4. Schema Evolution

5. Native-Mode Enum Support

6. Native-Mode Tuple Support

7. Custom Serializers

8. Standard Row Format

Cross-Language Serialization

Performance

Documentation

Use Cases

Object Serialization

Standard Row Format

Development

Building

Testing

Code Quality

License

Contributing

Support