fory/lib.rs
1// Licensed to the Apache Software Foundation (ASF) under one
2// or more contributor license agreements. See the NOTICE file
3// distributed with this work for additional information
4// regarding copyright ownership. The ASF licenses this file
5// to you under the Apache License, Version 2.0 (the
6// "License"); you may not use this file except in compliance
7// with the License. You may obtain a copy of the License at
8//
9// http://www.apache.org/licenses/LICENSE-2.0
10//
11// Unless required by applicable law or agreed to in writing,
12// software distributed under the License is distributed on an
13// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14// KIND, either express or implied. See the License for the
15// specific language governing permissions and limitations
16// under the License.
17
18//! # Apache Fory™ Rust
19//!
20//! **Apache Fory™** is a blazingly fast multi-language serialization framework powered by
21//! **JIT compilation** and **zero-copy** techniques, providing up to ultra-fast performance
22//! while maintaining ease of use and safety.
23//!
24//! The Rust implementation provides versatile and high-performance serialization with
25//! automatic memory management and compile-time type safety.
26//!
27//! **GitHub**: <https://github.com/apache/fory>
28//!
29//! ## Why Apache Fory™ Rust?
30//!
31//! Apache Fory™ Rust solves the fundamental serialization dilemma: **you shouldn't have to
32//! choose between performance and developer experience**. Traditional frameworks force you
33//! to pick between fast but fragile binary formats, flexible but slow text-based protocols,
34//! or complex solutions that don't support your language's advanced features.
35//!
36//! **Key differentiators:**
37//!
38//! - **🔥 Blazingly Fast**: Zero-copy deserialization and optimized binary protocols
39//! - **🌍 Cross-Language**: Seamlessly serialize/deserialize data across Java, Python, C++, Go, JavaScript, and Rust
40//! - **🎯 Type-Safe**: Compile-time type checking with derive macros
41//! - **🔄 Circular References**: Automatic tracking of shared and circular references with `Rc`/`Arc` and weak pointers
42//! - **🧬 Polymorphic**: Serialize trait objects with `Box<dyn Trait>`, `Rc<dyn Trait>`, and `Arc<dyn Trait>`
43//! - **📦 Schema Evolution**: Compatible mode for independent schema changes
44//! - **⚡ Two Formats**: Object graph serialization and zero-copy row-based format
45//!
46//! ## Quick Start
47//!
48//! Add Apache Fory™ to your `Cargo.toml`:
49//!
50//! ```toml
51//! [dependencies]
52//! fory = "0.13"
53//! fory-derive = "0.13"
54//! ```
55//!
56//! ### Basic Example
57//!
58//! ```rust
59//! use fory::{Fory, Error, Reader};
60//! use fory::ForyObject;
61//!
62//! #[derive(ForyObject, Debug, PartialEq)]
63//! struct User {
64//! name: String,
65//! age: i32,
66//! email: String,
67//! }
68//!
69//! # fn main() -> Result<(), Error> {
70//! let mut fory = Fory::default();
71//! fory.register::<User>(1)?;
72//!
73//! let user = User {
74//! name: "Alice".to_string(),
75//! age: 30,
76//! email: "alice@example.com".to_string(),
77//! };
78//!
79//! // Serialize and deserialize
80//! let bytes = fory.serialize(&user)?;
81//! let decoded: User = fory.deserialize(&bytes)?;
82//! assert_eq!(user, decoded);
83//!
84//! // Serialize to specified buffer and deserialize from it
85//! let mut buf: Vec<u8> = vec![];
86//! fory.serialize_to(&mut buf, &user)?;
87//! let mut reader = Reader::new(&buf);
88//! let decoded: User = fory.deserialize_from(&mut reader)?;
89//! assert_eq!(user, decoded);
90//! # Ok(())
91//! # }
92//! ```
93//!
94//! ## Core Features
95//!
96//! Apache Fory™ Rust provides seven major feature categories, each designed to solve
97//! specific serialization challenges in modern applications.
98//!
99//! ### 1. Object Graph Serialization
100//!
101//! **What it does:** Automatically serializes complex nested data structures while
102//! preserving relationships and hierarchies.
103//!
104//! **Why it matters:** Most real-world applications deal with complex domain models,
105//! not just flat data structures. Apache Fory™ handles arbitrary nesting depth,
106//! collections, and optional fields without manual mapping code.
107//!
108//! **Technical approach:** The `#[derive(ForyObject)]` macro generates efficient
109//! serialization code at compile time using procedural macros. This eliminates runtime
110//! reflection overhead while maintaining type safety.
111//!
112//! **Key capabilities:**
113//!
114//! - Nested struct serialization with arbitrary depth
115//! - Collection types (`Vec`, `HashMap`, `HashSet`, `BTreeMap`)
116//! - Optional fields with `Option<T>`
117//! - Automatic handling of primitive types and strings
118//! - Efficient binary encoding with variable-length integers
119//!
120//! ```rust
121//! use fory::{Fory, Error};
122//! use fory::ForyObject;
123//! use std::collections::HashMap;
124//!
125//! #[derive(ForyObject, Debug, PartialEq)]
126//! struct Person {
127//! name: String,
128//! age: i32,
129//! address: Address,
130//! hobbies: Vec<String>,
131//! metadata: HashMap<String, String>,
132//! }
133//!
134//! #[derive(ForyObject, Debug, PartialEq)]
135//! struct Address {
136//! street: String,
137//! city: String,
138//! country: String,
139//! }
140//!
141//! # fn main() -> Result<(), Error> {
142//! let mut fory = Fory::default();
143//! fory.register::<Address>(100);
144//! fory.register::<Person>(200);
145//!
146//! let person = Person {
147//! name: "John Doe".to_string(),
148//! age: 30,
149//! address: Address {
150//! street: "123 Main St".to_string(),
151//! city: "New York".to_string(),
152//! country: "USA".to_string(),
153//! },
154//! hobbies: vec!["reading".to_string(), "coding".to_string()],
155//! metadata: HashMap::from([
156//! ("role".to_string(), "developer".to_string()),
157//! ]),
158//! };
159//!
160//! let bytes = fory.serialize(&person)?;
161//! let decoded: Person = fory.deserialize(&bytes)?;
162//! assert_eq!(person, decoded);
163//! # Ok(())
164//! # }
165//! ```
166//!
167//! ### 2. Shared and Circular References
168//!
169//! **What it does:** Automatically tracks and preserves reference identity for shared
170//! objects using `Rc<T>` and `Arc<T>`, and handles circular references using weak pointers.
171//!
172//! **Why it matters:** Graph-like data structures (trees, linked lists, object-relational
173//! models) are common in real applications but notoriously difficult to serialize. Most
174//! frameworks either panic on circular references or require extensive manual handling.
175//!
176//! **Technical approach:** Apache Fory™ maintains a reference tracking map during
177//! serialization. When the same object is encountered multiple times, it's serialized
178//! only once and subsequent references use IDs. Weak pointers (`RcWeak<T>`, `ArcWeak<T>`)
179//! break cycles by serializing as references without strong ownership.
180//!
181//! **Benefits:**
182//!
183//! - **Space efficiency**: No data duplication in serialized output
184//! - **Reference identity preservation**: Deserialized objects maintain the same sharing relationships
185//! - **Circular reference support**: Use `RcWeak<T>` and `ArcWeak<T>` to break cycles
186//! - **Forward reference resolution**: Callbacks handle weak pointers appearing before targets
187//!
188//! #### Shared References with Rc/Arc
189//!
190//! ```rust
191//! use fory::Fory;
192//! use fory::Error;
193//! use std::rc::Rc;
194//!
195//! # fn main() -> Result<(), Error> {
196//! let fory = Fory::default();
197//!
198//! let shared = Rc::new(String::from("shared_value"));
199//! let data = vec![shared.clone(), shared.clone(), shared.clone()];
200//!
201//! let bytes = fory.serialize(&data)?;
202//! let decoded: Vec<Rc<String>> = fory.deserialize(&bytes)?;
203//!
204//! assert_eq!(decoded.len(), 3);
205//! assert!(Rc::ptr_eq(&decoded[0], &decoded[1]));
206//! assert!(Rc::ptr_eq(&decoded[1], &decoded[2]));
207//! # Ok(())
208//! # }
209//! ```
210//!
211//! For thread-safe shared references, use `Arc<T>`:
212//!
213//! ```rust
214//! use fory::Fory;
215//! use fory::Error;
216//! use std::sync::Arc;
217//!
218//! # fn main() -> Result<(), Error> {
219//! let fory = Fory::default();
220//! let shared = Arc::new(String::from("shared_value"));
221//! let data = vec![shared.clone(), shared.clone()];
222//!
223//! let bytes = fory.serialize(&data)?;
224//! let decoded: Vec<Arc<String>> = fory.deserialize(&bytes)?;
225//!
226//! assert!(Arc::ptr_eq(&decoded[0], &decoded[1]));
227//! # Ok(())
228//! # }
229//! ```
230//!
231//! #### Circular References with Weak Pointers
232//!
233//! **How it works:**
234//!
235//! - Weak pointers serialize as references to their target objects
236//! - If the strong pointer has been dropped, weak serializes as `Null`
237//! - Forward references (weak appearing before target) are resolved via callbacks
238//! - All clones of a weak pointer share the same internal cell for automatic updates
239//!
240//! ```rust
241//! use fory::{Fory, Error, RcWeak};
242//! use fory::ForyObject;
243//! use std::rc::Rc;
244//! use std::cell::RefCell;
245//!
246//! #[derive(ForyObject, Debug)]
247//! struct Node {
248//! value: i32,
249//! parent: RcWeak<RefCell<Node>>,
250//! children: Vec<Rc<RefCell<Node>>>,
251//! }
252//!
253//! # fn main() -> Result<(), Error> {
254//! let mut fory = Fory::default().track_ref(true);
255//! fory.register::<Node>(2000);
256//!
257//! let parent = Rc::new(RefCell::new(Node {
258//! value: 1,
259//! parent: RcWeak::new(),
260//! children: vec![],
261//! }));
262//!
263//! let child1 = Rc::new(RefCell::new(Node {
264//! value: 2,
265//! parent: RcWeak::from(&parent),
266//! children: vec![],
267//! }));
268//!
269//! parent.borrow_mut().children.push(child1.clone());
270//!
271//! let bytes = fory.serialize(&parent)?;
272//! let decoded: Rc<RefCell<Node>> = fory.deserialize(&bytes)?;
273//!
274//! assert_eq!(decoded.borrow().children.len(), 1);
275//! let upgraded_parent = decoded.borrow().children[0].borrow().parent.upgrade().unwrap();
276//! assert!(Rc::ptr_eq(&decoded, &upgraded_parent));
277//! # Ok(())
278//! # }
279//! ```
280//!
281//! **Thread-Safe Circular Graphs with Arc:**
282//!
283//! ```rust
284//! use fory::{Fory, Error, ArcWeak};
285//! use fory::ForyObject;
286//! use std::sync::{Arc, Mutex};
287//!
288//! #[derive(ForyObject)]
289//! struct Node {
290//! val: i32,
291//! parent: ArcWeak<Mutex<Node>>,
292//! children: Vec<Arc<Mutex<Node>>>,
293//! }
294//!
295//! # fn main() -> Result<(), Error> {
296//! let mut fory = Fory::default().track_ref(true);
297//! fory.register::<Node>(6000);
298//!
299//! let parent = Arc::new(Mutex::new(Node {
300//! val: 10,
301//! parent: ArcWeak::new(),
302//! children: vec![],
303//! }));
304//!
305//! let child = Arc::new(Mutex::new(Node {
306//! val: 20,
307//! parent: ArcWeak::from(&parent),
308//! children: vec![],
309//! }));
310//!
311//! parent.lock().unwrap().children.push(child.clone());
312//!
313//! let bytes = fory.serialize(&parent)?;
314//! let decoded: Arc<Mutex<Node>> = fory.deserialize(&bytes)?;
315//!
316//! assert_eq!(decoded.lock().unwrap().children.len(), 1);
317//! # Ok(())
318//! # }
319//! ```
320//!
321//! ### 3. Trait Object Serialization
322//!
323//! **What it does:** Enables polymorphic serialization through trait objects, supporting
324//! dynamic dispatch and type flexibility.
325//!
326//! **Why it matters:** Rust's trait system is powerful for abstraction, but serializing
327//! `Box<dyn Trait>` is notoriously difficult. This feature is essential for plugin systems,
328//! heterogeneous collections, and extensible architectures.
329//!
330//! **Technical approach:** The `register_trait_type!` macro generates type registration
331//! and dispatch code for trait implementations. During serialization, type IDs are written
332//! alongside data, enabling correct deserialization to the concrete type.
333//!
334//! **Supported trait object types:**
335//!
336//! - `Box<dyn Trait>` - Owned trait objects
337//! - `Rc<dyn Trait>` - Reference-counted trait objects
338//! - `Arc<dyn Trait>` - Thread-safe reference-counted trait objects
339//! - `Rc<dyn Any>` / `Arc<dyn Any>` - Runtime type dispatch without custom traits
340//! - Collections: `Vec<Box<dyn Trait>>`, `HashMap<K, Box<dyn Trait>>`
341//!
342//! #### Basic Trait Object Serialization
343//!
344//! ```rust
345//! use fory::{Fory, register_trait_type, Serializer, Error};
346//! use fory::ForyObject;
347//!
348//! trait Animal: Serializer {
349//! fn speak(&self) -> String;
350//! fn name(&self) -> &str;
351//! }
352//!
353//! #[derive(ForyObject, Debug)]
354//! struct Dog { name: String, breed: String }
355//!
356//! impl Animal for Dog {
357//! fn speak(&self) -> String { "Woof!".to_string() }
358//! fn name(&self) -> &str { &self.name }
359//! }
360//!
361//! #[derive(ForyObject, Debug)]
362//! struct Cat { name: String, color: String }
363//!
364//! impl Animal for Cat {
365//! fn speak(&self) -> String { "Meow!".to_string() }
366//! fn name(&self) -> &str { &self.name }
367//! }
368//!
369//! register_trait_type!(Animal, Dog, Cat);
370//!
371//! #[derive(ForyObject)]
372//! struct Zoo {
373//! star_animal: Box<dyn Animal>,
374//! }
375//!
376//! # fn main() -> Result<(), Error> {
377//! let mut fory = Fory::default().compatible(true);
378//! fory.register::<Dog>(100);
379//! fory.register::<Cat>(101);
380//! fory.register::<Zoo>(102);
381//!
382//! let zoo = Zoo {
383//! star_animal: Box::new(Dog {
384//! name: "Buddy".to_string(),
385//! breed: "Labrador".to_string(),
386//! }),
387//! };
388//!
389//! let bytes = fory.serialize(&zoo)?;
390//! let decoded: Zoo = fory.deserialize(&bytes)?;
391//!
392//! assert_eq!(decoded.star_animal.name(), "Buddy");
393//! assert_eq!(decoded.star_animal.speak(), "Woof!");
394//! # Ok(())
395//! # }
396//! ```
397//!
398//! #### Serializing `dyn Any` Trait Objects
399//!
400//! **What it does:** Supports serializing `Rc<dyn Any>` and `Arc<dyn Any>` for maximum
401//! runtime type flexibility without defining custom traits.
402//!
403//! **When to use:** Plugin systems, dynamic type handling, or when you need runtime type
404//! dispatch without compile-time trait definitions.
405//!
406//! **Key points:**
407//!
408//! - Works with any type that implements `Serializer`
409//! - Requires downcasting after deserialization to access the concrete type
410//! - Type information is preserved during serialization
411//!
412//! ```rust
413//! use fory::Fory;
414//! use fory::Error;
415//! use std::rc::Rc;
416//! use std::any::Any;
417//! use fory::ForyObject;
418//!
419//! #[derive(ForyObject)]
420//! struct Dog { name: String }
421//!
422//! # fn main() -> Result<(), Error> {
423//! let mut fory = Fory::default();
424//! fory.register::<Dog>(100);
425//!
426//! let dog: Rc<dyn Any> = Rc::new(Dog {
427//! name: "Rex".to_string()
428//! });
429//!
430//! let bytes = fory.serialize(&dog)?;
431//! let decoded: Rc<dyn Any> = fory.deserialize(&bytes)?;
432//!
433//! let unwrapped = decoded.downcast_ref::<Dog>().unwrap();
434//! assert_eq!(unwrapped.name, "Rex");
435//! # Ok(())
436//! # }
437//! ```
438//!
439//! For thread-safe scenarios, use `Arc<dyn Any>`:
440//!
441//! ```rust
442//! use fory::Fory;
443//! use fory::Error;
444//! use std::sync::Arc;
445//! use std::any::Any;
446//! use fory::ForyObject;
447//!
448//! #[derive(ForyObject)]
449//! struct Cat { name: String }
450//!
451//! # fn main() -> Result<(), Error> {
452//! let mut fory = Fory::default();
453//! fory.register::<Cat>(101);
454//!
455//! let cat: Arc<dyn Any> = Arc::new(Cat {
456//! name: "Whiskers".to_string()
457//! });
458//!
459//! let bytes = fory.serialize(&cat)?;
460//! let decoded: Arc<dyn Any> = fory.deserialize(&bytes)?;
461//!
462//! let unwrapped = decoded.downcast_ref::<Cat>().unwrap();
463//! assert_eq!(unwrapped.name, "Whiskers");
464//! # Ok(())
465//! # }
466//! ```
467//!
468//! #### Rc/Arc-Based Trait Objects in Structs
469//!
470//! For struct fields containing `Rc<dyn Trait>` or `Arc<dyn Trait>`, Apache Fory™
471//! automatically handles the conversion without needing wrappers:
472//!
473//! ```rust
474//! use fory::{Fory, register_trait_type, Serializer, Error};
475//! use fory::ForyObject;
476//! use std::sync::Arc;
477//! use std::rc::Rc;
478//!
479//! trait Animal: Serializer {
480//! fn name(&self) -> &str;
481//! }
482//!
483//! #[derive(ForyObject, Debug)]
484//! struct Dog { name: String }
485//! impl Animal for Dog {
486//! fn name(&self) -> &str { &self.name }
487//! }
488//!
489//! #[derive(ForyObject, Debug)]
490//! struct Cat { name: String }
491//! impl Animal for Cat {
492//! fn name(&self) -> &str { &self.name }
493//! }
494//!
495//! register_trait_type!(Animal, Dog, Cat);
496//!
497//! #[derive(ForyObject)]
498//! struct AnimalShelter {
499//! animals_rc: Vec<Rc<dyn Animal>>,
500//! animals_arc: Vec<Arc<dyn Animal>>,
501//! }
502//!
503//! # fn main() -> Result<(), Error> {
504//! let mut fory = Fory::default().compatible(true);
505//! fory.register::<Dog>(100);
506//! fory.register::<Cat>(101);
507//! fory.register::<AnimalShelter>(102);
508//!
509//! let shelter = AnimalShelter {
510//! animals_rc: vec![
511//! Rc::new(Dog { name: "Rex".to_string() }),
512//! Rc::new(Cat { name: "Mittens".to_string() }),
513//! ],
514//! animals_arc: vec![
515//! Arc::new(Dog { name: "Buddy".to_string() }),
516//! ],
517//! };
518//!
519//! let bytes = fory.serialize(&shelter)?;
520//! let decoded: AnimalShelter = fory.deserialize(&bytes)?;
521//!
522//! assert_eq!(decoded.animals_rc[0].name(), "Rex");
523//! assert_eq!(decoded.animals_arc[0].name(), "Buddy");
524//! # Ok(())
525//! # }
526//! ```
527//!
528//! #### Standalone Trait Object Serialization with Wrappers
529//!
530//! Due to Rust's orphan rule, `Rc<dyn Trait>` and `Arc<dyn Trait>` cannot implement
531//! `Serializer` directly. For standalone serialization (not inside struct fields),
532//! the `register_trait_type!` macro generates wrapper types.
533//!
534//! **Note:** If you don't want to use wrapper types, you can serialize as `Rc<dyn Any>`
535//! or `Arc<dyn Any>` instead (see the `dyn Any` section above).
536//!
537//! The `register_trait_type!` macro generates `AnimalRc` and `AnimalArc` wrapper types:
538//!
539//! ```rust
540//! use fory::{Fory, Error, register_trait_type, Serializer};
541//! use fory::ForyObject;
542//! use std::sync::Arc;
543//! use std::rc::Rc;
544//!
545//! trait Animal: Serializer {
546//! fn name(&self) -> &str;
547//! }
548//!
549//! #[derive(ForyObject, Debug)]
550//! struct Dog { name: String }
551//! impl Animal for Dog {
552//! fn name(&self) -> &str { &self.name }
553//! }
554//!
555//! register_trait_type!(Animal, Dog);
556//!
557//! # fn main() -> Result<(), Error> {
558//! let mut fory = Fory::default().compatible(true);
559//! fory.register::<Dog>(100);
560//!
561//! // For Rc<dyn Trait>
562//! let dog_rc: Rc<dyn Animal> = Rc::new(Dog { name: "Rex".to_string() });
563//! let wrapper = AnimalRc::from(dog_rc);
564//!
565//! let bytes = fory.serialize(&wrapper)?;
566//! let decoded: AnimalRc = fory.deserialize(&bytes)?;
567//!
568//! // Unwrap back to Rc<dyn Animal>
569//! let unwrapped: Rc<dyn Animal> = decoded.unwrap();
570//! assert_eq!(unwrapped.name(), "Rex");
571//!
572//! // For Arc<dyn Trait>
573//! let dog_arc: Arc<dyn Animal> = Arc::new(Dog { name: "Buddy".to_string() });
574//! let wrapper = AnimalArc::from(dog_arc);
575//!
576//! let bytes = fory.serialize(&wrapper)?;
577//! let decoded: AnimalArc = fory.deserialize(&bytes)?;
578//!
579//! let unwrapped: Arc<dyn Animal> = decoded.unwrap();
580//! assert_eq!(unwrapped.name(), "Buddy");
581//! # Ok(())
582//! # }
583//! ```
584//!
585//! ### 4. Schema Evolution
586//!
587//! **What it does:** Supports schema evolution in **Compatible mode**, allowing
588//! serialization and deserialization peers to have different type definitions.
589//!
590//! **Why it matters:** In distributed systems and microservices, different services
591//! evolve independently. Schema evolution enables zero-downtime deployments where
592//! services can be updated gradually without breaking communication.
593//!
594//! **Technical approach:** In Compatible mode, Apache Fory™ includes field names and
595//! type metadata in the serialized data. During deserialization, fields are matched by
596//! name, allowing for additions, deletions, and reordering.
597//!
598//! **Features:**
599//!
600//! - Add new fields with default values
601//! - Remove obsolete fields (skipped during deserialization)
602//! - Change field nullability (`T` ↔ `Option<T>`)
603//! - Reorder fields (matched by name, not position)
604//! - Type-safe fallback to default values for missing fields
605//!
606//! **Compatibility rules:**
607//!
608//! - Field names must match (case-sensitive)
609//! - Type changes are not supported (except nullable/non-nullable)
610//! - Nested struct types must be registered on both sides
611//!
612//! ```rust
613//! use fory::{Fory, Error};
614//! use fory::ForyObject;
615//! use std::collections::HashMap;
616//!
617//! #[derive(ForyObject, Debug)]
618//! struct PersonV1 {
619//! name: String,
620//! age: i32,
621//! address: String,
622//! }
623//!
624//! #[derive(ForyObject, Debug)]
625//! struct PersonV2 {
626//! name: String,
627//! age: i32,
628//! phone: Option<String>,
629//! metadata: HashMap<String, String>,
630//! }
631//!
632//! # fn main() -> Result<(), Error> {
633//! let mut fory1 = Fory::default().compatible(true);
634//! fory1.register::<PersonV1>(1);
635//!
636//! let mut fory2 = Fory::default().compatible(true);
637//! fory2.register::<PersonV2>(1);
638//!
639//! let person_v1 = PersonV1 {
640//! name: "Alice".to_string(),
641//! age: 30,
642//! address: "123 Main St".to_string(),
643//! };
644//!
645//! let bytes = fory1.serialize(&person_v1)?;
646//! let person_v2: PersonV2 = fory2.deserialize(&bytes)?;
647//!
648//! assert_eq!(person_v2.name, "Alice");
649//! assert_eq!(person_v2.age, 30);
650//! assert_eq!(person_v2.phone, None);
651//! # Ok(())
652//! # }
653//! ```
654//!
655//! ### 5. Enum Support
656//!
657//! **What it does:** Comprehensive enum support with three variant types (unit, unnamed, named)
658//! and full schema evolution in Compatible mode.
659//!
660//! **Why it matters:** Enums are essential for state machines, status codes, type discriminators,
661//! and domain modeling. Supporting all variant types with schema evolution enables flexible API
662//! evolution without breaking compatibility.
663//!
664//! **Technical approach:** Each variant is assigned an ordinal value (0, 1, 2, ...). In compatible
665//! mode, variants are encoded with both a tag (ordinal) and a type marker (2 bits: 0b0=Unit,
666//! 0b1=Unnamed, 0b10=Named). Named variants generate meta types for field-level evolution.
667//!
668//! **Variant Types:**
669//!
670//! - **Unit**: C-style enums (`Status::Active`)
671//! - **Unnamed**: Tuple-like variants (`Message::Pair(String, i32)`)
672//! - **Named**: Struct-like variants (`Event::Click { x: i32, y: i32 }`)
673//!
674//! **Features:**
675//!
676//! - Efficient varint encoding for variant ordinals
677//! - Schema evolution support (add/remove variants, add/remove fields)
678//! - Default variant support with `#[default]`
679//! - Automatic type mismatch handling
680//!
681//! ```rust
682//! use fory::Fory;
683//! use fory::Error;
684//! use fory::ForyObject;
685//!
686//! #[derive(Default, ForyObject, Debug, PartialEq)]
687//! enum Value {
688//! #[default]
689//! Null,
690//! Bool(bool),
691//! Number(f64),
692//! Text(String),
693//! Object { name: String, value: i32 },
694//! }
695//!
696//! # fn main() -> Result<(), Error> {
697//! let mut fory = Fory::default();
698//! fory.register::<Value>(1)?;
699//!
700//! let value = Value::Object { name: "score".to_string(), value: 100 };
701//! let bytes = fory.serialize(&value)?;
702//! let decoded: Value = fory.deserialize(&bytes)?;
703//! assert_eq!(value, decoded);
704//! # Ok(())
705//! # }
706//! ```
707//!
708//! **Schema Evolution:**
709//!
710//! Compatible mode enables robust schema evolution with variant type encoding:
711//!
712//! ```rust
713//! use fory::Fory;
714//! use fory::Error;
715//! use fory::ForyObject;
716//!
717//! // Old version with 2 fields
718//! #[derive(ForyObject, Debug)]
719//! enum OldEvent {
720//! Click { x: i32, y: i32 },
721//! }
722//!
723//! // New version with 3 fields - added timestamp
724//! #[derive(ForyObject, Debug)]
725//! enum NewEvent {
726//! Click { x: i32, y: i32, timestamp: u64 },
727//! }
728//!
729//! # fn main() -> Result<(), Error> {
730//! let mut fory_old = Fory::default().compatible(true);
731//! fory_old.register::<OldEvent>(5)?;
732//!
733//! let mut fory_new = Fory::default().compatible(true);
734//! fory_new.register::<NewEvent>(5)?;
735//!
736//! // Serialize with old schema (2 fields)
737//! let old_bytes = fory_old.serialize(&OldEvent::Click { x: 100, y: 200 })?;
738//!
739//! // Deserialize with new schema (3 fields) - timestamp gets default value (0)
740//! let new_event: NewEvent = fory_new.deserialize(&old_bytes)?;
741//! match new_event {
742//! NewEvent::Click { x, y, timestamp } => {
743//! assert_eq!(x, 100);
744//! assert_eq!(y, 200);
745//! assert_eq!(timestamp, 0); // Default value for missing field
746//! }
747//! }
748//! # Ok(())
749//! # }
750//! ```
751//!
752//! **Evolution capabilities:**
753//!
754//! - Unknown variants fall back to default variant
755//! - Named variant fields: add/remove fields (missing fields use defaults)
756//! - Unnamed variant elements: add/remove elements (extras skipped, missing use defaults)
757//! - Variant type mismatches automatically use default value of current variant
758//!
759//! ### 6. Tuple Support
760//!
761//! **What it does:** Supports tuples up to 22 elements with automatic heterogeneous type
762//! handling and schema evolution in compatible mode.
763//!
764//! **Why it matters:** Tuples provide lightweight aggregation without defining full structs,
765//! useful for temporary groupings, function return values, and ad-hoc data structures.
766//!
767//! **Technical approach:** Each tuple size (1-22) has a specialized `Serializer` implementation.
768//! In non-compatible mode, elements are serialized sequentially without overhead. In compatible
769//! mode, the tuple is serialized as a heterogeneous collection with type metadata for each element.
770//!
771//! **Features:**
772//!
773//! - Automatic serialization for tuples from 1 to 22 elements
774//! - Heterogeneous type support (each element can be a different type)
775//! - Schema evolution in Compatible mode (handles missing/extra elements)
776//! - Default values for missing elements during deserialization
777//!
778//! ```rust
779//! use fory::Fory;
780//! use fory::Error;
781//!
782//! # fn main() -> Result<(), Error> {
783//! let mut fory = Fory::default();
784//!
785//! // Tuple with heterogeneous types
786//! let data: (i32, String, bool, Vec<i32>) = (
787//! 42,
788//! "hello".to_string(),
789//! true,
790//! vec![1, 2, 3],
791//! );
792//!
793//! let bytes = fory.serialize(&data)?;
794//! let decoded: (i32, String, bool, Vec<i32>) = fory.deserialize(&bytes)?;
795//! assert_eq!(data, decoded);
796//! # Ok(())
797//! # }
798//! ```
799//!
800//! ### 7. Custom Serializers
801//!
802//! **What it does:** Allows manual implementation of the `Serializer` trait for types
803//! that don't support `#[derive(ForyObject)]`.
804//!
805//! **When to use:**
806//!
807//! - External types from other crates that you can't modify
808//! - Types with special serialization requirements
809//! - Legacy data format compatibility
810//! - Performance-critical custom encoding
811//! - Complex types that require special handling
812//!
813//! **Technical approach:** Implement the `Serializer` trait's `fory_write_data()` and
814//! `fory_read_data()` methods to control exactly how data is written to and read from
815//! the binary buffer.
816//!
817//! ```rust
818//! use fory::{Fory, TypeResolver, ReadContext, WriteContext, Serializer, ForyDefault, Error};
819//! use std::any::Any;
820//!
821//! #[derive(Debug, PartialEq, Default)]
822//! struct CustomType {
823//! value: i32,
824//! name: String,
825//! }
826//!
827//! impl Serializer for CustomType {
828//! fn fory_write_data(&self, context: &mut WriteContext) -> Result<(), Error> {
829//! context.writer.write_i32(self.value);
830//! context.writer.write_var_uint32(self.name.len() as u32);
831//! context.writer.write_utf8_string(&self.name);
832//! Ok(())
833//! }
834//!
835//! fn fory_read_data(context: &mut ReadContext) -> Result<Self, Error> {
836//! let value = context.reader.read_i32()?;
837//! let len = context.reader.read_varuint32()? as usize;
838//! let name = context.reader.read_utf8_string(len)?;
839//! Ok(Self { value, name })
840//! }
841//!
842//! fn fory_type_id_dyn(&self, type_resolver: &TypeResolver) -> Result<fory::TypeId, Error> {
843//! Self::fory_get_type_id(type_resolver)
844//! }
845//!
846//! fn as_any(&self) -> &dyn Any {
847//! self
848//! }
849//! }
850//!
851//! impl ForyDefault for CustomType {
852//! fn fory_default() -> Self {
853//! Self::default()
854//! }
855//! }
856//!
857//! # fn main() -> Result<(), Error> {
858//! let mut fory = Fory::default();
859//! fory.register_serializer::<CustomType>(100);
860//!
861//! let custom = CustomType {
862//! value: 42,
863//! name: "test".to_string(),
864//! };
865//! let bytes = fory.serialize(&custom)?;
866//! let decoded: CustomType = fory.deserialize(&bytes)?;
867//! assert_eq!(custom, decoded);
868//! # Ok(())
869//! # }
870//! ```
871//!
872//! ### 7. Row-Based Serialization
873//!
874//! **What it does:** Provides a high-performance **row format** for zero-copy
875//! deserialization, enabling random access to fields directly from binary data
876//! without full object reconstruction.
877//!
878//! **Why it matters:** Traditional serialization reconstructs entire objects in memory.
879//! For analytics workloads or when you only need a few fields from large objects,
880//! this is wasteful. Row format provides O(1) field access without deserialization.
881//!
882//! **Technical approach:** Fields are encoded in a binary row with fixed offsets for
883//! primitives. Variable-length data (strings, collections) are stored with offset
884//! pointers. A null bitmap tracks which fields are present. The generated code provides
885//! accessor methods that read directly from the binary buffer.
886//!
887//! **Key benefits:**
888//!
889//! - **Zero-copy access**: Read fields without allocating or copying data
890//! - **Partial deserialization**: Access only the fields you need
891//! - **Memory-mapped files**: Work with data larger than RAM
892//! - **Cache-friendly**: Sequential memory layout for better CPU cache utilization
893//! - **Lazy evaluation**: Defer expensive operations until field access
894//!
895//! **When to use row format:**
896//!
897//! - Analytics workloads with selective field access
898//! - Large datasets where only a subset of fields is needed
899//! - Memory-constrained environments
900//! - High-throughput data pipelines
901//! - Reading from memory-mapped files or shared memory
902//!
903//! **Performance characteristics:**
904//!
905//! | Operation | Object Format | Row Format |
906//! |----------------------|-------------------------------|---------------------------------|
907//! | Full deserialization | Allocates all objects | Zero allocation |
908//! | Single field access | Full deserialization required | Direct offset read (O(1)) |
909//! | Memory usage | Full object graph in memory | Only accessed fields in memory |
910//! | Suitable for | Small objects, full access | Large objects, selective access |
911//!
912//! ```rust
913//! use fory::{to_row, from_row};
914//! use fory_derive::ForyRow;
915//! use std::collections::BTreeMap;
916//!
917//! #[derive(ForyRow)]
918//! struct UserProfile {
919//! id: i64,
920//! username: String,
921//! email: String,
922//! scores: Vec<i32>,
923//! preferences: BTreeMap<String, String>,
924//! is_active: bool,
925//! }
926//!
927//! # fn main() {
928//! let profile = UserProfile {
929//! id: 12345,
930//! username: "alice".to_string(),
931//! email: "alice@example.com".to_string(),
932//! scores: vec![95, 87, 92, 88],
933//! preferences: BTreeMap::from([
934//! ("theme".to_string(), "dark".to_string()),
935//! ("language".to_string(), "en".to_string()),
936//! ]),
937//! is_active: true,
938//! };
939//!
940//! let row_data = to_row(&profile).unwrap();
941//! let row = from_row::<UserProfile>(&row_data);
942//!
943//! assert_eq!(row.id(), 12345);
944//! assert_eq!(row.username(), "alice");
945//! assert_eq!(row.is_active(), true);
946//!
947//! let scores = row.scores();
948//! assert_eq!(scores.size(), 4);
949//! assert_eq!(scores.get(0).unwrap(), 95);
950//! # }
951//! ```
952//!
953//! ## Supported Types
954//!
955//! Apache Fory™ supports a comprehensive type system for maximum flexibility.
956//!
957//! ### Primitive Types
958//!
959//! - `bool` - Boolean values
960//! - `i8`, `i16`, `i32`, `i64` - Signed integers
961//! - `f32`, `f64` - Floating point numbers
962//! - `String` - UTF-8 encoded strings
963//!
964//! ### Collections
965//!
966//! - `Vec<T>` - Dynamic arrays
967//! - `HashMap<K, V>` - Hash-based maps
968//! - `BTreeMap<K, V>` - Ordered maps
969//! - `HashSet<T>` - Hash-based sets
970//! - `Option<T>` - Optional values
971//!
972//! ### Smart Pointers
973//!
974//! - `Box<T>` - Heap allocation
975//! - `Rc<T>` - Reference counting (shared references tracked automatically)
976//! - `Arc<T>` - Thread-safe reference counting (shared references tracked)
977//! - `RcWeak<T>` - Weak reference to `Rc<T>` (breaks circular references)
978//! - `ArcWeak<T>` - Weak reference to `Arc<T>` (breaks circular references)
979//! - `RefCell<T>` - Interior mutability with runtime borrow checking
980//! - `Mutex<T>` - Thread-safe interior mutability
981//!
982//! ### Date and Time (requires `chrono` feature)
983//!
984//! - `chrono::NaiveDate` - Date without timezone
985//! - `chrono::NaiveDateTime` - Timestamp without timezone
986//!
987//! ### Custom Types
988//!
989//! - Structs with `#[derive(ForyObject)]` - Object graph serialization
990//! - Structs with `#[derive(ForyRow)]` - Row-based serialization
991//! - C-style enums with `#[derive(ForyObject)]` - Enum support
992//! - Manual `Serializer` implementation - Custom serialization logic
993//!
994//! ### Trait Objects
995//!
996//! - `Box<dyn Trait>` - Owned trait objects
997//! - `Rc<dyn Trait>` - Reference-counted trait objects
998//! - `Arc<dyn Trait>` - Thread-safe reference-counted trait objects
999//! - `Rc<dyn Any>` - Runtime type dispatch without custom traits
1000//! - `Arc<dyn Any>` - Thread-safe runtime type dispatch
1001//!
1002//! ## Serialization Modes
1003//!
1004//! Apache Fory™ supports two serialization modes to balance between performance
1005//! and flexibility:
1006//!
1007//! ### SchemaConsistent Mode (Default)
1008//!
1009//! **When to use:** Maximum performance when schemas are guaranteed to match.
1010//!
1011//! **Characteristics:**
1012//! - Type declarations must match exactly between serialization and deserialization
1013//! - Smaller payload size (no field names or metadata)
1014//! - Faster serialization and deserialization
1015//! - Suitable for monolithic applications or tightly coupled services
1016//!
1017//! ```rust
1018//! use fory::Fory;
1019//!
1020//! let fory = Fory::default();
1021//! ```
1022//!
1023//! ### Compatible Mode
1024//!
1025//! **When to use:** Schema evolution in distributed systems or microservices.
1026//!
1027//! **Characteristics:**
1028//! - Type declarations can differ between peers
1029//! - Allows field additions, deletions, and reordering
1030//! - Larger payload size (includes field names and metadata)
1031//! - Slightly slower due to metadata processing
1032//! - Essential for zero-downtime deployments
1033//!
1034//! ```rust
1035//! use fory::Fory;
1036//!
1037//! let fory = Fory::default().compatible(true);
1038//! ```
1039//!
1040//! ## Cross-Language Serialization
1041//!
1042//! **What it enables:** Seamless data exchange across Java, Python, C++, Go,
1043//! JavaScript, and Rust implementations.
1044//!
1045//! **Why it matters:** Microservices architectures often use multiple languages.
1046//! Apache Fory™ provides a common binary protocol without IDL files or code generation.
1047//!
1048//! **How to enable:**
1049//!
1050//! ```rust
1051//! use fory::Fory;
1052//! use fory::ForyObject;
1053//!
1054//! let mut fory = Fory::default()
1055//! .compatible(true)
1056//! .xlang(true);
1057//!
1058//! #[derive(ForyObject)]
1059//! struct MyStruct {
1060//! field1: i32,
1061//! field2: String,
1062//! }
1063//!
1064//! fory.register_by_namespace::<MyStruct>("com.example", "MyStruct");
1065//! ```
1066//!
1067//! **Type registration strategies:**
1068//!
1069//! - **ID-based registration**: `fory.register::<T>(id)` - Fastest, requires coordination
1070//! - **Namespace-based registration**: `fory.register_by_namespace::<T>(namespace, name)` - Automatic cross-language mapping
1071//!
1072//! ## Performance Characteristics
1073//!
1074//! Apache Fory™ Rust is designed for maximum performance through multiple techniques:
1075//!
1076//! **Compile-time code generation:**
1077//! - Procedural macros generate specialized serialization code
1078//! - Zero runtime overhead, no reflection
1079//! - Monomorphization for type-specific optimizations
1080//!
1081//! **Zero-copy techniques:**
1082//! - Row format enables direct memory access
1083//! - No intermediate object allocation
1084//! - Memory-mapped file support
1085//!
1086//! **Space efficiency:**
1087//! - Variable-length integer encoding
1088//! - Reference deduplication (shared objects serialized once)
1089//! - Compact binary format
1090//!
1091//! **Buffer management:**
1092//! - Pre-allocation based on `fory_reserved_space()` hints
1093//! - Minimized reallocations
1094//! - Little-endian layout for modern CPUs
1095//!
1096//! ## Error Handling
1097//!
1098//! Apache Fory™ uses `Result<T, Error>` for all fallible operations, providing
1099//! comprehensive error handling:
1100//!
1101//! ```rust
1102//! use fory::{Fory, Error};
1103//! use fory::ForyObject;
1104//!
1105//! #[derive(ForyObject)]
1106//! struct Data {
1107//! value: i32,
1108//! }
1109//!
1110//! fn process_data(bytes: &[u8]) -> Result<Data, Error> {
1111//! let mut fory = Fory::default();
1112//! fory.register::<Data>(100);
1113//!
1114//! let data: Data = fory.deserialize(bytes)?;
1115//! Ok(data)
1116//! }
1117//! ```
1118//!
1119//! ## Thread Safety
1120//!
1121//! `Fory` implements `Send` and `Sync`, so a single instance can be shared across threads
1122//! (for example via `Arc<Fory>`) for concurrent serialization and deserialization. The
1123//! internal context pools grow lazily and rely on thread-safe primitives, allowing multiple
1124//! workers to reuse buffers without additional coordination.
1125//!
1126//! ```rust
1127//! use std::sync::Arc;
1128//! use std::thread;
1129//! use fory::Fory;
1130//! use fory::ForyObject;
1131//!
1132//! #[derive(ForyObject, Clone, Copy, Debug)]
1133//! struct Item {
1134//! value: i32,
1135//! }
1136//!
1137//! let mut fory = Fory::default();
1138//! fory.register::<Item>(1000).unwrap();
1139//! let fory = Arc::new(fory);
1140//! let handles: Vec<_> = (0..8)
1141//! .map(|i| {
1142//! let shared = Arc::clone(&fory);
1143//! thread::spawn(move || {
1144//! let item = Item { value: i };
1145//! shared.serialize(&item).unwrap()
1146//! })
1147//! })
1148//! .collect();
1149//!
1150//! for handle in handles {
1151//! let bytes = handle.join().unwrap();
1152//! let item: Item = fory.deserialize(&bytes).unwrap();
1153//! assert!(item.value >= 0);
1154//! }
1155//! ```
1156//!
1157//! **Best practice:** Perform type registration (e.g., `fory.register::<T>(id)`) before
1158//! spawning worker threads so metadata is ready, then share the configured instance.
1159//!
1160//! ## Examples
1161//!
1162//! See the `tests/` directory for comprehensive examples:
1163//!
1164//! - `tests/tests/test_complex_struct.rs` - Complex nested structures
1165//! - `tests/tests/test_rc_arc_trait_object.rs` - Trait object serialization
1166//! - `tests/tests/test_weak.rs` - Circular reference handling
1167//! - `tests/tests/test_cross_language.rs` - Cross-language compatibility
1168//!
1169//! ## Troubleshooting
1170//!
1171//! - **Type registry errors**: Errors such as `TypeId ... not found in type_info registry` mean
1172//! the type was never registered with the active `Fory` instance. Ensure every serializable
1173//! struct, enum, or trait implementation calls `register::<T>(type_id)` before use, and reuse
1174//! the same IDs when deserializing.
1175//! - **Quick error lookup**: Always prefer the static constructors on
1176//! [`fory_core::error::Error`]—for example `Error::type_mismatch`, `Error::invalid_data`, or
1177//! `Error::unknown`. They keep diagnostics consistent and allow optional panic-on-error
1178//! debugging.
1179//! - **Panic on error for backtraces**: Set `FORY_PANIC_ON_ERROR=1` (or `true`) together with
1180//! `RUST_BACKTRACE=1` while running tests or binaries to panic exactly where an error is
1181//! constructed. Unset the variable afterwards so production paths keep returning `Result`.
1182//! - **Struct field tracing**: Add the `#[fory(debug)]` attribute (or `#[fory(debug = true)]`)
1183//! next to `#[derive(ForyObject)]` when you need per-field instrumentation. Once compiled with
1184//! debug hooks, call `set_before_write_field_func`, `set_after_write_field_func`,
1185//! `set_before_read_field_func`, or
1186//! `set_after_read_field_func` from `fory_core::serializer::struct_` to install custom
1187//! callbacks, and use `reset_struct_debug_hooks()` to restore defaults.
1188//! - **Lightweight logging**: If custom callbacks are unnecessary, enable
1189//! `ENABLE_FORY_DEBUG_OUTPUT=1` to have the default hook handlers print field-level read/write
1190//! events. This is useful for spotting cursor misalignment or unexpected buffer growth.
1191//! - **Test-time hygiene**: Some integration tests expect `FORY_PANIC_ON_ERROR` to stay unset.
1192//! Export it only during focused debugging, and rely on targeted commands such as
1193//! `cargo test --features tests -p tests --test <case>` when isolating failures.
1194//!
1195//! ## Documentation
1196//!
1197//! - **[Protocol Specification](https://fory.apache.org/docs/specification/fory_xlang_serialization_spec)** - Binary protocol details
1198//! - **[Row Format Specification](https://fory.apache.org/docs/specification/fory_row_format_spec)** - Row format internals
1199//! - **[Type Mapping](https://fory.apache.org/docs/guide/xlang_type_mapping)** - Cross-language type mappings
1200//! - **[API Documentation](https://docs.rs/fory)** - Complete API reference
1201//! - **[GitHub Repository](https://github.com/apache/fory)** - Source code and issue tracking
1202
1203pub use fory_core::{
1204 error::Error, fory::Fory, register_trait_type, row::from_row, row::to_row, types::TypeId,
1205 ArcWeak, ForyDefault, RcWeak, ReadContext, Reader, Serializer, TypeResolver, WriteContext,
1206 Writer,
1207};
1208pub use fory_derive::{ForyObject, ForyRow};