interoptopus/wire/mod.rs
1//! Serialize complex objects into flat byte buffers and transfer them over FFI.
2//!
3//! Types like `String`, `Vec<T>`, `HashMap<K, V>`, and structs containing them cannot be
4//! passed directly over an FFI boundary. Their in-memory form is a small header (pointer,
5//! length, capacity) into a Rust-managed heap allocation — the foreign side cannot read,
6//! resize, or free that memory. `Wire<T>` solves this by serializing the value into a flat
7//! byte buffer on one side and deserializing it on the other.
8//!
9//! # Examples
10//!
11//! ### Accepting a complex argument
12//!
13//! ```
14//! use interoptopus::ffi;
15//! use interoptopus::wire::Wire;
16//! use std::collections::HashMap;
17//!
18//! #[ffi]
19//! pub fn lookup(mut map: Wire<HashMap<String, String>>) -> u32 {
20//! let map = map.unwire();
21//! map.len() as u32
22//! }
23//! ```
24//!
25//! ### Returning a complex value
26//!
27//! ```
28//! use interoptopus::ffi;
29//! use interoptopus::wire::Wire;
30//!
31//! #[ffi]
32//! pub fn greeting() -> Wire<String> {
33//! Wire::from("hello".to_string())
34//! }
35//! ```
36//!
37//! ### Structs containing non-FFI types
38//!
39//! Any `#[ffi]` struct whose fields are not all `repr(C)` can be wrapped in
40//! `Wire<T>`. The proc macro generates matching serialization code on both
41//! sides.
42//!
43//! ```
44//! use interoptopus::ffi;
45//! use interoptopus::wire::Wire;
46//!
47//! #[ffi]
48//! pub struct UserProfile {
49//! pub name: String,
50//! pub tags: Vec<String>,
51//! }
52//!
53//! #[ffi]
54//! pub fn accept_profile(mut profile: Wire<UserProfile>) {
55//! let profile = profile.unwire();
56//! println!("{}: {:?}", profile.name, profile.tags);
57//! }
58//! ```
59//!
60//! ### Deeply nested types
61//!
62//! `Wire<T>` handles arbitrarily nested structures, including `Vec`, `HashMap`,
63//! and `Option` at any depth:
64//!
65//! ```
66//! use interoptopus::ffi;
67//! use interoptopus::wire::Wire;
68//! use std::collections::HashMap;
69//!
70//! #[ffi]
71//! pub struct Inner { pub score: u32 }
72//!
73//! #[ffi]
74//! pub struct Outer {
75//! pub items: HashMap<u32, Vec<Inner>>,
76//! }
77//!
78//! #[ffi]
79//! pub fn process(mut data: Wire<Outer>) -> u32 {
80//! let data = data.unwire();
81//! data.items.values().flatten().map(|i| i.score).sum()
82//! }
83//! ```
84//!
85//! ### Registering the helpers
86//!
87//! Every crate that uses `Wire<T>` must call [`builtins_wire!`](crate::builtins_wire)
88//! in its inventory so the create/destroy helpers are available to the foreign side:
89//!
90//! ```
91//! use interoptopus::ffi;
92//! use interoptopus::wire::Wire;
93//! use interoptopus::{builtins_wire, function};
94//! use interoptopus::inventory::RustInventory;
95//!
96//! #[ffi]
97//! pub fn greeting() -> Wire<String> {
98//! Wire::from("hello".to_string())
99//! }
100//!
101//! pub fn ffi_inventory() -> RustInventory {
102//! RustInventory::new()
103//! .register(function!(greeting))
104//! .register(builtins_wire!())
105//! .validate()
106//! }
107//! ```
108//!
109//! # Wire vs. Protobuf
110//!
111//! The natural alternative to `Wire<T>` for passing complex 'variably-sized' types over FFI is
112//! [Protocol Buffers](https://protobuf.dev/). Protobuf works, but it comes with significant
113//! friction: you need to maintain `.proto` schema files alongside your Rust types, install and
114//! run an external code generator as part of your build, integrate that generator into both the Rust
115//! and the foreign-language project, and keep all three in sync whenever a type changes.
116//! The result is a more complex project setup with more moving parts — and you still have to
117//! wire the generated types into your FFI layer by hand.
118//!
119//! `Wire<T>` eliminates all of that. Types are defined once in Rust with `#[ffi]`, and both
120//! the serialization logic and the foreign-language deserialization code are generated
121//! automatically as part of the normal interoptopus build. There are no `.proto` files, no
122//! external tools, and no schema drift.
123//!
124//! Beyond ergonomics, `Wire<T>` is in most cases also faster. Because both sides share the
125//! exact same compiled type layout, there is no field-tag overhead, no varint encoding, and
126//! no dynamic dispatch — just a straight sequential read/write of the exact bytes needed.
127//! In our benchmarks, `Wire<T>` usually outperformed Protobuf by roughly 20–200%
128//! depending on the payload shape.
129//!
130//! 
131//!
132//! Note, in the benchmarks above, Protobuf was given a slight advantage over `Wire<T>` by not having to
133//! FFI allocate. This made Protobuf's performance look slightly better, but would make it unsuitable for
134//! `async` use.
135//!
136//! # Under the Hood
137//!
138//! A [`Wire<T>`] is essentially a serialized buffer that is safe to pass through
139//! FFI boundaries.
140//!
141//! ## Rust -> Foreign
142//!
143//! 1. **Serialize** — [`Wire::from`] (or [`Wire::try_from`]) serializes the value into a new Rust-allocated buffer.
144//! 2. **Transfer** — the `Wire<T>` is returned from an `#[ffi]` function; as a `repr(C)` struct it crosses the FFI boundary by value.
145//! 3. **Deserialize** — the foreign side (e.g., C#) reads the buffer bytes and reconstructs the managed type.
146//! 4. **Free** — the foreign side calls `Dispose()` or similar on the wire object, which invokes `interoptopus_wire_destroy`
147//! (emitted by `builtins_wire!`) to drop the Rust-allocated buffer.
148//!
149//! ## Foreign -> Rust
150//!
151//! 1. **Allocate** — the generated `WireOf*.From(value)` helper calls `interoptopus_wire_create` (emitted by
152//! `builtins_wire!`) so that Rust allocates the buffer; the foreign side never allocates directly.
153//! 2. **Serialize** — the value is serialized into that Rust-allocated buffer.
154//! 3. **Transfer** — the `Wire<T>` is passed into an `#[ffi]` function. Rust receives ownership.
155//! 4. **Deserialize** — [`Wire::unwire`] or [`Wire::try_unwire`] reads `T` from the buffer.
156//! 5. **Free** — Rust drops the `Wire<T>` when the function returns, freeing the buffer.
157//!
158//!
159//! ## Wire format
160//!
161//! All values are written in **little-endian** byte order, sequentially, with no padding
162//! or alignment between fields:
163//!
164//! | Type | Format |
165//! |---|---|
166//! | `u8`..`u64`, `i8`..`i64`, `f32`, `f64` | Fixed-size little-endian bytes |
167//! | `usize` / `isize` | Platform-width little-endian (8 bytes on 64-bit) |
168//! | `bool` | 1 byte (`0x00` = false, non-zero = true) |
169//! | `String` | `u32` byte-length (LE), then UTF-8 bytes |
170//! | `Vec<T>` | `u32` element count (LE), then each element serialized in order |
171//! | `HashMap<K,V>` | `u32` entry count (LE), then each key followed by value |
172//! | `(A, B, …)` | Each element serialized in order |
173//! | User structs | Each field serialized in declaration order |
174//!
175//! The wire format is not self-describing, both sides must agree on the exact type
176//! layout.
177//!
178//! **Note:** This section describes an internal implementation detail that may change
179//! between versions without notice. Do not rely on it for persistent storage or
180//! cross-version compatibility.
181mod buffer;
182
183use crate::bad_wire;
184use crate::inventory::{Inventory, TypeId};
185use crate::lang::meta::{Docs, Visibility, common_or_module_emission};
186use crate::lang::types::{Type, TypeInfo, TypeKind, TypePattern, WireIO};
187use buffer::WireBuffer;
188use std::marker::PhantomData;
189
190/// Wraps and transfers complex objects over FFI.
191///
192/// The backing storage uses a (ptr, size) representation that can safely cross
193/// FFI boundaries. See the [module documentation](crate::wire) for more details and examples.
194#[repr(C)]
195pub struct Wire<T>
196where
197 T: ?Sized,
198{
199 buf: WireBuffer,
200 _phantom: PhantomData<T>,
201}
202
203impl<T: TypeInfo + WireIO> Wire<T> {
204 /// Serialize `value` into a new owned [`Wire`].
205 ///
206 /// # Errors
207 /// Returns [`SerializationError`] if `value` cannot be serialized into the buffer.
208 pub fn try_from(value: T) -> Result<Self, SerializationError> {
209 let size = value.live_size();
210 let mut buf = WireBuffer::with_size(size);
211 value.write(&mut buf.writer())?;
212 Ok(Self { buf, _phantom: PhantomData })
213 }
214
215 /// Serialize `value` into a new owned [`Wire`].
216 ///
217 /// # Panics
218 /// Panics at compile time if `T::WIRE_SAFE` is false.
219 pub fn from(value: T) -> Self {
220 const { assert!(T::WIRE_SAFE) }
221 let size = value.live_size();
222 let mut buf = WireBuffer::with_size(size);
223 value.write(&mut buf.writer()).expect("Types with T::WIRE_SAFE must be wirable!");
224 Self { buf, _phantom: PhantomData }
225 }
226
227 /// Deserialize the value from this Wire's buffer.
228 ///
229 /// # Errors
230 ///
231 /// Returns [`SerializationError`] if the buffer contents cannot be deserialized
232 /// into `T` (e.g., truncated buffer, malformed data).
233 pub fn try_unwire(&mut self) -> Result<T, SerializationError> {
234 T::read(&mut self.buf.reader())
235 }
236
237 /// Deserialize the value from this Wire's buffer.
238 ///
239 /// # Panics
240 ///
241 /// Panics at compile time if `T::WIRE_SAFE` is false.
242 pub fn unwire(&mut self) -> T {
243 const { assert!(T::WIRE_SAFE) }
244 T::read(&mut self.buf.reader()).expect("Types with T::WIRE_SAFE must be un-wirable!")
245 }
246}
247
248unsafe impl<T: TypeInfo + WireIO> TypeInfo for Wire<T> {
249 const WIRE_SAFE: bool = false;
250 const RAW_SAFE: bool = true;
251 const ASYNC_SAFE: bool = true;
252 const SERVICE_SAFE: bool = true;
253 const SERVICE_CTOR_SAFE: bool = true;
254
255 fn id() -> TypeId {
256 TypeId::new(0xE9EF32647BF9C7A70889DC642B63FAC9).derive_id(T::id())
257 }
258
259 fn kind() -> TypeKind {
260 TypeKind::TypePattern(TypePattern::Wire(T::id()))
261 }
262
263 fn ty() -> Type {
264 let t = T::ty();
265 Type {
266 name: format!("Wire<{}>", t.name),
267 visibility: Visibility::Public,
268 docs: Docs::empty(),
269 emission: common_or_module_emission(&[t.emission]),
270 kind: Self::kind(),
271 }
272 }
273
274 fn register(inventory: &mut impl Inventory) {
275 T::register(inventory);
276 inventory.register_type(Self::id(), Self::ty());
277 }
278}
279
280unsafe impl<T: WireIO> WireIO for Wire<T> {
281 fn write(&self, _: &mut impl std::io::Write) -> Result<(), SerializationError> {
282 bad_wire!()
283 }
284
285 fn read(_: &mut impl std::io::Read) -> Result<Self, SerializationError> {
286 bad_wire!()
287 }
288
289 fn live_size(&self) -> usize {
290 bad_wire!()
291 }
292}
293
294impl<T: ?Sized> Clone for Wire<T> {
295 fn clone(&self) -> Self {
296 Self { buf: self.buf.clone(), _phantom: PhantomData }
297 }
298}
299
300#[macro_export]
301#[doc(hidden)]
302macro_rules! __wire_create_body {
303 ($size:ident, $out_len:ident, $out_capacity:ident) => {{
304 if $size <= 0 {
305 *$out_len = 0;
306 *$out_capacity = 0;
307 return ::std::ptr::null_mut();
308 }
309 let size = usize::try_from($size).expect("Invalid Wire buffer size");
310 let mut vec: Vec<u8> = vec![0u8; size];
311 let data = vec.as_mut_ptr();
312 *$out_len = i32::try_from(vec.len()).expect("Too large Wire buffer");
313 *$out_capacity = i32::try_from(vec.capacity()).expect("Too large Wire buffer");
314 ::std::mem::forget(vec);
315 data
316 }};
317}
318
319/// Body of `interoptopus_wire_destroy`. Shared by [`builtins_wire!`] and
320/// [`register_wire_trampolines!`].
321#[macro_export]
322#[doc(hidden)]
323macro_rules! __wire_destroy_body {
324 ($data:ident, $len:ident, $capacity:ident) => {{
325 if $capacity <= 0 {
326 return;
327 }
328 let _ = unsafe { Vec::from_raw_parts($data, usize::try_from($len).expect("Invalid vec length"), usize::try_from($capacity).expect("Invalid vec capacity")) };
329 }};
330}
331
332/// Emits and registers helpers for [`Wire<T>`](crate::wire::Wire).
333///
334/// Backends (e.g., C#) use these functions internally so that foreign code can
335/// allocate and free Rust-owned wire buffers.
336///
337/// # Usage
338///
339/// Call once in your inventory function and register the result:
340///
341/// ```rust
342/// # use interoptopus::inventory::RustInventory;
343/// # use interoptopus::builtins_wire;
344/// pub fn inventory() -> RustInventory {
345/// RustInventory::new()
346/// .register(builtins_wire!())
347/// // ... other registrations ...
348/// .validate()
349/// }
350/// ```
351///
352/// # Implementation Details
353///
354/// This macro generates the following FFI functions:
355/// - `interoptopus_wire_create` — allocates a wire buffer of a given size.
356/// - `interoptopus_wire_destroy` — drops a wire buffer, freeing its memory.
357///
358/// Body of `interoptopus_wire_create`. Shared by `builtins_wire!` and
359/// `register_wire_trampolines!`.
360#[macro_export]
361macro_rules! builtins_wire {
362 () => {{
363 #[$crate::ffi(export = unique)]
364 #[allow(clippy::mem_forget)]
365 pub fn interoptopus_wire_create(size: i32, out_len: &mut i32, out_capacity: &mut i32) -> *mut u8 {
366 $crate::__wire_create_body!(size, out_len, out_capacity)
367 }
368
369 #[$crate::ffi(export = unique)]
370 #[allow(clippy::mem_forget)]
371 pub fn interoptopus_wire_destroy(data: *mut u8, len: i32, capacity: i32) {
372 $crate::__wire_destroy_body!(data, len, capacity)
373 }
374
375 |x: &mut $crate::inventory::RustInventory| {
376 <interoptopus_wire_create as $crate::lang::function::FunctionInfo>::register(x);
377 <interoptopus_wire_destroy as $crate::lang::function::FunctionInfo>::register(x);
378 }
379 }};
380}
381
382/// Registers wire buffer trampolines with a foreign plugin.
383///
384/// Defines local `extern "C"` functions (no exported symbols) that share
385/// the same body logic as [`builtins_wire!`], then passes their pointers
386/// to the given register callback.
387///
388/// # Example
389///
390/// ```rust,ignore
391/// interoptopus::register_wire_trampolines!(|id, ptr| {
392/// (plugin.register_trampoline)(id, ptr);
393/// });
394/// ```
395#[doc(hidden)]
396#[macro_export]
397macro_rules! register_wire_trampolines {
398 ($register_fn:expr) => {{
399 extern "C" fn __wire_create(size: i32, out_len: &mut i32, out_capacity: &mut i32) -> *mut u8 {
400 $crate::__wire_create_body!(size, out_len, out_capacity)
401 }
402 extern "C" fn __wire_destroy(data: *mut u8, len: i32, capacity: i32) {
403 $crate::__wire_destroy_body!(data, len, capacity)
404 }
405
406 let __register: &mut dyn FnMut(i64, *const u8) = &mut $register_fn;
407 __register($crate::trampoline::TRAMPOLINE_WIRE_CREATE, __wire_create as *const u8);
408 __register($crate::trampoline::TRAMPOLINE_WIRE_DESTROY, __wire_destroy as *const u8);
409 }};
410}
411
412/// Error returned when a wire-format serialization or deserialization fails.
413#[derive(Debug)]
414pub struct SerializationError {
415 message: String,
416}
417
418impl SerializationError {
419 pub fn new(message: impl Into<String>) -> Self {
420 Self { message: message.into() }
421 }
422
423 #[must_use]
424 pub fn invalid_discriminant(type_name: &str, discriminant: isize) -> Self {
425 Self { message: format!("invalid discriminant for {type_name}: {discriminant}") }
426 }
427}
428
429impl ::std::fmt::Display for SerializationError {
430 fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result {
431 f.write_str(&self.message)
432 }
433}
434
435impl From<::std::io::Error> for SerializationError {
436 fn from(e: ::std::io::Error) -> Self {
437 Self { message: e.to_string() }
438 }
439}
440
441impl From<::std::num::TryFromIntError> for SerializationError {
442 fn from(e: ::std::num::TryFromIntError) -> Self {
443 Self { message: e.to_string() }
444 }
445}