serde_rustler

serde_rustler provides a Serde Serializer and Deserializer for Rustler types, so you can easily serialize and deserialize native Rust types directly to and from native Elixir terms within your NIFs.

Installation

Install from Crates.io:

[dependencies]
serde_rustler = "0.0.3"

API Overview

#[macro_use] extern crate rustler;

use serde::{Serialize, Deserialize}
use serde_rustler::{from_term, to_term};

rustler_export_nifs! {
    "Elixir.SerdeRustlerTests",
    [("nif", 1, nif)],
    None
}

#[derive(Serialize, Deserialize)]
type Animal = ...;

fn nif<'a>(env: Env<'a>, args: &[Term<'a>]) -> NifResult<Term<'a>> {
    // Deserialize term into a native Rust type.
    let animal: Animal = from_term(args[0])?;

    // Serialize a type into an Elixir term.
    to_term(env, animal).map_err(|err| err.into())
}

Usage

Below is a more comprehensive example of how you might use serde_rustler within a rust NIF...

#[macro_use]
extern crate rustler;

use rustler::{Env, error::Error as NifError, NifResult, Term};
use serde::{Serialize, Deserialize};
use serde_rustler::{from_term, to_term};

rustler_export_nifs! {
    "Elixir.SerdeNif",
    [("readme", 1, readme)],
    None
}

// NOTE: to serialize to the correct Elixir record, you MUST tell serde to
// rename the variants to the full Elixir record module atom.
#[derive(Debug, Serialize, Deserialize)]
enum AnimalType {
    #[serde(rename = "Elixir.SerdeNif.AnimalType.Cat")]
    Cat(String),
    #[serde(rename = "Elixir.SerdeNif.AnimalType.Dog")]
    Dog(String),
}

// NOTE: to serialize to an actual Elixir struct (rather than a just map with
// a :__struct__ key), you MUST tell serde to rename the struct to the full
// Elixir struct module atom.
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename = "Elixir.SerdeNif.Animal")]
struct Animal {
    #[serde(rename = "type")]
    _type: AnimalType,
    name: String,
    age: u8,
    owner: Option<String>,
}

fn readme<'a>(env: Env<'a>, args: &[Term<'a>]) -> NifResult<Term<'a>> {
    let animal: Animal = from_term(args[0])?;
    println!("serialized animal: {:?}", animal);
    to_term(env, animal).map_err(|err| err.into())
}

... and how you might structure your corresponding Elixir types (code structure, imports, aliases and requires simplified or omitted for brevity):

defmodule SerdeNif do
  use Rustler, otp_app: :serde_nif

  def readme(_term), do: :erlang.nif_error(:nif_not_loaded)

  defmodule Animal do
    @type t :: %Animal{
      type: Cat.t() | Dog.t(),
      name: bitstring,
      age: pos_integer,
      owner: nil | bitstring
    }
    defstruct type: Cat.record(), name: "", age: 0, owner: nil

    @doc """
    Deserializes term as a Rust `Animal` struct, then serializes it back into
    an Elixir `Animal` struct. Should return true.
    """
    def test() do
      animal = %Animal{
        type: Animal.Cat.record(),
        name: "Garfield",
        age: 41,
      }

      SerdeNif.readme(animal) == animal
    end
  end

  defmodule AnimalType.Cat do
    require Record
    @type t {__MODULE__, String.t()}
    Record.defrecord(:record, __MODULE__, breed: "tabby")
  end

  defmodule AnimalType.Dog do
    # omitted
  end
end

Conversion Table

1: API still being decided / implemented.

2: Requires specifying a specific serialize implementation, such as serde_bytes.

3: When serializing unknown input to terms, atoms will not be created and will instead be replaced with Elixir bitstrings. Therefore "records" will be tuples ({bitstring, ...}) and "structs" will be maps containing %{:__struct__ => bitstring}. The unfortunate consequence of this is that deserialize_any will lack the necessary information needed deserialize many terms without type hints, such as structs, enums and enum variants, and tuples. (Feedback on how best to solve this is very welcome here).

Benchmarks

To run:

cd serde_rustler_tests
MIX_ENV=bench mix run test/benchmarks.exs

Benchmarks were ripped from the Poison repo. The NIFs being called were implemented using serde-transcode to translate between serde_rustler and serde_json and were compiled in :release mode by rustler.

NOTE: If someone can point out any mistakes I made that led to these ridiculous results, please let me know :)

Benchmarks suggest that serde_rustler is somewhat faster than jiffy when encoding JSON, and generally comparable to / no more than ~2-3x as slow as jiffy or jason when decoding JSON, and in almost all cases, serde_rustler seems to use significantly less memory than pure-Elixir alternatives.

Also take note of the results regarding the larger inputs govtrack.json (3.74 MB) and issue-90.json (7.75 MB) - the serde_rustler (dirty) NIFs are identical as before, but were tagged to run in a Dirty CPU Scheduler.

TODO

• finalize behaviour around chars, charlists, iolists, map keys
• still getting used to Rust, so may need to improve error handling and ergnomoics around API
• support for i128 and u128
• more extensive (i.e. possible addition of smoke, property-based) testing

Changelog

Contributing

1. Fork it https://github.com/your_username/serde_rustler/fork
2. Create your feature branch (git checkout -b feature/fooBar)
3. Commit your changes (git commit -am 'Add some fooBar')
4. Push to the branch (git push origin feature/fooBar)
5. Create a new Pull Request

Maintainers

• Sunny G - @sunny-g

License

MIT