Ditto Rust SDK
Overview
Describes the different layers of the Ditto Rust SDK.
Ditto's core codebase is written in Rust.
Currently, however, Rust does not have a stable ABI suitable for directly linking.
Therefore the Rust SDK, like the other Ditto SDKs, is exposed through an external interface that uses the C ABI calling conventions.
This core library is compiled as a both a static and dynamic library for a variety of architectures.
The dittolive-ditto-sys
crate contains Rust bindings to this C ABI library.
The build.rs
script will also attempt to identify the proper library binary for the host environment, download it from Ditto, and link to it.
The Rust SDK includes this -sys
crate as a dependency and then exposes an idiomatic Rust interface on top of this library, along with documentation and example apps.
Building an App with the Rust SDK
The following outlines the general process of getting started with a new App based on the Ditto Rust SDK. Rust currently does not support a stable ABI. To work around this, the Ditto Rust SDK is distributed in two parts: an ergonomic, open-source crate and a closed-source pre-compiled library. It is essential to have the correct library for both your development system and ultimate production target.
There are also some key terms you will need to know, especially for cross-compiling the RustSDK.
- HOST - The host system doing the compiling. This could be a developers MacBook Pro (
x86_64-apple-darwin
) for example. - DITTO_TARGET - The system where the final app is going to run. This will be a Raspberry Pi ZeroW with a CPU that supports ARMv6 32-bit instructions and hard floats:
arm-unknown-linux-gnueabihf
. - DITTO_ROOT - The absolute path for the root directory of the Ditto source tree (where available).
- APP_ROOT - The absolute path for the root directory of your app.
- LIBDITTO - The binary component of the Ditto SDK which exposes a C FFI wrapper.
- RustSDK - The Rust library component of the Ditto SDK which links, via a "-sys crate" to LIBDITTO.
Using a pre-built binary for your RustSDK-based App
The easiest way to get started with the Ditto Rust SDK is to use a pre-built Ditto library for your target architecture, where available.
- Obtain a Ditto License.
- Install the
nightly
tool chain for your current development machine using [rustup]https://rustup.rs - Create a new Rust repository using github or
cargo new
- In the root of the newly created project directory, edit the
Cargo.toml
file. Add adittolive-ditto
as a dependency. This is the crate for the Ditto Rust SDK. - Test building the automatically generated Hello World app by running
cargo build
. This will trigger linking against the Ditto SDK for the current host system. If Ditto is not present for the current host system, this step will report an error when run with--verbose
logging. Ifcurl
is present on the host system and the binary component of the Ditto SDK is absent, thebuild.rs
script will attempt to download the appropriate library for the compilation target automatically. This step will also create aTARGET_DIR
target directory (ie.target/debug
) in the project root. This is a good location for putting the binary component of the Ditto SDK, especially if the development and production hosts are different. - For cross-compilation (ie. development on MacOS for a Linux target), the Rust build script and machinery such as
pkg_config
may not automatically find the correct library for your target platform. In this event the Ditto library should be downloaded manually. Manually prefetching this library may also be desirable for offline CI pipelines and other situations where network access is not desired. - The URL for each target's Ditto SDK is
https://software.ditto.live/rust/Ditto/<version>/<target-tripple>/<profile>/<library-name>
For example, on an x86_64 MacOS developer machine, one would usehttps://software.ditto.live/rust/Ditto/1.0.3-alpha1/x86_64-apple-darwin/release/libdittoffi.dylib
. - The environment variable
DITTOFFI_SEARCH_PATH
may be used to manually set a directory to be used to find the binary component of the Ditto SDK. This is especially helpful when cross-compiling for different target architectures. - With the binary component now in place, verify that
cargo run
executes successfully and doesn't throw a linker error. - On some platforms, similar operations are required for other libraries such as various MacOS Frameworks (ie. CoreFoundation, Security). The
cargo
documentation provides guidance of various ways to provide these libraries to each platforms linker.
Compiling from libditto from source
If you have access to a copy of the full Ditto source code, you can compile the binary component of the Rust SDK, libdittoffi
yourself. The compilation host machine will need 4GB of RAM (or swap file) for linking purposes. If your target machine doesn't have this, you'll need to cross-compile (see section below).
- First, clone the Ditto source tree. We will refer to the absolute path to the root of the Ditto source tree as DITTO_ROOT going forward.
- Select a target system. This can be the same as your host system, or another target triple (cross-compilation). The full list of supported target systems is available here. Note that you will need a valid C toolchain (eg.
CC
,ld
) for both the host and target system. If you are compiling on Mac OS for Linux, this will require more set up (described in a following section).
- Build
libdittoffi
for your target platform. This can take a while.
( && )
- Confirm your build is successful. You should see
libdittoffi.a
andlibdittoffi.so
in$DITTO_ROOT/target/$DITTO_TARGET/release/
. This path is known as yourTARGET_DIR
and is configured by a file.cargo/config.toml
in DITTO_ROOT. Each target and profile (release, debug) will get its own sub-directory. Note that on Windows and Mac OS, the library file extensions will be different (.dll
and.dylib
, respectively). - Optionally, you can now install
libdittoffi.so
into a cannonical search location for the target's linker to find. When compiling on the execution device, we recommend a symlink to/usr/local/lib
or/opt/ditto/lib
. This will make it easy for any apps outside of DITTO_ROOT to findlibdittoffi.so
.
- Alternatively, you can specify your TARGET_DIR as the DITTOFFI_SEARCH_PATH that your app will search for this library.
- Create your new Ditto-powered App. We'll refer to the root of this app as APP_ROOT going forward.
- Add the ditto Rust SDK as a dependency by editing your app's Cargo.toml file. This should be the Path to the Rust SDK not libdittoffi.so.
[]
= "$DITTO_ROOT/rust"
- Test building your app from within the your apps root directory. Your app should build
dittolive-ditto
(the Rust SDK) which in turn will builddittolive-ditto-sys
which links tolibdittoffi.so
orlibdittoffi.a
.
- If you want to force static linking of your app to
libdittoffi.a
you can set theLIBDITTO_STATIC=1
env var. However, this may result in missing symbol errors on some platforms where shared system libraries are otherwise linked in by default. For example, on Mac OS Darwin you may see missing system framework symbols, because these symbols are only available as shared libraries. Your app will need to be configured to tell the linker how to source these symbols.
LIBDITTO_STATIC=1
- Finally, to build a release of your app
- Your final executable can be run as follows. This may be copied to another system, but be sure to also provide copies of
libdittoffi.so
and any other dynamically linked libraries.
How is the Ditto Library located
The build.rs
script for dittolive-ditto-sys makes a best effort attempt to locate, and if absent download, libdittoffi
in spite of each target OS having distinct linkers and conventions for managing shared libraries.
The absolute location of these files depends on whether the SDK is built from source or downloaded from crates.io as a dependency.
The search order is as follows:
- If
DITTOFFI_SEARCH_PATH
env var is set to a valid directory, use this. This takes priority over all other methods. - Look in the CARGO_BUILD_TARGET_DIR (ie.
target/$DITTO_TARGET/{debug,release}
). This is automatic for Cargo. - Look in the
deps
folder for the CARGO_BUILD_TARGET (ie.target/$DITTO_TARGET/{debug,release}/deps
). This is automatic for Cargo. - Look in the CARGO_MANIFEST_DIR.
- The current working directory of
cargo
. - Common POSIX paths defined in the Filesystem Hierarchy Standard:
/usr
,/lib
,/usr/local/lib
,/opt
. - If on a linux system, try using
pkg_config
. Note that Raspberry Pi OS does not ship withpkg_config
. - If on a windows system, try using
vcpkg
. - Download pre-built version from Ditto's S3 bucket
The end result of this search is to ensure that two key arguments are passed to the target linker when building your app:
-ldittoffi
- Your app should be linked against the ditto library-L DITTOFFI_SEARCH_PATH
- The directory where the ditto library for the target system can be found You can see this process by building withcargo build -vv
.
If desired, you can explicitly provide these two arguments using cargo rustc
to build your app, or otherwise configuring your apps build system to provide these values when linking.
We recommend explicitly specifying the search path for the ditto library, especially when cross-compiling, to ensure the correct DITTO_TARGET architecture is used.
Cross-compiling for IOT devices
Many IOT devices make poor compilation hosts for Rust, and thus it is better to cross-compile on a developer machine with more resources and then execute on the target IOT device.
Method 1 - Use a rustembedded/cross Docker and VS Studio Code "devContainer" extension to build from source
- Install docker on your HOST machine
- Install Visual Studio Code on your Host Machine
- Install the VS Code "Remote - Containers" extension.
- Configure your DITTO_TARGET so that you can ssh onto it and copy files using a utility such as
scp
,sftp
, orrsync
. - Check out the Ditto source tree in VS Code, and then in the root of the project edit
.devcontainer/devontainer.json
'sbuild.Dockerfile
key to point to the docker image for your DITTO_TARGET. In this caseDockerfile.armv-unknown-linux-gnueabihf
. - Use the "Remote - Containers" plug in to "Reload in Container" (the green >< icon in the lower left corner). This will download the image, build it, mount the Ditto repo into
/workspaces/ditto
, which will be our effective DITTO_ROOT going forward. - Open a terminal in the container and build libdittoffi. The machining target architecture will be automatically selected.
( && )
- You may now copy the built artifact to your target device (on your host)
- SSH onto your Raspberry Pi and Symlink libdittoffi.so into a linker search directory
- Repeat steps 1 through 8 for your app. If your app lives outside of the DITTO_ROOT, you will need to set up a distinct VS Code project for it. Alternatively, you can compile your app in place on the Raspberry Pi device.
- You can verify your app on the target device with the following
This should verify the target devices linker can resolve all the shared libraries (including libdittoffi.so) on the target device
Configuring the RUST SDK
The following env vars are commonly used to configure the Ditto SDK.
RUST_SDK_LOG_LEVEL
- Sets the log level of the Ditto libraries internal logger. Values are error, warn, info, debug, and verbose. Default is "info".DITTOFFI_SEARCH_PATH
- Explicitly define where to look forlibdittoffi
for your platform.DITTO_DB_PATH
- The absolute path where Ditto should store local copies of documents and attachments. This can also be provided programatically.
Common config patterns
The following are common patterns used to configure test and example apps
DITTO_LICENSE
- EnvVar containing a valid Ditto license token.DITTO_APP_NAME
- The full name of your app. Typically defined in reverse DNS format (ie. "live.ditto.carsapp").DITTO_SITE_ID
- A 64 bit unique identifier of this specific instance of your app. Often associated with a Security subject or Identity.DITTO_BINDIP
- The IP and/or Port (ie. 0.0.0.0:8080) where Ditto should listen for TCP or WebSocket Transport traffic.