jlrs
jlrs is a crate that provides access to most of the Julia C API, it can be used to embed Julia
in Rust applications and to use functionality from the Julia C API when writing ccall
able
functions in Rust. Currently this crate is only tested on Linux and Windows in combination
with Julia 1.6 and 1.8 and is not compatible with other versions of Julia. Using Julia 1.8 is
highly recommended. The minimum supported Rust version is currently 1.65.
The documentation assumes you're already familiar with the Julia programming language.
An incomplete list of features that are currently supported by jlrs:
- Access arbitrary Julia modules and their contents.
- Call Julia functions, including functions that take keyword arguments.
- Handle exceptions or convert them to an error message, optionally with color.
- Include and call your own Julia code.
- Use a custom system image.
- Create values that Julia can use, and convert them back to Rust, from Rust.
- Access the type information and fields of values. The contents of inline and bits-union fields can be accessed directly.
- Create and use n-dimensional arrays. The
jlrs-ndarray
feature can be enabled for integration with ndarray. - Support for mapping Julia structs to Rust structs that can be generated by JlrsReflect.jl.
- Structs that can be mapped to Rust include those with type parameters and bits unions.
- An async runtime is available which can be used from multiple threads and supports
scheduling Julia
Task
s andawait
ing the result without blocking the runtime thread.
NB: Active development happens on the dev
branch, the master
branch points to the most
recently released version.
Prerequisites
Julia must be installed before jlrs can be used. Only version 1.6 and 1.8 are
supported. Using version 1.6 requires enabling the lts
feature.
Linux
The recommended way to install Julia is to download the binaries from the official website,
which is distributed in an archive containing a directory called julia-x.y.z
. This directory
contains several other directories, including a bin
directory containing the julia
executable.
During compilation, the paths to the header and library are normally detected automatically by
executing the command which julia
. The path to julia.h
must be
$(which julia)/../include/julia/julia.h
and the path to the library
$(which julia)/../lib/libjulia.so
. If you want to override this default behaviour the
JULIA_DIR
environment variable must be set to the path to the appropriate julia.x-y-z
directory; in this case $JULIA_DIR/include/julia/julia.h
and
$JULIA_DIR/lib/libjulia.so
are used instead.
In order to be able to load libjulia.so
this file must be on the library search path. If
this is not the case you must add /path/to/julia-x.y.z/lib
to the LD_LIBRARY_PATH
environment variable. When the uv
feature is enabled, /path/to/julia-x.y.z/lib/julia
must
also be added to LD_LIBRARY_PATH
. The latter path should not be added to the default path
because this can break tools currently installed on your system.
Windows
Julia can be installed using juliaup, or with the installer or portable installation
downloaded from the official website. In the first case, Julia has been likely installed in
%USERPROFILE%\.julia\juliaup\julia-x.y.z+0~x64
, while using the installer or extracting
allows you to pick the destination. After installation or extraction a folder called
Julia-x.y.z
exists, which contains several folders including a bin
folder containing
julia.exe
. The path to the bin
folder must be added to the Path
environment variable.
Julia is automatically detected by executing the command where julia
. If this returns
multiple locations the first one is used. The default can be overridden by setting the
JULIA_DIR
environment variable. This doesn't work correctly with juliaup, in this case
the environment variable must be set.
Note that while Julia 1.6 is supported on Windows, several methods are currently unavailable when this version is used.
Features
Most functionality of jlrs is only available if the proper features are enabled. These features generally belong to one of two categories: runtimes and utilities.
A runtime lets you call Julia from Rust, you must enable one of them if you want to embed Julia in a Rust application. The following features enable a runtime:
-
sync-rt
Enables the sync runtime,
Julia
. The sync runtime provides single-threaded, blocking access to the Julia C API. -
async-rt
Enables the async runtime,
AsyncJulia
. The async runtime runs on a separate thread and can be used from multiple threads. While access to the C API is single-threaded, the async runtime can run multiple tasks in parallel by making use of Julia's task system and Rust's async/await syntax. To use this feature you must provide a backing runtime. -
tokio-rt
andasync-std-rt
These features provide a backing runtime for the async runtime. The first uses tokio, the second async-std. The
async-rt
feature is automatically enabled when one of these features is enabled.
If you're writing a library, either one that will be called from Julia or one that will be used by a Rust application that embeds Julia, no runtime is required.
In addition to these runtimes, the following utility features are available:
-
prelude
Provides a prelude module,
jlrs::prelude
. This feature is enabled by default. -
lts
Use the current LTS version of Julia (1.6) instead of the current stable version (1.8).
-
beta
Use the current beta version of Julia (1.9.0-alpha1) instead of the current stable version (1.8).
-
async
Enable the features of the async runtime which don't depend on the backing runtime. This can be used in libraries which provide implementations of tasks that the async runtime can handle.
-
jlrs-derive
This feature should be used in combination with the JlrsReflect.jl package. This package generates Rust bindings for Julia structs, these bindings use the custom derive macros to enable the safe conversion of data from Julia to Rust, and from Rust to Julia in some cases.
-
jlrs-ndarray
Access the contents of a Julia array as an
ArrayView
orArrayViewMut
from ndarray. -
f16
Adds support for working with Julia's
Float16
type from Rust using half'sf16
type. -
ccall
Julia's
ccall
interface can be used to call functions written in Rust from Julia. No runtime can be used in this case because Julia has already been initialized, when this feature is enabled theCCall
struct is available which offers the same functionality as the sync runtime without initializing Julia. -
uv
This feature enables the method
CCall::uv_async_send
, which can be used to wake a JuliaAsyncCondition
from Rust. Theccall
feature is automically enabled when this feature is used. -
pyplot
This feature lets you plot data using the Pyplot package and Gtk 3 from Rust.
-
i686
Link with a 32-bit build of Julia on Linux.
-
debug
Link with a debug build of Julia on Linux.
You can enable all features except lts
and debug
by enabling the full
feature.
Using this crate
If you want to embed Julia in a Rust application, you must enable a runtime feature:
jlrs = {version = "0.17", features = ["sync-rt"]}
jlrs = {version = "0.17", features = ["tokio-rt"]}
jlrs = {version = "0.17", features = ["async-std-rt"]}
When Julia is embedded in an application, it must be initialized before it can be used. The following snippet initializes the sync runtime:
use *;
// Initializing Julia is unsafe because this can load arbitrary
// Julia code, and because it can race with other crates unrelated
// to jlrs. It returns an error if Julia has already been
// initialized.
let mut julia = unsafe ;
// A StackFrame must be provided to ensure Julia's GC can be made aware
// of references to Julia data that exist in Rust code.
let mut frame = new;
let _instance = julia.instance;
To use the async runtime you must upgrade the RuntimeBuilder
to an
AsyncRuntimeBuilder
by providing a backing runtime. Implementations for tokio
and async-std are available if these features have been enabled. When starting the async
runtime, you must declare the maximum number of concurrent tasks as a const generic.
For example, an async runtime backed by tokio and an unbounded channel, that supports 3
concurrent task can be initialized as follows if the tokio-rt
feature is enabled:
// Initializing Julia is unsafe for the same reasons as the sync runtime.
let = unsafe ;
The async runtime can also be spawned as a blocking task on an existing executor:
use *;
async
If you're calling Rust from Julia everything has already been initialized. If the ccall
feature is enabled CCall
is available which provides the same functionality as the sync
runtime.
Calling Julia from Rust
This section will focus on some topics that are common between the sync and async runtimes.
After initialization you have an instance of Julia
or AsyncJulia
, both provide a
method called include
that lets you include files with custom Julia code. In order to
create Julia data and call Julia functions, a scope must be created first.
When the sync runtime is used this can be done by calling the method Julia::scope
. This
method takes a closure with a single argument, a GcFrame
(frame). This frame can be used
to access Julia data, and ensure it's not freed by the GC while it's accessible from Rust.
The async runtime can't create a new scope directly, AsyncJulia
is a handle to the async
runtime which runs on another thread. Instead, the async runtime deals with tasks, each task
runs in its own scope. The simplest kind of task is a blocking task, which can be executed by
calling AsyncJulia::(try_)blocking_task
. These methods accept any closure Julia::scope
can
handle with the additional requirement that it must be Send
and Sync
. It's called a
blocking task because the runtime is blocked while executing this task. The other kinds of
tasks that the async runtime can handle will be introduced later.
Inside the closure provided to Julia::scope
or AsyncJulia::blocking_task
it's possible to
interact with Julia. Unrooted Julia data can be accessed through its module system, the methods
Module::main
, Module::base
, and Module::core
can be used to access the Main
,
Base
, and Core
modules respectively. The contents of these modules can then be accessed by
calling Module::function
which returns a Function
, Module::global
which returns a
Value
, and Module::submodule
which returns another Module
.
Value
provides several methods to allocate new Julia data. The simplest one is
Value::eval_string
, which evaluates the contents of the string passed to it and returns
the result as a Value
. For example, you can evaluate 2
to convert it to Value
. In
practice, this method should rarely be used. It can be used to evaluate simple function calls
like sqrt(2)
, but it must be parsed, compiled, and can't take any non-literal arguments. Its most
important use-case is importing installed and standard library packages by evaluating an
import
or using
statement.
A more interesting method, Value::new
, can be used with data of any type that implements
IntoJulia
. This trait is implemented by primitive types like i8
and char
. Any type
that implements IntoJulia
also implements Unbox
which is used to extract the contents
of a Value
. Pointer wrapper types like Array
don't implement IntoJulia
or Unbox
,
if they can be created from Rust they provide methods to do so.
As a simple example, let's convert two numbers to Julia values and add them:
use *;
// Initializing Julia is unsafe because it can race with another crate that does
// the same.
let mut julia = unsafe ;
let mut frame = new;
let mut julia = julia.instance;
let res = julia.scope.unwrap;
assert_eq!;
Evaluating raw code and calling Julia functions is always unsafe. Nothing prevents you from
calling a function like nasaldemons() = unsafe_load(Ptr{Float64}(0x05391A445))
. Similarly,
mutating Julia data is unsafe because nothing prevents you from mutating data that shouldn't
be mutated, e.g. the contents of the Core
module. A full overview of the rules that you
should keep in mind can be found in the safety
module.
Async and persistent tasks
In addition to blocking tasks, the async runtime lets you execute async tasks which implement
the AsyncTask
trait, and persistent tasks which implement PersistentTask
. Both of
these traits are async traits.
An async task is similar to a blocking task, except that you must implement the async run
method instead of providing a closure. This method takes an AsyncGcFrame
. This new frame
type not only provides access to the same features as GcFrame
, it can also be used to call
async methods provided by the CallAsync
trait. These methods schedule a function call as a
new Julia Task
and can be await
ed until this task has completed. The async runtime can
switch to another task while the result is pending, allowing multiple tasks to run
concurrently.
The previous example can be rewritten as an async task:
use *;
// Only the runtime thread can call the Julia C API, so the async
// trait methods of `AsyncTask` must not return a future that
// implements `Send` or `Sync`.
While blocking and async tasks run once and return their result, a persistent task returns a
handle. This handle can be shared across threads and used to call its run
method. In
addition to a global and an async frame, this method can use the state and input data provided
by the caller.
As an example, let's accumulate some number of values in a Julia array and return the sum of its contents:
use *;
// Only the runtime thread can call the Julia C API, so the async trait
// methods of `PersistentTask` must not return a future that implements
// `Send` or `Sync`.
Calling Rust from Julia
Julia's ccall
interface can be used to call extern "C"
functions defined in Rust, for most
use-cases you shouldn't need jlrs. There are two major ways to use ccall
, with a pointer to
the function or a (:function, "library")
pair.
A function can be cast to a void pointer and converted to a Value
:
use *;
// This function will be provided to Julia as a pointer, so its name can be mangled.
unsafe extern "C"
let mut frame = new;
let mut julia = unsafe ;
let mut julia = julia.instance;
julia
.scope
.unwrap;
You can also use functions defined in dylib
and cdylib
libraries. In order to create such
a library you need to add
[]
= ["dylib"]
or
[]
= ["cdylib"]
respectively to your crate's Cargo.toml
. Use a dylib
if you want to use the crate in other
Rust crates, but if it's only intended to be called through ccall
a cdylib
is the better
choice. On Linux, such a crate will be compiled to lib<crate_name>.so
.
The functions you want to use with ccall
must be both extern "C"
functions to ensure the C
ABI is used, and annotated with #[no_mangle]
to prevent name mangling. Julia can find
libraries in directories that are either on the default library search path or included by
setting the LD_LIBRARY_PATH
environment variable on Linux. If the compiled library is not
directly visible to Julia, you can open it with Libdl.dlopen
and acquire function pointers
with Libdl.dlsym
. These pointers can be called the same way as the pointer in the previous
example.
If the library is visible to Julia you can access it using the library name. If call_me
is
defined in a crate called foo
, the following should work:
ccall((:call_me, "libfoo"), Int, (Bool,), false)
One important aspect of calling Rust from other languages in general is that panicking across
an FFI boundary is undefined behaviour. If you're not sure your code will never panic, wrap it
with std::panic::catch_unwind
.
Many features provided by jlrs require a Target
. This requires creating an instance of
CCall
first. Another method provided by CCall
is CCall::uv_async_send
, this method
can be used to wake an Base.AsyncCondition
. In particular, it can be used to write a
ccall
able function that does its actual work on another thread, returns early and then
wait
ing on the async condition from Julia. The advantage of this is that the long-running
function won't block Julia. In this case you will need to use GC.@preserve
to ensure Julia
is aware that the use of this data is still in use after the ccall
has returned.
Testing
The restriction that Julia can be initialized once must be taken into account when running
tests that use jlrs
. Because tests defined in a single crate are not guaranteed to be run
from the same thread you must guarantee that each crate has only one test that initializes
Julia. It's recommended you only use jlrs in integration tests because each top-level
integration test file is treated as a separate crate.
use *;
Because AsyncJulia
is thread-safe, it is possible to have multiple tests in a single crate
when the async runtime is used:
use ;
use *;
use OnceCell;
pub static JULIA: = new;
Custom types
In order to map a struct in Rust to one in Julia you can derive Unbox
, Typecheck
and
ValidLayout
. If the struct in Julia is immutable ValidField
can also be derived,
if it has no type parameters IntoJulia
.
You normally shouldn't need to implement these structs or traits manually. The JlrsReflect package can generate correct Rust struct and automatically derive the supported traits for types that have no atomic fields, nor any tuple or union fields with type parameters. The reason for this restriction is that the layout of such fields can be very different in a way that can't be easily represented.
These custom types can also be used when you call Rust from Julia with ccall
.