.. py:currentmodule:: starlark_pyoxidizer
.. _rust_rust_code:
=================================
Controlling Python From Rust Code
=================================
PyOxidizer can be used to embed Python in a Rust application.
This page documents what that looks like from a Rust code perspective.
Interacting with the ``pyembed`` Crate
======================================
When writing Rust code to interact with a Python interpreter, your
primary area of contact will be with the ``pyembed`` crate.
The ``pyembed`` crate is a standalone crate maintained as part of the
PyOxidizer project. This crate provides the core run-time functionality
for PyOxidizer, such as the implementation of
:ref:`PyOxidizer's custom importer <oxidized_importer>`. It also exposes
a high-level API for initializing a Python interpreter and running code
in it.
See :ref:`pyembed` for full documentation on the ``pyembed`` crate.
:ref:`pyembed_controlling_python` in particular describes how to interface
with the embedded Python interpreter.
The following documentation will be unique to PyOxidizer's use of the
``pyembed`` crate.
Using the Default ``OxidizedPythonInterpreterConfig``
=====================================================
When using a PyOxidizer-generated Rust project and that project is configured
to use PyOxidizer to build (the default), that project/crate's build script
will call into PyOxidizer to emit various build artifacts. This will process
the PyOxidizer configuration file and write some files somewhere.
One of the files generated is a Rust source file containing a
``fn default_python_config() -> pyembed::OxidizedPythonInterpreterConfig`` which
emits a ``pyembed::OxidizedPythonInterpreterConfig`` using the configuration
from the PyOxidizer configuration file. This configuration is based off the
:py:class:`PythonInterpreterConfig` defined in the PyOxidizer Starlark
configuration file.
The crate's build script will set the ``DEFAULT_PYTHON_CONFIG_RS``
environment variable to the path to this file, exposing it to Rust code.
This all means that to use the auto-generated
``pyembed::OxidizedPythonInterpreterConfig`` instance with your Rust application,
you simply need to do something like the following:
.. code-block:: rust
include!(env!("DEFAULT_PYTHON_CONFIG_RS"));
fn create_interpreter() -> Result<pyembed::MainPythonInterpreter> {
// Calls function from include!()'d file.
let config: pyembed::OxidizedPythonInterpreterConfig = default_python_config();
pyembed::MainPythonInterpreter::new(config)
}
Using a Custom ``OxidizedPythonInterpreterConfig``
--------------------------------------------------
If you don't want to use the default
``pyembed::OxidizedPythonInterpreterConfig`` instance, that's fine too! However,
this will be slightly more complicated.
First, if you use an explicit ``OxidizedPythonInterpreterConfig``, the
:py:class:`PythonInterpreterConfig` Starlark
type defined in your PyOxidizer configuration file doesn't matter that much.
The primary purpose of this Starlark type is to derive the default
``OxidizedPythonInterpreterConfig`` Rust struct. And if you are using your own
custom ``OxidizedPythonInterpreterConfig`` instance, you can ignore most of the
arguments when creating the ``PythonInterpreterConfig`` instance.
An exception to this is the ``raw_allocator`` argument/field. If you
are using a custom allocator (like jemalloc, mimalloc, or snmalloc), you will need
to enable a Cargo feature when building the ``pyembed`` crate or else you will get
a run-time error that the specified allocator is not available.
``pyembed::OxidizedPythonInterpreterConfig::default()`` can be used to
construct a new instance, pre-populated with default values for each field.
The defaults should match what the :py:class:`PythonInterpreterConfig`
Starlark type would yield.
The main catch to constructing the instance manually is that the custom
*meta path importer* won't be able to service Python ``import`` requests
unless you populate a few fields. In fact, if you just use the defaults,
things will blow up pretty hard at run-time::
$ myapp
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007fa0e2cbe9c0 (most recent call first):
Aborted (core dumped)
What's happening here is that Python interpreter initialization hits a fatal
error because it can't ``import encodings`` (because it can't locate the
Python standard library) and Python's C code is exiting the process. Rust
doesn't even get the chance to handle the error, which is why we're seeing
a segfault.
The reason we can't ``import encodings`` is twofold:
1. The default filesystem importer is disabled by default.
2. No Python resources are being registered with the
``OxidizedPythonInterpreterConfig`` instance.
This error can be addressed by working around either.
To enable the default filesystem importer:
.. code-block:: rust
let mut config = pyembed::OxidizedPythonInterpreterConfig::default();
config.filesystem_importer = true;
config.sys_paths.push("/path/to/python/standard/library");
As long as the default filesystem importer is enabled and ``sys.path``
can find the Python standard library, you should be able to
start a Python interpreter.
.. hint::
The ``sys_paths`` field will expand the special token ``$ORIGIN`` to the
directory of the running executable. So if the Python standard library is
in e.g. the ``lib`` directory next to the executable, you can do something
like ``config.sys_paths.push("$ORIGIN/lib")``.
If you want to use the custom :ref:`PyOxidizer Importer <oxidized_importer>`
to import Python resources, you will need to update a handful of fields:
.. code-block:: rust
let mut config = pyembed::OxidizedPythonInterpreterConfig::default();
config.packed_resources = ...;
config.oxidized_importer = true;
The ``packed_resources`` field defines a reference to *packed resources
data* (a ``PackedResourcesSource`` enum. This is a custom serialization
format for expressing *resources* to make available to a Python interpreter. See
:ref:`python_packed_resources` for more. The easiest way to obtain this
data blob is by using PyOxidizer and consuming the ``packed-resources``
build artifact/file, likely though ``include_bytes!``.
:ref:`oxidized_finder` can also be used to produce these data structures.
Finally, setting ``oxidized_importer = true`` is necessary to enable
:py:class:`oxidized_importer.OxidizedFinder`.