About
Northstar is an open source embedded container runtime optimized for speed and resource usage. Northstar combines several standard Linux process isolation and sandboxing features to gain a medium level of isolation between containers/processes. The Northstar runtime consists out of two parts: The container handling and process spawning. To build the most efficient and robust solution, Northstar is completely developed in Rust, a language designed to afford the performance of C/C++ without their footguns.
Containers
Northstar containers are called NPK
. The NPK format is heavily inspired by the
Android APEX technology. A
Northstar container contains:
- Root filesystem in a Squashfs file system image (optionally compressed)
- Northstar manifest with process configuration and container meta information
Northstar containers can be created with the Northstar utility sextant.
Processes
Started Northstar contains are Linux processes. The attributes and environment for a spawned container is described in a container manifest which is included in a NPK. The container manifest allows to configure the following Linux subsystems and features:
- Arguments passed to the containers init
- Environment variables set in the container context
- User and group id
- Mount namespace
- PID namespace
- Cgroups memory (optional)
- CGroups CPU (optional)
- Additional bind mounts (optional)
- Capabilities (optional)
- Stdout/stderr handling (optional)
- Seccomp configuration (optional)
Comparison
- Northstar containers are not portable and are tailored to a known system (uid/gid/mounts...)
- TODO
Quickstart
Northstar is written in Rust. The minimum supported Rust version (MSRV) is 1.59. Rust is best installed and managed by the rustup tool. Rust has a 6-week rapid release process and supports a great number of platforms, so there are many builds of Rust available at any time. rustup manages these builds in a consistent way on every platform that Rust supports, enabling installation of Rust from the beta and nightly release channels as well as support for additional cross-compilation targets.
Building Northstar is limited to Linux systems and runs on Linux systems only!
The Northstar build generates bindings for various system libraries and uses the
mksquashfs
command line tool for NPK creation.
Install build dependencies on Debian based distributions by running
The squashfs-tools
are required in version 4.5 or higher.
Northstar comes with a set of examples that demonstrate most of the Northstar features. Building the example binaries and packing its corresponding NPKs is done via:
Building and starting the example runtime main is triggered by a
The Northstar workspace configuration configures a cargo runner that invokes the runtimes example main binary with super user rights.
Use the nstar utility to inspect and modify the runtimes state e.g.
> ./target/release/northstar-nstar
}}}}
> ./target/release/northstar-nstar
}}
Configuration
The example executable northstar
reads a configuration file that represents
northstar_runtime::runtime::config::Config
.
# Directory where containers are mounted
= "target/northstar/run"
# Directory for `persist` mounts of containers
= "target/northstar/data"
# Log directory for debug logs and traces
= "target/northstar/logs"
# Top level cgroup name
= "northstar"
# Event loop buffer size
= 256
# Notification buffer size
= 64
# Device mapper device timeout
= "2s"
# Token validity
= "1m"
# Loop device timeout
= "2s"
# Debug TCP console on localhost with full access
[]
= "full"
# Start a `strace -p PID ...` instance after a container is started.
# The execution of the application is deferred until strace is attached.
[]
# Configure the output of the strace instance attached to a started
# application. "file" for a file named strace-<PID>-name.log or "log"
# to forward the strace output to the runtimes log.
= "log"
# Optional additional flags passed to `strace`
= "-f -s 256"
# Optional path to the strace binary
= /bin/strace
# Include the runtime system calls prior to exeve
= true
# Start a `perf record -p PID -o LOG_DIR/perf-PID-NAME.perf FLAGS` instance
# after a container is started.
[]
# Optional path to the perf binary
= "/bin/perf"
# Optional additional flags passed to `perf`
= ""
# NPK Repository `memory` configuration. This is a not persistent in memory repository
[]
= "examples/northstar.pub"
= "mem"
# NPK Repository `default` in `dir`
[]
# Mount the containers from this repository on runtime start. Default: false
= true
= "examples/northstar.pub"
= { = { = "target/northstar/repository" }}
Repositories
A repository is an entity that is able to store NPK's at runtime. Repositories
are configured at initialization time of the runtime. The repository
configuration cannot be changed at runtime. Each configured repository has a
unique identifier
id.
Currently two types of repositories exists: fs
and mem
.
The fs
type repositories are backed by file system storage. The configured
directory (dir
) is used to store NPK's. If this directory is read only, no
additional install requests can be performed at runtime. If a fs
repository
configuration contains a key
field, the repository is treated as "verified".
The configured key is used to verify the signature of the containers manifest
and it's verity root hash. When the container is mounted, the verity root hash
is used to configure a device mapper verity devices that is mounted instead of
the contained Squashfs image.
Repositories without a key
are treated as trustful sources. No signature
checks are performed. The root filesystems are mounted without verity. A
possibly present verity root hash with in the NPK is ignored. Trusted
repositories are should on verified and read only file systems.
Set the mount_on_start
flag of a fs
repository to true
to make the runtime
mount all containers present at startup. The mount operations are done in
parallel.
The mem
repositories uses
memfd for it's
storage. No data is persistently stored during an installation of a container.
Obviously it's not possible to have NPK's preinstalled in a mem
repository at
runtime startup. The mem
repositories are mainly used for testing.
Console
Northstar uses JSON to encode the messages shared with clients. The messages
are newline delimited. This is a common approach that facilitates clients being
implemented in any programming language. However, Northstar as a library,
provides a convenient
Client
type that can be used for a simpler client implementation using Rust.
Northstar interacts with clients through a TCP
socket bound to each address
configured in runtime::config::Config::console
.
- TODO: Move client guide part to a README-console.md etc
- TODO: Update guide/src/client/connect with the connect sequence.
- TODO: Client API docs
Integration tests
Integration tests start a runtime instance and assert on log output of container of notification sent from the runtime. The testsuite is invoked by the Rust test system:
and are executed by the project CI.
Portability
Northstar makes extensive use of Linux Kernel features and runs on Linux systems only. Northstar is tested on the architectures
aarch64-linux-android
aarch64-unknown-linux-gnu
aarch64-unknown-linux-musl
x86_64-unknown-linux-gnu
akaLinux Desktop
Northstar cannot be run on 32 bit systems! In order to verify that all needed
Kernel features are available, either run the
check_conf script or manually compare the target's
kernel configuration with the CONFIG_
entries in the check_conf.sh
script.
TODO: List required CONFIG_
items here. The check_config script runs on
*Android only
Internals
Container launch sequence
Manifest Format
The manifest format is described here.
Mounts
The options of a mount entry in the manifest are optional. To apply one of the
mount options rw
, noexec
, nosuid
, nodev
or rec
it must be explicitly set.
The default for a bind
and tmpfs
mount is read only. Mounts that are rw
are
usually hard to handle from an integration point of view (SELinux and
permissions). Nevertheless - here's the example how to mount the host systems
/tmp
directory to /tmp
. The bind mount is not remounted ro
. Note that
ro
bind mounts require two mount operations.
/tmp:
type: bind
host: /tmp
options: rw, nosuid, noexec
Resource containers cannot be mounted rw
. The filesystem of resource
containers is squashfs which is not writeable. Resource containers can be
mounted without the noexec
flag in order to provide binaries.
Example resource mount with the executable flag set but nodev
and nosuid
set (optional):
/bin/java:
type: resource
name: java13
version:
dir: /
options: nodev, nosuid
Example tmpfs
mount. tmpfs
mounts are never ro
;-):
/tmpfs:
type: tmpfs
size: 20M
Mounts of type persist
are support from the runtime for containers. The runtime
takes care to mount a read and writeable directory into the containers fs. The
directory is dedicated to this container. The directory is not shared with other
containers.
TODO: When are the directories removed? See #560
/data:
type: persist
To provide a minimal
/dev
file system to the container, add a mount entry of
type dev
.
/dev:
type: dev
to the manifest. The /dev
is populated with only:
full
null
random
tty
urandom
zero
If the container binary needs more devices, bind mount the host systems /dev
.
Roadmap
See the open issues for a list of proposed features and known issues).
Questions and Help
Ask us questions about anything related to Northstar! To add your question, create an issue in this repository.
Just a few guidelines to remember before you ask a question:
- Ensure your question hasn't already been answered. If it has been answered but do es not satisfy you, feel free to comment in the issue and we will re-open it.
- Use a succinct title and description. Add as much information as possible e.g manifests, npks, applications...
- If your question has already been asked and answered adequately, please add a thumbs-up (or the emoji of your choice!) to the issue. This helps us in identifying common problems that people usually face.
- Lastly, be civil, polite and patient. :)
Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
License
Distributed under the Apache 2.0 License. See LICENSE for more information.
Contact
Project Link: https://github.com/esrlabs/northstar
Acknowledgements
- The Rust Community for providing crates that form the foundation of Northstar
- The Android Open Source Project for the APEX inspiration
- youki for solving similar problems
- The manpage project for documenting the weird world of system calls