Write a Collectd Plugin in Rust
Collectd is a ubiquitous system statistics collection daemon.
collectd_plugin
leverages collectd's ability to dynamically load plugins and
creates an ergonomic, yet extremely low cost abstraction API to interface with
collectd.
Features:
- No unnecessary allocations when submitting / receiving values, logging
- Register multiple plugin instances
- Automatic deserialization of plugin configs via Serde (optional) feature
- Deployment: compile against collectd version and scp to server
- Referenced Rust libraries are statically linked
- Help writing thread safe plugins thanks to the Rust compiler
Usage
Add to your Cargo.toml
:
[]
= "0.8.0"
Serde support is enabled by default for configuration parsing.
Then put this in your crate root:
extern crate collectd_plugin;
This repo is tested on the following:
- collectd 5.4 (Ubuntu 14.04)
- collectd 5.5 (Ubuntu 16.04)
- collectd 5.7 (and above) (Ubuntu 18.04)
Quickstart
See what to add to your project's Cargo file
Below is a complete plugin that dummy reports load values to collectd, as it registers a READ
hook. For an implementation that reimplements collectd's own load plugin, see examples/load
extern crate collectd_plugin;
extern crate failure;
use ;
use Error;
;
// A manager decides the name of the family of plugins and also registers one or more plugins based
// on collectd's configuration files
// We pass in our plugin manager type
collectd_plugin!;
Motivation
There are five main ways to extend collectd:
- Write plugin against the C api:
<collectd/core/daemon/plugin.h>
- Write plugin for collectd-python
- Write plugin for collectd-java
- Write a cli for the exec plugin
- Write a service that writes to a unix socket
And my thoughts:
- I'm not confident enough to write C without leaks and there isn't a great package manager for C.
- Python and Java aren't self contained, aren't necessarily deployed on the server, are more heavy weight, and I suspect that maintenance plays second fiddle to the C api.
- The exec plugin is costly as it creates a new process for every collection
- Depending on the circumstances, writing to a unix socket could be good fit, but I enjoy the ease of deployment, and the collectd integration -- there's no need to re-invent logging scheme, configuration, and system init files.
Rust's combination of ecosystem, package manager, C ffi, single file dynamic library, and optimized code made it seem like a natural choice.
To Build
To ensure a successful build, adapt the below to your project's Cargo file.
[]
= ["cdylib"]
= "<your plugin name>"
[]
= ["collectd-plugin/bindgen"]
= []
- A collectd version is required. You can specify environment variable
COLLECTD_VERSION
as5.4
,5.5
, or5.7
, or rely oncollectd-rust-plugin
auto detecting the version by executingcollectd -h
. - The bindgen feature is optional (it will re-compute the Rust bindings from C code, which shouldn't be necessary). Make sure you have an appropriate version of clang installed and
collectd-dev
- collectd expects plugins to not be prefixed with
lib
, socp target/debug/libmyplugin.so /usr/lib/collectd/myplugin.so
- Add
LoadPlugin myplugin
to collectd.conf
Plugin Configuration
The load plugin in examples/load demonstrates how to expose configuration values to collectd.
# In this example configuration we provide short and long term load and leave
# Mid to the default value. Yes, this is very much contrived
ReportRelative true
Benchmarking Overhead
To measure the overhead of adapting collectd's datatypes when writing and reporting values:
If you'd like to use the timings on my machine:
- 100ns to create and submit a
ValueListBuilder
- 150ns to create a
ValueList
for plugins that write values
Unless you are reporting or writing millions of metrics every interval (in which case you'll most likely hit an earlier snap), you'll be fine.