Write a Collectd Plugin in Rust
Collectd is a ubiquitous system statistics collection daemon.
collectd_plugin
leverages collectd's ability to dynamically load plugins and
creates an ergonomic, yet extremely low cost abstraction API to interface with
collectd.
Features:
- No unnecessary allocations when submitting / receiving values, logging
- Register multiple plugin instances
- Automatic deserialization of plugin configs via Serde (can opt-out)
- Deployment: compile against collectd version and scp to server
- Referenced Rust libraries are statically linked
- Help writing thread safe plugins thanks to the Rust compiler
Usage
Add to your Cargo.toml
:
[]
= "0.14.0"
Serde support is enabled by default for configuration parsing.
Works with any collectd version 5.4+, but all users will need to specify the collectd api version they want to target via the COLLECTD_VERSION
environment variable (or rely on $(collectd -h)
or COLLECTD_PATH
variable).
COLLECTED_VERSION |
Compatible Range |
---|---|
5.4 | [5.4, 5.5) |
5.5 | [5.5, 5.7) |
5.7 | [5.7,) |
Quickstart
See what to add to your project's Cargo file
Below is a complete plugin that dummy reports load values to collectd, as it registers a READ
hook. For an implementation that reimplements collectd's own load plugin, see examples/load
use ;
use error;
;
// A manager decides the name of the family of plugins and also registers one or more plugins based
// on collectd's configuration files
// We pass in our plugin manager type
collectd_plugin!;
Motivation
There are five main ways to extend collectd:
- Write plugin against the C api:
<collectd/core/daemon/plugin.h>
- Write plugin for collectd-python
- Write plugin for collectd-java
- Write a cli for the exec plugin
- Write a service that writes to a unix socket
And my thoughts:
- I'm not confident enough to write C without leaks and there isn't a great package manager for C.
- Python and Java aren't self contained, aren't necessarily deployed on the server, are more heavy weight, and I suspect that maintenance plays second fiddle to the C api.
- The exec plugin is costly as it creates a new process for every collection
- Depending on the circumstances, writing to a unix socket could be good fit, but I enjoy the ease of deployment, and the collectd integration -- there's no need to re-invent logging scheme, configuration, and system init files.
Rust's combination of ecosystem, package manager, C ffi, single file dynamic library, and optimized code made it seem like a natural choice.
To Build
To ensure a successful build, adapt the below to your project's Cargo file.
[]
= ["cdylib"]
= "<your plugin name>"
[]
= ["collectd-plugin/bindgen"]
= []
- A collectd version is required to build. There are several ways one can specify it:
- Via environment variable:
COLLECTD_VERSION
=5.4
,5.5
, or5.7
. - Via environment variable:
COLLECTD_PATH
points to the root git directory for collectd. This option makes the most sense when coupled with thebindgen
feature. - Auto detection by executing
collectd -h
.
- Via environment variable:
- The bindgen feature is optional (it will re-compute the Rust bindings from C code, which shouldn't be necessary). Make sure you have an appropriate version of clang installed and
collectd-dev
(if not usingCOLLECTD_PATH
) - collectd expects plugins to not be prefixed with
lib
, socp target/debug/libmyplugin.so /usr/lib/collectd/myplugin.so
- Add
LoadPlugin myplugin
to collectd.conf
Plugin Configuration
The load plugin in examples/load demonstrates how to expose configuration values to collectd.
# In this example configuration we provide short and long term load and leave
# Mid to the default value. Yes, this is very much contrived
ReportRelative true
Benchmarking Overhead
To measure the overhead of adapting collectd's datatypes when writing and reporting values:
If you'd like to use the timings on my machine:
- 60ns to create and submit a
ValueListBuilder
- 130ns to create a
ValueList
for plugins that write values
Unless you are reporting or writing millions of metrics every interval (in which case you'll most likely hit an earlier bottleneck), you'll be fine.
Plugins
Do you use collectd-rust-plugin? Feel free to add your plugin to the list.
- pg-collectd: An alternative and opinionated postgres collectd writer