Expand description
The DRE plugin protocol, version 0.
Plugins ship in packages: one executable, dre-plugin-<package>, serving every plugin the
package provides (a single plugin may also be named dre-<kind>-<name>). It talks to DRE core
over stdin/stdout in length-prefixed frames. Each frame is a JSON control message or an Arrow IPC stream; stderr
is the plugin’s log channel. See docs/protocol.md in the repository for the public contract.
frame: the wire format.msg: every control message.host: the core side: spawning a plugin, the handshake, requests.plugin: the SDK plugin authors use to serve requests.options: the options a plugin declares, and how they’re checked.conformance: checks any plugin binary against the protocol.sessions: OAuth sessions plugins keep in~/.dre/oauth_sessions.json.
Modules§
- conformance
- The protocol conformance suite. Every first-party plugin runs it in its tests; third-party
plugin authors can run it too. It only checks protocol behaviour common to every plugin kind;
each plugin’s own tests cover what it does with real data. A package executable
(
dre-plugin-<package>) is checked once per plugin it provides. - frame
- Frames: a 4-byte big-endian length, then that many bytes. The first body byte is the frame
type:
J(0x4A) for a UTF-8 JSON control message,A(0x41) for an Arrow IPC stream holding a schema and zero or more record batches. - host
- Core’s side of the protocol: spawn a plugin, negotiate a version, send requests.
- msg
- Control messages. Every JSON frame is an object with a
typefield. - options
- Plugin options: the keys a report’s config block for a plugin may hold (a format’s
output:keys, a destination entry’s keys), declared once by the plugin and checked by the SDK. - plugin
- The plugin SDK: implement one of
Source,FormatorDestinationand call the matchingserve_*function frommain. The SDK owns stdin/stdout, the handshake, framing, error replies and panics; plugins log witheprintln!. - sessions
~/.dre/oauth_sessions.json: OAuth sessions plugins keep between runs.- util
- Small helpers shared by core and first-party plugins, so each rule has one implementation.
Structs§
- Plugin
Id - One plugin: its kind and name. Written
<kind>/<name>(destination/s3), in messages as well as indre.lock.
Enums§
Constants§
- CAP_
CHECK - Capability: can verify a statement without executing it.
- CAP_
LOAD - Source: loads rows into a temporary table on the session (
load), for large lookups. - CAP_
MULTI_ FILE - Capability (destinations): takes every file of one output in a single
deliver. - CAP_
READ_ ONLY - Capability: can open a connection read-only.
- CAP_
SESSIONS - Capability: can hold one session (and its temp objects) across requests.
- CAP_
VALIDATE - Answers
validate(checks a config block of options). The Rust SDK always advertises it. - CRATE_
VERSION - This crate’s version (its own, not DRE’s); the fixture plugins report it.
- MAX_
VERSION - MIN_
VERSION - Protocol versions this build speaks.
Functions§
- executable_
name - The executable file name for a single plugin on this platform.
- package_
executable_ name - The executable file name for a package on this platform:
dre-plugin-<package>. - parse_
executable_ name - Parse
dre-<kind>-<name>[.exe]into its kind and name. Names are[a-z0-9_]+. - parse_
package_ executable_ name - Parse
dre-plugin-<package>[.exe]into the package name. - valid_
name - Whether
nameis a valid plugin or package name:[a-z0-9_]+.