Crate dre_protocol

Expand description

The DRE plugin protocol, version 1.

Plugins ship in packages: one executable, dre-plugin-<package>, serving every plugin the package provides (a single plugin may also be named dre-<kind>-<name>). It talks to DRE core over stdin/stdout in length-prefixed frames. Each frame is a JSON control message or an Arrow IPC stream; plugins log through log messages, and stderr carries what can’t (panics, native drivers). See docs/protocol.md in the repository for the public contract.

frame: the wire format.
msg: every control message.
host: the core side: spawning a plugin, the handshake, requests.
plugin: the SDK plugin authors use to serve requests.
options: the options a plugin declares, and how they’re checked.
conformance: checks any plugin binary against the protocol.
sessions: OAuth sessions plugins keep in ~/.dre/oauth_sessions.json.
markdown: the portable Markdown subset of messages, and its translations.
delivery: the delivery and network rules every file destination shares.

Re-exports§

pub use log;

Modules§

conformance: The protocol conformance suite. Every first-party plugin runs it in its tests; third-party plugin authors can run it too. It only checks protocol behaviour common to every plugin kind; each plugin’s own tests cover what it does with real data. A package executable (dre-plugin-<package>) is checked once per plugin it provides.
connection: Static checks of a profile entry’s connection settings, from the fields a plugin declares: a required field missing, an unknown key, a value of the wrong FieldKind or not one of its choices. The SDK runs them for validate_connection and before open and deliver, then the plugin’s own rules, so dre validate and dre run say the same thing. Nothing here touches the network.
delivery: The delivery and network rules every file destination shares, written once.
frame: Frames: a 4-byte big-endian length, then that many bytes. The first body byte is the frame type: J (0x4A) for a UTF-8 JSON control message, A (0x41) for an Arrow IPC stream holding a schema and zero or more record batches.
host: Core’s side of the protocol: spawn a plugin, negotiate a version, send requests.
markdown: The portable Markdown subset a message’s text is written in, and its translations.
msg: Control messages. Every JSON frame is an object with a type field. From protocol 1, every message after the handshake also carries the id of the request it belongs to (Envelope).
options: Plugin options: the keys a report’s config block for a plugin may hold (a format’s output: keys, a destination entry’s keys), declared once by the plugin and checked by the SDK.
plugin: The plugin SDK: implement one of Source, Format or Destination and call the matching serve_* function from main. The SDK owns stdin/stdout, the handshake, framing, request ids, error replies and panics.
sessions: ~/.dre/oauth_sessions.json: OAuth sessions plugins keep between runs.
style: xlsx styling: the style: maps on an output, a query entry (tab) and a column. Shared by core (which checks a query entry’s style and passes it on) and the xlsx plugin (which checks the output’s and applies them all).
util: Small helpers shared by core and first-party plugins, so each rule has one implementation.

Structs§

PluginId: One plugin: its kind and name. Written <kind>/<name> (destination/s3), in messages as well as in dre.lock.

Enums§

Kind

Constants§

CAP_CHECK: Capability: can verify a statement without executing it.
CAP_LOAD: Source: loads rows into a temporary table on the session (load), for large lookups.
CAP_MESSAGE: Capability (destinations): takes a message (deliver with message), not only files.
CAP_MESSAGE_ONLY: Capability (destinations): takes only messages; core never sends it a file output.
CAP_MULTI_FILE: Capability (destinations): takes every file of one output in a single deliver.
CAP_READ_ONLY: Capability: can open a connection read-only.
CAP_SESSIONS: Capability: can hold one session (and its temp objects) across requests.
CAP_VALIDATE: Answers validate (checks a config block of options). The Rust SDK always advertises it.
CORE_MIN_VERSION: The oldest protocol version core accepts from a plugin.
CRATE_VERSION: This crate’s version (its own, not DRE’s); the fixture plugins report it.
MAX_VERSION
MIN_VERSION: Protocol versions the plugin SDK speaks: a plugin built on it also works with an older core.

Functions§

executable_name: The executable file name for a single plugin on this platform.
package_executable_name: The executable file name for a package on this platform: dre-plugin-<package>.
parse_executable_name: Parse dre-<kind>-<name>[.exe] into its kind and name. Names are [a-z0-9_]+.
parse_package_executable_name: Parse dre-plugin-<package>[.exe] into the package name.
valid_name: Whether name is a valid plugin or package name: [a-z0-9_]+.