Crate deimos

Expand description

§Deimos

Control program and data integrations for the Deimos data acquisition ecosystem.

See the project readme for contact details as well as commentary about the goals and state of the project.

The control program and the firmware-software shared library share a changelog at the workspace level.

§Install - Rust

cargo add deimos

§Install - Python

pip install deimos-daq

§Features & Roadmap

✅ Implemented | 💡 Planned

Feature Category	Features
Control Loop	✅ Fixed-dt roundtrip control loop ✅ Network scanning for available hardware ✅ Planned loop termination ✅ Global event logging ✅ Reconnection ✅ Low-CPU-usage background operation
Control Calcs	✅ User-defined custom calcs ✅ Explicit (acyclic) calc expression ✅ Low-pass filters ✅ Sequenced state machines ✅ Polynomial calibration curves 💡 Cyclic expressions with explicit time-delay 💡 Prototype calc w/ rhai script-defined inner function
Data Integrations	✅ User-defined custom targets ✅ Manual read/write ✅ CSV ✅ In-memory dataframe ✅ TimescaleDB 💡 InfluxDB 💡 Zarr file/bucket 💡 Generic sqlite, postgres, etc.
Hardware Peripherals	✅ Deimos DAQs ✅ User-defined custom hardware ✅ User-defined hardware drivers ✅ Hardware-out-of-the-loop wrapper
Socket Interfaces (peripheral I/O)	✅ User-defined custom interfaces ✅ UDP/IPV4 ✅ Unix socket ✅ Thread channel sideloading 💡 TCP 💡 UDP/IPV6

§Concept of Operation

The control program follows the hardware peripheral state machine, which is linear except that an error in any peripheral state results in returning to Connecting.

Peripheral states:

Connecting (no communication with the control machine)
Binding (waiting to associate with a control machine)
Configuring (waiting for operation-specific configuration from control machine)
Operating (roundtrip control)

The controller initialization schedule is:

                  --------------------binding timeout windows
                 /                          /
                /    Controller init       /           
             |====|=====================|====|         timeout to operating
     scan for|                          |==============|
  peripherals|                          |      \
   (broadcast|              sent binding|       \
     binding)|                          |    configuring window
                             peripherals|
                              transition|
                          to configuring|

In words,

Scan network for peripherals available to bind
- Broadcast binding request, but do not send configuration; allow Configuring to time out back to Connecting
Initialize controller (set up data integrations, initialize internal state, etc)
Send binding request
Wait for binding responses
Send configuration
Wait for start of Operating

Peripherals acknowledge binding and transition to configuring on their next internal cycle (usually 1ms) after receiving a request to bind. They will then wait for configuration input until the timeout to operating, and either proceed to Operating if configuration was successful, or return to Connecting (and likely return immediately to Binding) if configuration was not successful.

The control loop then follows a fixed schedule on each cycle:

Send control input to peripherals
Wait for outputs from peripherals
- Target synchronization to middle of cycle
Update time-sync control
Run calculations on peripheral outputs

Control loop timing uses the control machine’s best available monotonic clock. Both system time in UTC with nanoseconds and monotonic clock time are stored in order to support post-processing adjustments to account for the slow drift of the monotonic clock relative to system time.

§Example: 200Hz Control Program w/ 2 DAQs

use std::time::Duration;

use deimos::calc::{Constant, Sin};
use deimos::peripheral::{PluginMap, analog_i_rev_3::AnalogIRev3};
use controller::context::ControllerCtx;
use deimos::*;

// The name of the operation will be used as the table name for databases,
// or as the file name for local storage.
let op_name = "test_op".to_owned();

// An optional dictionary mapping user-defined custom hardware
// peripheral model numbers to initializer functions.
let peripheral_plugins: Option<PluginMap> = None;

// Configure the controller
//    Sample interval is taken as integer nanoseconds
//    so that any rounding and loss of precision is visible to the user
let rate_hz = 200.0;
let dt_ns = (1e9_f64 / rate_hz).ceil() as u32;  // Control cycle period

// Define idle controller
let mut ctx = ControllerCtx::default();
ctx.op_name = op_name;
ctx.dt_ns = dt_ns;
let mut controller = Controller::new(ctx);

// Set up any number of data integrations,
// all of which will receive the same data at each cycle of the control loop
//    TSDB-flavored postgres database
let buffer_window = Duration::from_nanos(1); // Non-buffering mode
let retention_time_hours = 1;
let timescale_dispatcher: Box<dyn Dispatcher> = TimescaleDbDispatcher::new(
    "<database name>",  // Database name
    "<database address>", // URL or unix socket interface
    "<username>",  // Login name; for unix socket, must match OS username
    "<token env var>",  // Environment variable containing password or token
    buffer_window,
    retention_time_hours,
);
controller.add_dispatcher("tsdb", timescale_dispatcher);

//    A 50MB CSV file that will be wrapped and overwritten when full
let csv_dispatcher: Box<dyn Dispatcher> =
    CsvDispatcher::new(50, dispatcher::Overflow::Wrap);
controller.add_dispatcher("csv", csv_dispatcher);

// Associate hardware peripherals that we expect to find on the network
// The controller can also run with no peripherals at all, and simply do
// calculations on a fixed time interval.
controller.add_peripheral("p1", Box::new(AnalogIRev3 { serial_number: 1 }));
controller.add_peripheral("p2", Box::new(AnalogIRev3 { serial_number: 2 }));

// Set up calcs that will be run at each cycle
//     Add a constant for duty cycle and a sine wave for frequency
let freq = Sin::new(1.0 / (rate_hz / 100.0), 0.25, 100.0, 250_000.0, true);
let duty = Constant::new(0.5, true);
controller.add_calc("freq", freq);
controller.add_calc("duty", duty);
//     Set a PWM on the first peripheral to change its frequency in time
controller.set_peripheral_input_source("p1.pwm0_freq", "freq.y");  // A value to be written to the hardware
controller.set_peripheral_input_source("p1.pwm0_duty", "duty.y");
//     Set a PWM frequency on one peripheral based on a measured temperature from the other peripheral
controller.set_peripheral_input_source("p2.pwm0_duty", "duty.y");  // Values can be referenced any number of times
controller.set_peripheral_input_source("p2.pwm0_freq", "p1_rtd_5.temperature_K");

// Serialize and deserialize the controller (for demonstration purposes).
// All of the configuration up to this point, including any custom peripheral plugins
// or user-defined calcs, are serialized with the controller and can be written to and read from a json file.
let serialized_controller: String = serde_json::to_string_pretty(&controller).unwrap();
let _deserialized_controller: Controller = serde_json::from_str(&serialized_controller).unwrap();


// Run the control program
// (skipped here because there are no peripherals
// or databases on the network in the test environment).
// controller.run(&peripheral_plugins, None);

Re-exports§

pub use controller::Controller;
pub use controller::RunHandle;
pub use controller::Snapshot;
pub use controller::context::ControllerCtx;
pub use controller::context::LoopMethod;
pub use controller::context::LossOfContactPolicy;
pub use controller::context::Termination;
pub use dispatcher::ChannelFilter;
pub use dispatcher::CsvDispatcher;
pub use dispatcher::DecimationDispatcher;
pub use dispatcher::Dispatcher;
pub use dispatcher::LowPassDispatcher;
pub use dispatcher::Overflow;
pub use socket::unix::UnixSocket;
pub use socket::Socket;
pub use socket::SocketAddr;
pub use socket::SocketId;
pub use socket::thread_channel::ThreadChannelSocket;
pub use socket::udp::UdpSocket;
pub use dispatcher::DataFrameDispatcher;
pub use dispatcher::TimescaleDbDispatcher;
pub use peripheral::HootlDriver;
pub use peripheral::HootlPeripheral;
pub use peripheral::HootlRunHandle;
pub use peripheral::HootlTransport;

Modules§

calc: Calculations that are run at each cycle during operation.
controller: Control loop and integration with data pipeline and calc orchestrator
dispatcher: Dispatchers send data to an outside consumer, usually a database or display
logging
math
peripheral: Peripherals are timing-controlled external I/O modules, usually a DAQ
python
socket: Packetized socket interface for message-passing to/from peripherals on different I/O media.

Macros§

calc_config: Build functions for getting and setting calc config fields
calc_input_names: Build function for getting calc input field names
calc_output_names: Build function for getting calc output field names
py_json_methods: Generate a pymethods block with a provided py_new plus to_json/from_json.
py_peripheral_methods: Generate Python bindings and JSON helpers for peripherals.

Constants§

SOCKET_BUFFER_LEN: [bytes] Fixed maximum size of socket packets.

Crate deimos

Crate deimos Copy item path

§Deimos

§Install - Rust

§Install - Python

§Features & Roadmap

§Concept of Operation

§Example: 200Hz Control Program w/ 2 DAQs

Re-exports§

Modules§

Macros§

Constants§

Crate deimos