Skip to main content

Crate openreadout_core

Crate openreadout_core 

Source
Expand description

Core types shared by every OpenReadout format reader and front end.

The vocabulary here is deliberately vendor-neutral and mirrors the OME data model (the open standard for microscopy metadata), so that an agent sees the same field names whether a file came from ZEISS, Nikon or Leica. Vendor-specific metadata is preserved verbatim under a separate vendor tree.

§Overview

  • FormatReader is implemented once per file format (in the openreadout-<format> crates). It recognises a file from its first bytes and opens it.
  • Dataset is an opened file. info() reads headers only and returns a FileInfo; pixels, table rows, spectra and signal samples are read lazily, one piece at a time.
  • Registry holds readers in detection order and picks the right one for a file.
  • source is where the bytes come from: a local file, a buffer in memory, or a callback the host provides (a Python file-like object, a browser File). Readers open an Input and read it through SourceFile and Fs, which behave like std::fs.
  • model is the normalized metadata. A file holds any mix of images (microscopy planes), tables (flow-cytometry events, plate-reader wells), spectra (mass spectrometry scans) and traces (uniformly sampled signals: electrophysiology sweeps, chromatograms, NMR signals).
  • Error carries a stable code, a process exit code and an actionable hint; Envelope is the JSON wrapper every --json command prints.
  • provenance records where the meaning of each normalized field came from.
  • experiment derives the experiment (sample, instrument, method, acquisition, measurements) from the normalized model, with terms from vocab; info prints it as InfoOutput.
  • stats and trace implement those commands; batch and select parse their inputs. Other command-level operations (compare, info --view explain, extract, --only) live in the openreadout-ops crate.
  • live judges whether an incomplete file is still being written (acquisition in info, info --view structure, check and planes).
  • bytes, cfb, xml, xmljson, zip, time and parallel are helpers for format crates.

§Exit codes

Part of the public interface (agents branch on them):

codemeaningError variant
0success
1other errorError::Other
2bad arguments or impossible requestError::Usage
3unrecognised file formatError::UnknownFormat
4corrupt or truncated fileError::Corrupt
5I/O errorError::Io
6known format, unsupported featureError::Unsupported

§Example

use std::path::Path;

use openreadout_core::{Error, Registry};

// Format crates register their readers: `Registry::new().with(Box::new(CziReader))`.
// An empty registry recognises nothing, which shows the error contract.
let registry = Registry::new();
let err = registry
    .detect_bytes(b"not an instrument file", Path::new("notes.txt"))
    .map(|_| ())
    .unwrap_err();
assert!(matches!(err, Error::UnknownFormat { .. }));
assert_eq!(err.code(), "unknown_format");
assert_eq!(err.exit_code(), 3);
assert!(err.hint().is_some());

Enums and option structs that will grow are #[non_exhaustive]: match them with a wildcard arm, and build option structs from Default::default().

Re-exports§

pub use envelope::Envelope;
pub use envelope::ErrorBody;
pub use envelope::SCHEMA_VERSION;
pub use error::Error;
pub use error::Result;
pub use experiment::Experiment;
pub use experiment::InfoOutput;
pub use pixel::PixelType;
pub use pixel::Plane;
pub use provenance::Confidence;
pub use provenance::ProvenanceMap;
pub use provenance::Source;
pub use reader::Dataset;
pub use reader::Detection;
pub use reader::FormatReader;
pub use reader::PlaneIndex;
pub use reader::Registry;
pub use reader::SpectrumView;
pub use region::Region;
pub use region::ResolutionLevel;
pub use scans::ScanFilter;
pub use scans::ScanHeader;
pub use scans::ScanList;
pub use source::ByteSource;
pub use source::DirSource;
pub use source::Fs;
pub use source::Input;
pub use source::SourceFile;
pub use model::*;

Modules§

acquisition_mode
One readable vocabulary for ChannelInfo::acquisition_mode, and its mapping to and from the OME 2016-06 Channel/@AcquisitionMode and Channel/@ContrastMethod enumerations (open standard, https://www.openmicroscopy.org/Schemas/OME/2016-06).
annotation
The caller’s marks on a picture: one shape for previews (annotations) and plots (annotate). Coordinates are those the picture shows: full-resolution pixels on images (as on the rulers), axis units on plots (as on the tick labels), wells on plates.
assurance
Per-file assurance: does this file lie inside what the reader has been validated on?
batch
Batch inputs: turn command-line paths (files, directories, glob patterns) into the list of items a batch command processes, in a deterministic order.
bytes
Bounds-checked access to byte buffers read from a file, shared by the binary format readers. Every accessor returns None past the end of the buffer, so a short or truncated file never panics.
cfb
A reader for compound files (OLE2 structured storage), written from Microsoft’s public “[MS-CFB]: Compound File Binary File Format” open specification: header, FAT/DIFAT, mini FAT, directory tree and stream reads. Shared by the readers of compound-file formats (Shimadzu .lcd/.gcd, Zeiss .zvi); errors carry the calling format’s id.
diffraction
Diffraction geometry shared by the area-detector readers (CBF, SMV, NeXus NXmx): what a reader knows about the beam, the detector and the rotation, stored on an image as extra.diffraction in one shape, so that info, the experiment summary and the preview (log contrast, masked pixels, resolution rings) treat every diffraction format alike.
envelope
The JSON envelope every command emits with --json.
error
Error type and the exit-code contract.
experiment
The experiment model: what was measured, on what, how, by whom and when, in the same shape for a confocal stack, an LC-MS run and a plate read (book/src/guides/metadata.md).
flow
JSON shapes of flow-cytometry analysis: openreadout analyze gate / openreadout_gate gate (FlowJo workspace or Gating-ML populations with event counts) and the processing record that table --compensate/--transform/--gatingml/--workspace attaches to a table slice. The computation lives in the FCS reader crate; these types are shared by the CLI and MCP.
gzip
Random access into gzip-compressed files (.mzML.gz, .mzXML.gz, …).
jet
A reader for Microsoft Jet 4 / ACE databases (the .mdb/.accdb container that Arbin .res result files use): the catalog, table definitions and the rows of a table, column by column. Written from the documentation of two permissively licensed readers, access_parser (Apache-2.0) and Jackcess (Apache-2.0); see docs/provenance/arbin-res.md. Read-only; encrypted and Jet 3 (Access 97) databases are refused. Errors carry the calling format’s id.
limits
Allocation guards shared by every reader.
live
Files that are still being written: the reader’s structural evidence (WriteState) and the in-progress heuristic (assess) that turns it into an Acquisition status.
measure
Measurements on one image plane: an intensity profile along a line, statistics inside a polygon, and objects found by a threshold (connected components with their sizes), so that “how bright is the membrane”, “what is in this region” and “how many nuclei” each take one call.
model
The normalized metadata model. Field names follow the OME data model wherever one exists, so agents can rely on a single vocabulary across vendors.
output
Helpers for commands that write files: refusing an output that is one of the inputs, and writing through a temporary file that is read back before it is renamed into place.
parallel
Parallel plane decoding with ordered delivery.
pixel
Pixel types and the in-memory plane representation.
plate
Multi-well plates (high-content screening): the plate layout info reports under plate, well names, and per-well pixel statistics (stats --per well, MCP openreadout_stats with per: "well").
provenance
Honest confidence: where each normalized field’s meaning came from.
reader
The traits every format crate implements, and the registry that dispatches on them.
region
Region reads and resolution levels.
scans
Header-only listing of the spectra (scans) of a mass-spectrometry run: scan number, MS level, retention time, polarity, precursor m/z and charge, isolation window, activation, filter string, and the instrument’s stored TIC/base peak where the file keeps them, without decoding any peak arrays.
select
Plane selection syntax shared by every command that reads planes: c=0, z=2-5, t=0,3,7, as separate arguments or combined in one (c=0,1,z=2).
series
A ready-made Dataset for readers whose files are small sets of sampled columns: an EPR spectrum, a diffraction scan, a potentiostat or battery-cycler record, a thermal-analysis run. Such a reader parses the whole file at open into a SeriesFile (traces of named channels, tables, experiment facts, the vendor tree, findings) and wraps it in a SeriesDataset, which serves info, trace, table, check, info --view structure and info --view full from memory.
session
Recording sessions: the files of one recording, written side by side in a directory by the acquisition software (a Neuralynx session folder, a SpikeGLX run folder, a Blackrock NSx + NEV set, an Intan “one file per channel” folder), opened as one dataset.
source
Where a reader’s bytes come from: a local file, a buffer in memory, or a callback the host provides (a Python file-like object, a browser File, an S3 or HTTP range reader written by the host). The binary itself never opens a network connection (rule 5): remote sources are implemented by the program embedding the library and handed in as a CallbackSource.
stats
Pixel statistics (openreadout stats, MCP openreadout_stats): per plane, per channel and per image (the whole selection), streaming one plane at a time per worker.
strict
--strict: a dataset wrapper that refuses outputs its file’s assurance does not validate.
time
Minimal timestamp helpers (no chrono dependency in the core).
trace
Sampled-signal slices for JSON consumers (openreadout trace, the openreadout_trace MCP tool): a window of one sweep, per-channel statistics over the whole window, and at most a capped number of the samples themselves.
transitions
The SRM/MRM transitions of a run as one table, so info can state how many there are and table can list them. Readers build the rows from what the file records (method transitions, or the scan records of each transition) and say how they counted.
vocab
Controlled vocabulary: the ontology terms and UCUM unit codes the experiment model emits.
xml
Element lookups over a parsed roxmltree document, matching elements by local name (any namespace), shared by the readers of XML metadata.
xmljson
Lossless-enough XML → JSON conversion for vendor metadata blocks.
zip
ZIP archives (PKWARE APPNOTE, public specification): a read-only index over the central directory with stored and deflated members, partial reads of stored members, and a small deterministic writer.