Expand description
Core types shared by every OpenReadout format reader and front end.
The vocabulary here is deliberately vendor-neutral and mirrors the OME data model
(the open standard for microscopy metadata), so that an agent sees the same field
names whether a file came from ZEISS, Nikon or Leica. Vendor-specific metadata is
preserved verbatim under a separate vendor tree.
§Overview
FormatReaderis implemented once per file format (in theopenreadout-<format>crates). It recognises a file from its first bytes and opens it.Datasetis an opened file.info()reads headers only and returns aFileInfo; pixels, table rows, spectra and signal samples are read lazily, one piece at a time.Registryholds readers in detection order and picks the right one for a file.sourceis where the bytes come from: a local file, a buffer in memory, or a callback the host provides (a Python file-like object, a browserFile). Readers open anInputand read it throughSourceFileandFs, which behave likestd::fs.modelis the normalized metadata. A file holds any mix of images (microscopy planes), tables (flow-cytometry events, plate-reader wells), spectra (mass spectrometry scans) and traces (uniformly sampled signals: electrophysiology sweeps, chromatograms, NMR signals).Errorcarries a stable code, a process exit code and an actionable hint;Envelopeis the JSON wrapper every--jsoncommand prints.provenancerecords where the meaning of each normalized field came from.experimentderives the experiment (sample, instrument, method, acquisition, measurements) from the normalized model, with terms fromvocab;infoprints it asInfoOutput.statsandtraceimplement those commands;batchandselectparse their inputs. Other command-level operations (compare,info --view explain,extract,--only) live in theopenreadout-opscrate.livejudges whether an incomplete file is still being written (acquisitionininfo,info --view structure,checkandplanes).bytes,cfb,xml,xmljson,zip,timeandparallelare helpers for format crates.
§Exit codes
Part of the public interface (agents branch on them):
| code | meaning | Error variant |
|---|---|---|
| 0 | success | |
| 1 | other error | Error::Other |
| 2 | bad arguments or impossible request | Error::Usage |
| 3 | unrecognised file format | Error::UnknownFormat |
| 4 | corrupt or truncated file | Error::Corrupt |
| 5 | I/O error | Error::Io |
| 6 | known format, unsupported feature | Error::Unsupported |
§Example
use std::path::Path;
use openreadout_core::{Error, Registry};
// Format crates register their readers: `Registry::new().with(Box::new(CziReader))`.
// An empty registry recognises nothing, which shows the error contract.
let registry = Registry::new();
let err = registry
.detect_bytes(b"not an instrument file", Path::new("notes.txt"))
.map(|_| ())
.unwrap_err();
assert!(matches!(err, Error::UnknownFormat { .. }));
assert_eq!(err.code(), "unknown_format");
assert_eq!(err.exit_code(), 3);
assert!(err.hint().is_some());Enums and option structs that will grow are #[non_exhaustive]: match them with a
wildcard arm, and build option structs from Default::default().
Re-exports§
pub use envelope::Envelope;pub use envelope::ErrorBody;pub use envelope::SCHEMA_VERSION;pub use error::Error;pub use error::Result;pub use experiment::Experiment;pub use experiment::InfoOutput;pub use pixel::PixelType;pub use pixel::Plane;pub use provenance::Confidence;pub use provenance::ProvenanceMap;pub use provenance::Source;pub use reader::Dataset;pub use reader::Detection;pub use reader::FormatReader;pub use reader::PlaneIndex;pub use reader::Registry;pub use reader::SpectrumView;pub use region::Region;pub use region::ResolutionLevel;pub use scans::ScanFilter;pub use scans::ScanHeader;pub use scans::ScanList;pub use source::ByteSource;pub use source::DirSource;pub use source::Fs;pub use source::Input;pub use source::SourceFile;pub use model::*;
Modules§
- acquisition_
mode - One readable vocabulary for
ChannelInfo::acquisition_mode, and its mapping to and from the OME 2016-06Channel/@AcquisitionModeandChannel/@ContrastMethodenumerations (open standard, https://www.openmicroscopy.org/Schemas/OME/2016-06). - annotation
- The caller’s marks on a picture: one shape for previews (
annotations) and plots (annotate). Coordinates are those the picture shows: full-resolution pixels on images (as on the rulers), axis units on plots (as on the tick labels), wells on plates. - assurance
- Per-file assurance: does this file lie inside what the reader has been validated on?
- batch
- Batch inputs: turn command-line paths (files, directories, glob patterns) into the list of items a batch command processes, in a deterministic order.
- bytes
- Bounds-checked access to byte buffers read from a file, shared by the binary format readers.
Every accessor returns
Nonepast the end of the buffer, so a short or truncated file never panics. - cfb
- A reader for compound files (OLE2 structured storage), written from Microsoft’s public
“[MS-CFB]: Compound File Binary File Format” open specification: header, FAT/DIFAT, mini
FAT, directory tree and stream reads. Shared by the readers of compound-file formats
(Shimadzu
.lcd/.gcd, Zeiss.zvi); errors carry the calling format’s id. - diffraction
- Diffraction geometry shared by the area-detector readers (CBF, SMV, NeXus NXmx): what a
reader knows about the beam, the detector and the rotation, stored on an image as
extra.diffractionin one shape, so thatinfo, the experiment summary and the preview (log contrast, masked pixels, resolution rings) treat every diffraction format alike. - envelope
- The JSON envelope every command emits with
--json. - error
- Error type and the exit-code contract.
- experiment
- The experiment model: what was measured, on what, how, by whom and when, in the same shape
for a confocal stack, an LC-MS run and a plate read (
book/src/guides/metadata.md). - flow
- JSON shapes of flow-cytometry analysis:
openreadout analyze gate/openreadout_gategate(FlowJo workspace or Gating-ML populations with event counts) and the processing record thattable --compensate/--transform/--gatingml/--workspaceattaches to a table slice. The computation lives in the FCS reader crate; these types are shared by the CLI and MCP. - gzip
- Random access into gzip-compressed files (
.mzML.gz,.mzXML.gz, …). - jet
- A reader for Microsoft Jet 4 / ACE databases (the
.mdb/.accdbcontainer that Arbin.resresult files use): the catalog, table definitions and the rows of a table, column by column. Written from the documentation of two permissively licensed readers, access_parser (Apache-2.0) and Jackcess (Apache-2.0); seedocs/provenance/arbin-res.md. Read-only; encrypted and Jet 3 (Access 97) databases are refused. Errors carry the calling format’s id. - limits
- Allocation guards shared by every reader.
- live
- Files that are still being written: the reader’s structural evidence (
WriteState) and the in-progress heuristic (assess) that turns it into anAcquisitionstatus. - measure
- Measurements on one image plane: an intensity profile along a line, statistics inside a polygon, and objects found by a threshold (connected components with their sizes), so that “how bright is the membrane”, “what is in this region” and “how many nuclei” each take one call.
- model
- The normalized metadata model. Field names follow the OME data model wherever one exists, so agents can rely on a single vocabulary across vendors.
- output
- Helpers for commands that write files: refusing an output that is one of the inputs, and writing through a temporary file that is read back before it is renamed into place.
- parallel
- Parallel plane decoding with ordered delivery.
- pixel
- Pixel types and the in-memory plane representation.
- plate
- Multi-well plates (high-content screening): the plate layout
inforeports underplate, well names, and per-well pixel statistics (stats --per well, MCPopenreadout_statswithper: "well"). - provenance
- Honest confidence: where each normalized field’s meaning came from.
- reader
- The traits every format crate implements, and the registry that dispatches on them.
- region
- Region reads and resolution levels.
- scans
- Header-only listing of the spectra (scans) of a mass-spectrometry run: scan number, MS level, retention time, polarity, precursor m/z and charge, isolation window, activation, filter string, and the instrument’s stored TIC/base peak where the file keeps them, without decoding any peak arrays.
- select
- Plane selection syntax shared by every command that reads planes:
c=0,z=2-5,t=0,3,7, as separate arguments or combined in one (c=0,1,z=2). - series
- A ready-made
Datasetfor readers whose files are small sets of sampled columns: an EPR spectrum, a diffraction scan, a potentiostat or battery-cycler record, a thermal-analysis run. Such a reader parses the whole file at open into aSeriesFile(traces of named channels, tables, experiment facts, the vendor tree, findings) and wraps it in aSeriesDataset, which servesinfo,trace,table,check,info --view structureandinfo --view fullfrom memory. - session
- Recording sessions: the files of one recording, written side by side in a directory by the acquisition software (a Neuralynx session folder, a SpikeGLX run folder, a Blackrock NSx + NEV set, an Intan “one file per channel” folder), opened as one dataset.
- source
- Where a reader’s bytes come from: a local file, a buffer in memory, or a callback the host
provides (a Python file-like object, a browser
File, an S3 or HTTP range reader written by the host). The binary itself never opens a network connection (rule 5): remote sources are implemented by the program embedding the library and handed in as aCallbackSource. - stats
- Pixel statistics (
openreadout stats, MCPopenreadout_stats): per plane, per channel and per image (the whole selection), streaming one plane at a time per worker. - strict
--strict: a dataset wrapper that refuses outputs its file’s assurance does not validate.- time
- Minimal timestamp helpers (no chrono dependency in the core).
- trace
- Sampled-signal slices for JSON consumers (
openreadout trace, theopenreadout_traceMCP tool): a window of one sweep, per-channel statistics over the whole window, and at most a capped number of the samples themselves. - transitions
- The SRM/MRM transitions of a run as one table, so
infocan state how many there are andtablecan list them. Readers build the rows from what the file records (method transitions, or the scan records of each transition) and say how they counted. - vocab
- Controlled vocabulary: the ontology terms and UCUM unit codes the experiment model emits.
- xml
- Element lookups over a parsed
roxmltreedocument, matching elements by local name (any namespace), shared by the readers of XML metadata. - xmljson
- Lossless-enough XML → JSON conversion for vendor metadata blocks.
- zip
- ZIP archives (PKWARE APPNOTE, public specification): a read-only index over the central directory with stored and deflated members, partial reads of stored members, and a small deterministic writer.