Ecological State Toolkit
Breaking 0.12 update:
ecological-model-coreis superseded by theecological-state-toolkitcrate andecological_state_toolkitRust/Python import. There are no compatibility aliases. The ecological schema provider identity is nowecological-state-toolkit.ecological-state.v1.
An end-to-end, model-neutral package for ecological state construction, validation, schema provision, trajectory products, and conversion of completed Workflow recordings into analysis-efficient NumPy arrays. It deliberately contains no simulation engine or Workflow task: each downstream model and study remains independently runtime-integrated.
The public modules are:
artifact: shared immutable descriptor, publication disposition, and typed integrity failures; byte publication itself remains behind the ecological modules that know the document semantics;initial_state: reproducible categorical lattice initial states and verified ecological input artifacts;inputs: the canonical validated(InteractionMatrix, InitialState)envelope shared by independent engines;interaction: validated ecological interaction matrices, reproducible random sources, and composable PiP-backed transformations;state_schema: the sole embedded canonical ecological Workflow state schema and its typed provider descriptor;trajectory: a bounded, allocation-conscious trajectory observer with disabled, terminal-only, and detection modes;terminal_state: validated terminal composition and auditable equilibrium, periodic-orbit, and absorbing-state results.
The companion Python module ecological_state_toolkit exposes generic,
verified recording conversion; its API is documented below.
Construction parameters are called recipes, keeping scientific realization instructions distinct from Workflow configuration. Engines submit borrowed abundance slices, so continuous and lattice models retain their native state representations and only copy the bounded samples the selected observer mode needs.
InitialStateRecipe::BalancedUniform constructs the unique minimum-spread
uniform count allocation: every taxon receives either floor(N/K) or
ceil(N/K) sites. PiP then applies one reproducible unbiased permutation to
the complete row-major lattice tensor. The resulting counts differ by at most
one, and InitialState::frequencies exposes their exact aggregate composition
for a continuous model using the same initial condition.
The Rust crate depends on Scientific Workflow only for the neutral
StateSchemaProvider descriptor. It never reads or resolves a Workflow
project, implements an execution unit, or writes a Workflow recording. It has
no dependency on Dispatcher, GLV, Simulator, or another private downstream
crate. Callers provide ordinary resolved values (Vec, slices, PathBuf,
recipes, and descriptors); an orchestrator or model owns configuration
decoding and path-key resolution.
For comparative applications, inputs::EcologicalInputs is the canonical
model-neutral envelope. It pairs one final model-ready
InteractionArtifactReference with one InitialStateArtifactReference,
validates their shared taxon dimension without IO, and resolves both with full
digest verification. Multiple models can receive the same initial-state
reference: a categorical model consumes its space and counts, while a
continuous model derives exact frequencies from that same state. The envelope
does not select recipes, apply model-specific transformations, or own runtime
configuration.
After verification, ResolvedEcologicalInputs is exactly the typed wrapper
around (InteractionMatrix, InitialState). Models may borrow the two values or
call into_parts() to take that tuple directly; Ecological State Toolkit does
not construct a GLV- or Simulator-specific state around it.
Installation
[]
= "0.12.2"
= "0.11.5"
= "3.7.0"
The companion Python distribution is installed from the same tagged source:
python -m pip install \
"ecological-state-toolkit @ git+https://github.com/dingyisun0101/Ecological-Model-Core.git@v0.12.2#subdirectory=python"
Use this crate when multiple ecological models need the same validated initial-state, interaction, trajectory, or terminal-product semantics. A model that needs only one small local calculation may be clearer without the extra dependency.
When used with Scientific Workflow 0.11.5, put recipes and other resolved
scientific values in the model's custom Constants type. The registered model
still directly owns its Workflow SystemState; Ecological State Toolkit owns
only the standard layout supplied to that state, not the model, observation
plan, recording destination, or runtime.
Recording conversion
The Python distribution is the analysis-facing half of the toolkit. It uses the official Workflow reader to verify every JSONL chunk and writes each selected stream directly into atomic, C-contiguous NPY memmaps. Conversion is bounded by a caller-selected process count and resumes at recording granularity only when both the source metadata checksum and complete conversion request still match.
Applications use the library API as the stable boundary:
=
=
= .
RecordingSpec identifies one completed recording, its generic stream
contracts, and optional JSON metadata retained for its caller. StreamSpec
names one Workflow stream. FieldSpec maps a field to an output basename and
one ArrayEncoding: f64 tensor, nonnegative f64 tensor/vector, nonnegative
u32 vector, categorical lattice, finite float scalar, or nonnegative integer
scalar. Categorical
lattices additionally require category_count, which determines validation
and the smallest safe unsigned storage dtype. convert_recordings takes a
sequence of specifications, an output Path, a positive worker count, and an
optional parent-process progress callback. It returns typed
ConvertedRecording values with array descriptors, source checksums, source
user metadata, and caller metadata.
For shell use, ecological-state-convert --request REQUEST.json --output DIR
accepts an ecological-state-toolkit.conversion-request.v1 document and
publishes a generic batch manifest. This CLI is deliberately a thin adapter
over the library, not a raw script API.
Conversion is not an ExecutionUnit: it consumes finalized recordings after
scientific execution and has no evolving state, step, completion, or ensemble
semantics. A study may call the library after its export phase and then add its
own selection, pairing, and manifest semantics. If conversion later needs to
be scheduled independently, a study-owned program/task adapter should invoke
this same library without moving study semantics into the toolkit.
Canonical ecological state schema
Ecological State Toolkit is the sole source of
schemas/ecological_state.json. A receiving model does not vendor or load
another copy. It forwards the provider from its Workflow execution-unit
implementation:
use ecological_state_schema;
use *;
The model's study.json task can then omit state, and paths.states can be
absent when no project-owned schema is needed. Study validates the embedded
document once, passes the resulting SystemStateSchema directly to the unit,
and records ecological-state-toolkit.ecological-state.v1 as state provenance.
An explicit project state remains a deliberate override.
Verified ecological inputs
Initial states and interaction matrices can be published as immutable, content-addressed inputs beneath an explicit artifact root:
use Path;
use ArtifactDisposition;
use ;
use ;
let initial = BalancedUniform .create?;
let root = new;
let persisted = persist_initial_state?;
assert_eq!;
let restored = load_verified_initial_state?;
assert_eq!;
# Ok::
Publication creates <artifact-root>/inputs, writes and synchronizes complete
temporary bytes, then publishes a digest-derived immutable filename. Repeating
the same publication returns Reused; conflicting or corrupted bytes fail.
Loading rejects malformed descriptors, path traversal, symlink escape, and
SHA-256 mismatch before semantic decoding.
These are application-prepared scientific inputs, not an alternative Workflow recording writer. During a Workflow model run, Workflow Persistence remains the sole owner of model recording output.
Migrating from 0.10
- Replace
scientific_workflow::ArtifactDispositionwithecological_state_toolkit::artifact::ArtifactDisposition. - Pass an artifact-root
&Pathtopersist_initial_stateorpersist_interaction_matrix; do not construct a WorkflowExecutionScope. - Pass the same root to verified loading. Serialized references now expose
artifact_root()and serializeartifact_root, notexecution_directory. - Replace initial-state
rng_record()withrng_config()and interactiongenerator_rng_record()withgenerator_rng_config(). Both return the resolved PiP method and seed directly. - Remove the old initial-state and interaction RNG namespace constants. The domain identity/version live in the typed ecological provenance; PiP owns RNG method identity and seed encoding.
- Regenerate v1 initial-state documents and portable references. Version 0.11 deliberately provides no legacy parser or alias.
Interaction matrices are sampled before model-specific transformations are
applied. Every entry of RandomUniform and RandomGaussian, including the
diagonal, is sampled independently. Transformations return new matrices, leave
their sources unchanged, and retain a complete derived-provenance chain:
use InteractionMatrixRecipe;
use RngConfig;
let recipe = RandomGaussian ;
let lattice = recipe.generate?
.scale?
.abs?
.normalize?;
lattice.ensure_max_abs_at_most?;
let antisymmetric = lattice.antisymmetrize?;
# Ok::
abs applies elementwise absolute value. clamp_min and clamp_max provide
independent finite lower and upper bounds. ensure_max_abs_at_most validates
a bound without changing the matrix, which is useful when exceeding the
threshold should be an error.
Matrix constructors infer the species count from the square PiP matrix; callers
do not repeat it. Only random recipes require a species count because it defines
the matrix that has not yet been created. InteractionMatrix delegates scalar,
transpose, subtraction, absolute-value, reduction, and vector-application work
to PiP. mul_vectors_into applies the same matrix to a contiguous batch while
reusing caller-owned output storage.
use ;
let observer = from_policy?;
assert!;
# Ok::
The crate is a clean-start API. It does not read artifacts produced by the
former ecological-initial-state crate or ecological-model-core 0.10.
Initial-state documents use ecological.initial-state.v2; serialized
initial-state and interaction references use their respective *.v2 formats.