Expand description
§fountain-arena-operator
An owned, contiguous-memory implementation of fountain_engine::DataOperator,
with reusable vector slots and CPU arithmetic with portable fallback. It supports the
Fountain 2.x operation interface and can be passed directly to its encoders
and decoders.
This repository contains the operator extracted from CART for independent maintenance and review by Fountain maintainers. It can be used as a standalone crate with the Fountain 2.x engine.
§Use the operator
Add the crates.io dependencies:
[dependencies]
fountain-arena-operator = "0.1.1"
fountain_engine = "2.0.1"The Rust import name is fountain_arena_operator. Encoding-plan caches belong
to the codec or caller; this package supplies vector storage and arithmetic.
The only normal dependency is fountain_engine, which depends on smallvec.
Raptor code schemes, the reference operator, and JSON output support are
development dependencies. The crate builds without CART, RDMA, or native libraries.
use fountain_arena_operator::MemoryDataOperator;
use fountain_engine::{DataOperator, Operation};
let mut operator = MemoryDataOperator::with_capacity(4, 3);
operator.insert_vector(&[1, 2, 3, 4], 0);
operator.insert_vector(&[4, 3, 2, 1], 1);
operator.execute(&Operation::AddTwoToVector {
s0: 0,
s1: 1,
target_id: 2,
});
assert_eq!(operator.get_vector(2), &[5, 1, 1, 5]);To use an encoder, populate source IDs 0..K before handing over the operator:
use fountain_arena_operator::MemoryDataOperator;
use fountain_engine::{DataOperator, Encoder};
use fountain_raptor_10::Raptor10SysCode;
let code = Raptor10SysCode::new_with_default_setting(32);
let mut operator = MemoryDataOperator::new(64);
for id in 0..32 {
operator.insert_vector(&vec![id as u8; 64], id);
}
let encoder = Encoder::new_with_operator_execute_only(&code, Box::new(operator));Full recovery examples:
cargo run --locked --example roundtrip -- raptor10
cargo run --locked --example roundtrip -- raptorqThe RaptorQ example uses the engine’s explicit source count to cover K < K′
padding. Packet framing, repair IDs, encoding plans, and scheduling are owned
by the codec and its caller.
The round-trip example constructs and runs the codec on a caller-owned thread with an 8 MiB stack. This accommodates the RaptorQ debug example on Windows, where the default main-thread stack overflowed. Applications should validate their codec stack budget for their own build and workload.
§Public API and behavior
MemoryDataOperator::new(width)creates an empty operator.MemoryDataOperator::with_capacity(width, count)reserves vector storage.operator.clear()removes all IDs and retains storage for another block of the same width. The backend and field configuration are preserved. This is an explicit caller decision, not an automatic codec pooling policy.MemoryDataOperator::with_backend(width, backend)selects a concreteVectorOperationBackendfor statically dispatched arithmetic.CpuVectorBackendprovides fused two/three-source XOR, scalar operations, and finite-field configuration.
Every live ID owns one slot. Move transfers ownership; removal and overwritten targets make slots reusable. The arena retains its high-water capacity until drop. Dense IDs use direct indexing, and larger IDs use sparse metadata.
The default CpuVectorBackend uses vectorizable Rust loops for fused XOR and
multiplication by 2. General GF(256) multiplication and multiply-add select NEON
on aarch64 or AVX2 on x86/x86_64 at runtime, with scalar fallback and cached
field tables.
RuntimeVectorBackend is an optional bulk-vector backend. It shares the standard
field arithmetic and selects AVX2 XOR at runtime for x86 slices of at least
128 bytes. Four independent 32-byte accumulators and bounded 16/8/1-byte tail
handling support unaligned buffers. Other CPUs and short slices fall back to
the standard loops. No target-cpu=native flag is required. Select a backend
when constructing an operator with MemoryDataOperator::with_backend_and_capacity.
The standard backend remains the default: whole-codec measurements found gains
for packet-sized symbols but regressions for short symbols in a CART integration
candidate. See the qualification report.
Storage and operation routing forbid unsafe code. Private arithmetic modules contain the SIMD intrinsics behind checked slice boundaries and CPU feature detection. Performance depends on the compiler, CPU, width, and workload.
The public contract specifies zero-length vectors, missing IDs, panic behavior,
aliasing, field configuration, and memory lifetime. It also identifies three
differences from fountain_utility 2.0.0::VecDataOperater: zero-coefficient
MulAdd, source-aliasing BroadcastAdd, and missing arithmetic targets.
See the operation contract.
§Validate and measure
Rust 1.89 is the supported compiler baseline. CI checks that version and stable.
cargo test --locked
cargo clippy --locked --all-targets -- -D warnings
cargo doc --locked --no-deps
cargo package --locked
mkdir -p results
cargo bench --locked --bench operator_comparison -- 12 > results/samples.ndjson
python3 scripts/analyze.py results/samples.ndjson --output results/summaryThe benchmark verifies bytes for every measured encoder/decoder pair. It reports both identical operation replay and the full upstream codec with only the operator replaced. Historical aarch64 measurements, raw samples, timing definitions, and reproduction instructions are in the benchmark guide. The three-scope measurements separately compare operator replay, full upstream encoding/decoding and encoding-plan cache hits for Arena, Slab and Simd, with raw rounds, setup costs and host-load evidence.
The detailed operator comparison covers current Slab, Simd and Arena performance, storage and operation semantics. The GF(256) optimization results compare Arena before and after its SIMD changes.
The reports are included in the source package. Their raw benchmark archives remain in the GitHub repository and require collaborator access while that repository is private.
§Maintenance and license
See CONTRIBUTING.md for review and validation, and the release procedure for registry releases.
This crate is licensed under AGPL-3.0-only, retaining the license of the
source from CART. fountain_engine has its own AGPL/commercial licensing terms.
See LICENSE and
provenance.
Memory initialization and explicit reuse measurements are documented in Memory reuse qualification. These APIs are available in version 0.1.1.
Structs§
- CpuVector
Backend - CPU kernels with vectorizable XOR and runtime-selected GF(256) SIMD. Binary sessions allocate no field tables. Default-field tables are shared and initialized lazily; other fields are owned by the backend.
- Memory
Data Operator - Owned equal-width vectors for incremental fountain codecs.
- Runtime
Vector Backend - CPU backend for bulk vectors, with runtime-selected AVX2 XOR on x86.
Traits§
- Vector
Operation Backend - Arithmetic on equal-length, disjoint source and target slices. The operator resolves IDs and aliasing before calling the backend. A concrete backend is selected at construction so its contiguous byte kernels can be inlined.