fastpasta 1.0.5

CLI for verifying or examining readout data from the ALICE detector.
Documentation

fastPASTA

pipeline status coverage report Latest Release

fast Protocol Analysis Scanner Tool for ALICE

fastPASTA uses Semantic Versioning.

For extensive documentation of public facing source code see documentation or invoke cargo doc --open.

For an exhaustive list of the data verification done via the check subcommand, see list of checks.

Releases and associated changelogs can be found at releases.

Purpose

To verify or view curated content of the scanned raw binary data from ALICE.

Table of Contents

Quickstart

Prerequisite

The rust toolchain is required to compile the binary. Use the link to download a Windows installer. On macOS, Linux or other Unix-like OS simply run

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

and follow the on-screen instructions.

Install via cargo (comes with Rust)

cargo install fastpasta

Updating fastpasta simply requires rerunning cargo install fastpasta

Building from source

Run cargo build -r and find the binary in /target/release/fastpasta

See help, including examples of use

$ fastpasta -h

Examples of use

Read from stdin -> filter link -> view RDHs


$ lz4 -d input.raw -c | fastpasta --filter-link 3 | fastpasta view rdh
#        ^^^^                      ^^^^                       ^^^^
#       INPUT       --->          FILTER          --->        VIEW
# Decompressing with `lz4`

Piping is often optional and avoiding it will improve performance. e.g. the following is equivalent to the previous example, but saves significant IO overhead, by using one less pipe.

$ lz4 -d input.raw -c | fastpasta --filter-link 3 view rdh

Read from file -> filter by link -> validate

Enable all generic checks: sanity (stateless) AND running (stateful)

$ fastpasta input.raw --filter-link 0 check all

Same as above but only enable sanity checks

$ fastpasta input.raw check sanity -f 0

Enable all sanity and running checks and include checks applicable to ITS only

$ fastpasta input.raw check all ITS --filter-link 0

Filter link 3 and check sanity include sanity checks specific to ITS

$ fastpasta input.raw -f 3 check sanity its # target `its` is case-insensitive

Read from file -> view HBFs with less

Generate HBF view

$ fastpasta input.raw view hbf | less

View only HBFs from link #3

$ fastpasta input.raw view hbf -f 3 | less

Error messages

Messages are formatted as follows:

MEMORY_OFFSET: [ERROR_CODE] ERROR_MESSAGE

Example of failed RDH sanity check

0xE450FFD: [E10] RDH sanity check failed: data_format = 255

Error codes are not unique

But they signify categories of errors. E.g. all RDH sanity checks have the same error code, but the error message will specify which field failed. The following is a list of error codes and their meaning, x is a placeholder for any number 0-9.

  • [Ex0] - Sanity check
  • [E1x] - RDH
  • [E3x] - IHW
  • [E4x] - TDH
  • [E5x] - TDT
  • [E6x] - DDW0
  • [E7x] - Data word (Even number: IB, Odd number: OB) E70 is sanity check for both IB/OB.
  • [E8x] - CDW
  • [E99] - Miscellaneous, such as error in ID when 2 or more words could be valid in the current state.

Verbosity levels

  • 0: Errors
  • 1: Errors and warnings
  • 2: Errors, warnings and info
  • 3: Errors, warnings, info and debug
  • 4: Errors, warnings, info, debug and trace

Running tests

Run the full test suite with:

$ cargo test -- --test-threads=1 --nocapture

Passing --test-threads=1 and --nocapture is necessary as several tests asserts that content written to stdout matches expectations, which will break when tests are run concurrently or writing to stdout is suppressed.

License

Apache 2.0 or MIT at your option.

Project status

Passively Maintained. There are no plans for new features, but the maintainer intends to respond to issues that get filed.

Benchmarks and comparisons

In the tables below fastPASTA is compared with rawdata-parser and decode.py in typical verification tasks. Hyperfine is used for benchmarking, with cache warmup.

Verifying all RDHs of 260MB file with data from 1 link

Tool Command Mean [s] Min [s] Max [s]
fastPASTA fastpasta input.raw check all 0.039 ± 0.001 0.037 0.043
rawdata-parser ./rawdata-parser --skip-packet-counter-checks input.raw 0.381 ± 0.012 0.356 0.438
decode.py python3 decode.py -i 20522 -f input.raw --skip_data 13.674 ± 0.386 13.610 14.499

Verifying all RDHs in 2GB file with data from 12 different links

Tool Command Mean [s] Min [s] Max [s]
fastPASTA fastpasta input.raw check all 0.459 ± 0.018 0.411 0.535
rawdata-parser ./rawdata-parser --skip-packet-counter-checks input.raw 3.080 ± 0.047 3.005 3.178
decode.py Verifying multiple links simultaneously is not supported N/A N/A N/A

Verifying all RDHs and payloads in 260MB file with data from 1 link

Tool Command Mean [s] Min [s] Max [s]
fastPASTA fastpasta input.raw check all ITS 0.106 ± 0.002 0.103 0.111
rawdata-parser Verifying payloads is not supported N/A N/A N/A
decode.py python3 decode.py -i 20522 -f input.raw 55.903 ± 0.571 54.561 56.837

Need more performance?

The primary release profile of fastPASTA is already very fast, but if you absolutely need 10-20% more speed, a faster build profile exists that utilizes the experimental rust nightly toolchain.

Background

The rust compiler rustc does not yet provide access to all the features that its backend LLVM has. But the experimental nightly rust toolchain allows passing flags directly to LLVM. fastPASTA includes configuration for a build profile release-nightly which utilizes LLVM to achieve more speed at the cost of compilation time and binary size. As of this writing, the stable channel of Rust does not have a way to pass compiler flags to the LLVM backend. The increased speed is mainly achieved through configuring a higher threshold for inlining, which will increase speed but also compilation time and binary size, and most crucially, cache pressure. The performance impact will be highly dependent on the machine fastPASTA runs on. Better/more CPU cache will lead to a higher performance gain. With >1 GB individual link data, the performance on one particular CERN machine running CentOS Stream 8, as measured by hyperfine increased by ~17%.

To install the nightly toolchain (and check your installation)

$ rustup toolchain install nightly
$ rustup run nightly rustc --version

Compile the optimized release-nightly experimental build profile

$ cargo +nightly build --profile release-nightly

Path to binary: /target/release-nightly/fastpasta