Crate webgraph_cli 

CLI for WebGraph

Command-line interface for the Rust implementation of the WebGraph framework for graph compression.

This crate provides the webgraph CLI tool with various subcommands for working with compressed graphs. The tool supports:

• Building accessory data structures (Elias-Fano offsets, DCF)
• Converting graphs between formats (arcs, ASCII, endianness)
• Transforming graphs (transpose, simplify)
• Analyzing graphs (code statistics)
• Running algorithms (Layered Label Propagation)
• Benchmarking graph operations

Each module correspond to a group of commands, and each command is implemented as a submodule.

Subcommands

• analyze: Compute statistics on graphs
• bench: Benchmark graph operations
• build: Build accessory graph data structures
• check: Check coherence of graph files
• from: Ingest data into graphs
• perm: Create and manipulate permutations
• run: Run algorithms on graphs
• to: Convert graphs between representations
• transform: Apply transformations to graphs

Separate Binaries

This crate also provides specialized standalone binaries:

• webgraph-dist: Tools for computing graph properties based on distances (HyperBall, ExactSumSweep)
• webgraph-sccs: Compute strongly connected components

Environment Variables

• RUST_MIN_STACK: Minimum thread stack size (in bytes); we suggest RUST_MIN_STACK=8388608 (8MiB)
• TMPDIR: Where to store temporary files (potentially very large ones)
• RUST_LOG: Configuration for env_logger

Acknowledgments

This software has been partially supported by project SERICS (PE00000014) under the NRRP MUR program funded by the EU - NGEU, and by project ANR COREGRAPHIE, grant ANR-20-CE23-0002 of the French Agence Nationale de la Recherche. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the Italian MUR. Neither the European Union nor the Italian MUR can be held responsible for them.

Modules

analyze

bench

build

build_info

check

dist

from

perm

run

sccs

to

transform

Structs

ArcsArgs

Shared CLI arguments for reading files containing arcs.

Cli

Webgraph tools to build, convert, modify, and analyze graphs. Noteworthy environment variables:

CompressArgs

Shared CLI arguments for compression.

GlobalArgs

GranularityArgs

Shared CLI arguments for commands that specify a granularity.

MemoryUsageArg

Shared CLI arguments for commands that specify a memory usage.

NumThreadsArg

Shared CLI arguments for commands that specify a number of threads.

Enums

FloatVectorFormat

How to store vectors of floats.

IntVectorFormat

How to store vectors of integers.

PrivCode

Enum for instantaneous codes.

SubCommands

Functions

append

Appends a string to the filename of a path.

cli_main

The entry point of the command-line interface.

create_parent_dir

Creates all parent directories of the given file path.

get_thread_pool

Creates a ThreadPool with the given number of threads.

init_env_logger

Initializes the env_logger logger with a custom format including timestamps with elapsed time since initialization.

memory_usage_parser

Parses a batch size.

num_threads_parser

Parses the number of threads from a string.