createrepo_rs ๐ฆ
100% pure Rust implementation of createrepo_c โ generates RPM repository metadata (repodata).
Drop-in replacement for the C version with identical output, zero FFI, and 3.5MB static binary.
๐ฏ Why createrepo_rs?
| createrepo_c (C) | createrepo_rs (Rust) | |
|---|---|---|
| Output compatibility | โ | โ 100% byte-compatible |
| CLI arguments | 55 | 52/55 (3 hard: split, zchunk) |
| Dependencies | librpm, libxml2, glib2, zchunk... | zero FFI โ pure Rust crates |
| Binary size | ~200KB + shared libs | 3.5MB static (musl) |
| Memory safety | โ manual malloc/free | โ borrow checker |
| Cross-compile | painful | cargo zigbuild |
dnf compatible |
โ | โ verified |
๐ Quick Start
# Install
# Or clone and build
# Generate metadata for a directory of RPMs
# Or with options
๐ฆ Features
Core
- โ primary.xml, filelists.xml, other.xml generation
- โ repomd.xml with correct checksums
- โ
SQLite database generation (
--no-databaseto disable) - โ
Multi-threaded RPM parsing (configurable
--workers) - โ Changlog extraction
- โ Provides/Requires/Conflicts/Obsoletes/Suggests/Recommends
- โ Weak dependencies (Supplements/Enhances)
- โ File type detection (dir/symlink/regular)
- โ Graceful Ctrl+C handling
- โ
--updateincremental mode
Compression
- โ gzip (default)
- โ bzip2
- โ zstd
- โ xz
- โ Configurable level
CLI (52/55 params โ 100% of commonly used)
Notable:
| Flag | Description |
|---|---|
--workers=N |
Parallel RPM parsing threads |
--compress-type=zstd |
Compression algorithm |
--checksum=sha512 |
Hash algorithm for metadata |
--revision=12345 |
Custom repository revision |
--baseurl=https://... |
Base URL for repository |
--simple-md-filenames |
Clean filenames (no hash prefix) |
--unique-md-filenames |
Hash-prefixed filenames (default) |
--location-prefix=subdir/ |
Prefix before location href |
--cut-dirs=2 |
Strip N directory components |
--repomd-checksum=sha512 |
Checksum type for repomd.xml |
--general-compress-type=xz |
Separate compression for XML files |
--duplicated-nevra=error |
Error on duplicate packages |
--retain-old-md-by-age=30d |
Auto-cleanup old metadata |
--compatibility |
Max compatibility mode (gzip + simple filenames) |
-q / -v |
Quiet / Verbose output |
--no-pretty |
Compact XML without indentation |
๐ Performance
Tested on macOS (M1 Pro, 16GB RAM), createrepo_c run in Docker (Fedora 40).
All tests use --compress-type=zstd --no-database unless noted.
Full Generation Time (lower is better)
| RPMs | createrepo_rs (4w) | createrepo_c (4w) | Speedup |
|---|---|---|---|
| 10 | 0.01s | 0.10s | 10x |
| 100 | 0.01s | 0.11s | 11x |
| 500 | 0.03s | 0.12s | 4x |
| 1000 | 0.05s | 0.20sยน | 4xยน |
ยน Estimated: createrepo_c ~O(n) for RPM parsing
Worker Scaling (500 RPMs)
| Workers | createrepo_rs | createrepo_c |
|---|---|---|
| 1 | 0.05s | 0.25s |
| 4 | 0.03s | 0.12s |
| 8 | 0.03s | โ |
Compression Comparison (500 RPMs, 4 workers)
| Algorithm | Time | Output Size | Best For |
|---|---|---|---|
| zstd | 0.03s | 20KB | Speed + size (default) |
| gzip | 0.03s | 40KB | Max compatibility |
| xz | 0.11s | 20KB | Smallest output |
| bzip2 | 0.14s | 24KB | Legacy support |
Incremental Update (--update --skip-stat)
| RPMs | createrepo_rs | Notes |
|---|---|---|
| 100 | 0.01s | Cache hit, skips re-parsing |
| 500 | 0.03s | Only processes changed packages |
| 1000 | 0.05s | Near-constant time for unchanged repos |
Optimizations Applied (v0.1.4)
| Optimization | Impact |
|---|---|
| LTO + opt-level=3 + panic=abort | ~5-10% runtime, ~7% binary size |
| Multi-worker deadlock fix | Enables 2+ workers (was broken) |
| Arc<Package> update cache | Avoids full clone on --update hits |
| SQLite batch transactions | 10-50x filelists insert speed |
| Redundant stat() eliminated | -2 syscalls per XML file |
| XML Vec::with_capacity | Avoids reallocation during dump |
| SHA buffer 8KBโ64KB | Fewer read() syscalls |
| From<DependencyInfo> impl | -114 lines duplicated code |
Primary XML generation is byte-identical to the createrepo_c C version.
๐ณ Docker Test
Output:
โ
โ
โ
Success! createrepo_rs generated metadata recognized and downloaded by dnf!
๐จ Building
# Debug build
# Release build
# Linux static binary (musl) โ requires Zig
# Install: https://ziglang.org/download/
# Then: cargo install cargo-zigbuild
# Cross-compile for ARM
๐๏ธ Architecture
createrepo_rs/
โโโ lib.rs # Library root + prelude re-exports
โโโ src/main.rs # Binary entry point, CLI handling, orchestration
โโโ cli/mod.rs # Clap argument parser (52/55 params)
โโโ pool/mod.rs # Parallel worker pool
โโโ rpm/mod.rs # RPM header parsing via `rpm` crate
โโโ types/mod.rs # Core types: Package, Dependency, RepomdRecord
โโโ compression/ # gzip, bzip2, zstd, xz
โโโ db/mod.rs # SQLite database generation
โโโ xml/
โ โโโ error.rs # XML error types
โ โโโ mod.rs # XML helpers
โ โโโ parse.rs # XML parsing
โ โโโ repomd.rs # repomd.xml generation
โโโ walk/mod.rs # Directory traversal
๐ Library Usage
createrepo_rs can also be used as a library:
[]
= { = "https://github.com/jamesarch/createrepo_rs" }
use Path;
use *;
let mut reader = open.unwrap;
let pkg = reader.read_package.unwrap;
println!;
The [prelude] module re-exports commonly used types and functions:
- Compression:
gzip_compress,gzip_decompress,zstd_compress,zstd_decompress, etc. - Types:
Package,Dependency,ChecksumType,CompressionType,ChangelogEntry - RPM:
RpmReader,parse_dep_version - DB:
RepomdDb,DbError - Pool:
WorkerPool,Job,ProcessingResult - XML:
XmlError - Walk:
DirectoryWalker,WalkError
๐ License
GPL-2.0-or-later โ same as createrepo_c.
๐ Credits
Original C implementation: rpm-software-management/createrepo_c
Built with:
- rpm-rs/rpm โ Pure Rust RPM parser
- dralley/rpmrepo_metadata โ EVR parsing reference (by @dralley at Red Hat)
- quick-xml โ Fast XML writer
- rusqlite โ SQLite bindings