convert-rs 2.0.0

Extensible file conversion library with graph routing and optional native tools
Documentation

convert-rs

convert-rs is a Rust library that routes files between formats through an adapter graph. It plans conversions by cost, prefers integrated Rust adapters, validates every intermediate artifact, and can discover selected native tools when explicitly enabled. A command-line application built on the library is included for direct use, but the library API is the primary interface.

The API is at an initial stage. Conversion support depends on the enabled Cargo features and, for native adapters, the installed external tools.

Library Quick Start

Add the library to an application:

[dependencies]
convert-rs = "2.0"

Bytes in, bytes out:

use convert_rs::{ConversionOptions, Converter};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let converter = Converter::new();
    let png = std::fs::read("input.png")?;
    let jpeg = converter
        .convert_bytes(&png, "png", "jpeg", &ConversionOptions::default())
        .await?;
    std::fs::write("output.jpg", jpeg)?;
    Ok(())
}

For callers without a Tokio runtime, convert_blocking runs the same byte conversion on a private single-threaded runtime. Do not call it from inside an async context.

Path-based conversion returns a report that owns its output files:

use std::path::PathBuf;

use convert_rs::{ConversionOptions, Converter};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let converter = Converter::new();
    let report = converter
        .convert(
            &[PathBuf::from("input.png")],
            "jpeg",
            &ConversionOptions::default(),
        )
        .await?;
    // write_all_to copies every artifact into a destination directory and
    // returns the written paths. Artifacts also stay readable through
    // report.artifacts while the report is alive; its temporary workspace is
    // deleted on drop.
    let written = report.write_all_to(&PathBuf::from("out"))?;
    println!("wrote {} files", written.len());
    Ok(())
}

Library API

The main types are:

  • Converter: registers adapters, identifies inputs, plans routes, and executes conversions. Converter::new() uses built-in adapters only; Converter::discover().await adds native adapters for the tools installed on the host.
  • ConversionOptions: per-request controls; see the section below.
  • ConversionReport: the executed Plan, final Artifact values, non-fatal Warning values, per-step StepCost notes, and a RouteAttempt log of every route tried, including the fallbacks. Its temporary workspace is deleted on drop, so read artifacts or call write_all_to before dropping.
  • Policy: controls bridges, reinterpretation, packaging, external tools, decode-only formats, route count, step count, and graph search work. Native tools stay behind Policy::allow_external.
  • Limits: bounds input and output bytes, artifacts, decoded pixels, audio samples, archive entries, expansion ratio, memory, and step timeouts.
  • Adapter: the extension point for application-defined formats and conversion operations.

Call Converter::plan before executing a conversion when the application must inspect route costs or reject lossy and bridge steps. Call Converter::identify when an application needs the detected format and identification method without executing a conversion.

Conversion reports own their files. All successful adapter output is checked for regular-file status, workspace containment, declared format, and output limits. A failed route is discarded before the next route is attempted, and the reason is recorded in report.attempts.

ConversionOptions

  • from: override format identification with a format ID, extension, or MIME type. This is a caller assertion; content sniffing is skipped.
  • backend: require every executed step to use one named adapter.
  • forced_plan: skip search and attempt a caller-supplied plan. Forced plans are not a safe boundary for untrusted callers.
  • params: opaque key-value knobs forwarded to every executed step. The reserved key quality carries a 0-100 integer and maps onto each adapter's closest encoder control (JPEG/WebP quality, ffmpeg -q, ImageMagick -quality). Unknown keys are ignored. Params are never interpreted as paths or shell input.
  • verify: re-sniff every step's artifacts by content signature before accepting them; a contradiction fails the route and triggers fallback.

CLI

Install or run from this repository:

cargo install --path apps/convert-rs-cli
convert-rs convert input.png --to jpeg -o output.jpg

Convert a batch into a directory; files collide-renamed as stem-1.ext unless --force is passed:

convert-rs convert *.png --to webp --output-dir webp/

Stream to stdout, read from stdin, decode quality, JSON output:

convert-rs convert input.wav --to mp3 --quality 8 --stdout > track.mp3
cat data.json | convert-rs convert --from json --to toml -
convert-rs plan png pdf --json

Inspect what a build can do:

convert-rs formats            # table of registered formats
convert-rs backends           # adapter status, including native discovery
convert-rs doctor             # engine health, route examples, lossy edge count
convert-rs inspect input.bin  # detected format and identification method
convert-rs verify a.png b.png # re-check that files match their claimed formats

Native tools are off unless --allow-external is passed. See the caveats below.

Features

  • Graph-based route planning with cost ordering and route retry.
  • Pure Rust text, data, media, image, audio, and archive adapters.
  • Byte, path, blocking, and asynchronous entry points.
  • Content identification and optional artifact re-verification.
  • Quality and adapter params plumbing with strict validation.
  • Input, output, archive, image, audio, memory, and timeout limits.
  • Native process sandboxing (resource limits, scrubbed environment, process group kill) behind --allow-external.

Cargo Features

Feature Adapters and dependencies
data JSON, TOML, and CSV through serde_json, toml, and csv
media Images through image, WAV through hound, FFT through rustfft, audio decoding through symphonia
archives ZIP, TAR, and gzip through zip, tar, and flate2
external Native process adapters; pulls in libc on Unix

All features are enabled by default in the library and CLI packages.

Native Tools

Adapter Binary Conversions
FFmpeg ffmpeg Audio and video containers, video frames, still-image clips
ImageMagick convert or magick OpenEXR and PSD image conversions to and from raster formats
Poppler pdftotext, pdftoppm PDF text extraction and first-page PNG rasterization
LibreOffice soffice Office and OpenDocument files to PDF or text
Inkscape inkscape SVG to PNG

The adapter is registered only when its binary can be started. A delegate library required by a native tool may still be missing, in which case that route fails and the planner tries another route.

Native execution is constrained but not contained. On Unix, each child runs in its own process group with scrubbed environment variables and resource limits for CPU time, output size, and address space; on timeout the whole group is killed. There is no filesystem or network isolation, and no sandbox at all on Windows. Native tools receive the current process permissions. Use --allow-external only for trusted inputs.

Supported Formats

Built-in support includes JSON, TOML, CSV, base64 and hex envelopes, PNG, JPEG, GIF, BMP, TIFF, WebP, PNM, TGA, ICO, DDS, QOI, HDR, Farbfeld, AVIF, JPEG XL, WAV, FLAC, OGG, MP3, AAC, AIFF, WOFF, WOFF2, TrueType/OpenType, glTF binary, ZIP, TAR, tar.gz, and related archive and bridge routes. Native discovery adds video containers, PDF operations, Office and OpenDocument formats, SVG, PSD, and OpenEXR according to the installed tools. Run convert-rs formats or inspect Converter::formats() for the exact formats registered by a build.

Format IDs are stable within the library API. Extensions are aliases used for identification and may differ from the format ID, such as jpg for jpeg. Compound extensions such as archive.tar.gz are preserved when naming output files.

Last Resort

--last-resort (library: Policy bridges and reinterpretation) enables reinterpretation and archive packaging routes. These routes can preserve bytes or express them through another representation, but they do not establish semantic equivalence between unrelated formats. Binary input that is not valid UTF-8 still fails the text decoder; callers should use an explicit binary representation or a suitable media bridge when available.

Development

Check formatting, lints, docs, packaged crates, and the full test matrix locally:

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --all-features --no-deps --locked
cargo test --workspace --all-features --locked
cargo package --workspace --all-features --locked
python3 -m unittest discover -s ci -p 'test_*.py'

cargo package verifies each crate against published registry versions, so run it on a release-prepared checkout (or add --allow-dirty only when deliberately simulating the release bump). Set REQUIRE_NATIVE_TOOLS=1 to require native integration coverage instead of skipping those tests.

License

AGPL-3.0-or-later. See LICENSE. The graph design is an independent implementation and does not copy source code from other conversion projects.