convert-rs 0.2.0

Extensible file conversion library with graph routing and optional native tools
Documentation
//! File conversion through a policy-controlled graph of independent adapters.
//!
//! Use [`Converter::new`] for compiled-in pure-Rust adapters. With the `external`
//! feature, `Converter::discover().await` also probes installed native tools.
//! Discovery does not enable external routes; set [`Policy::allow_external`]
//! explicitly. Register additional implementations through [`Converter::register`].
//!
//! # Convert a file
//!
//! Conversion requires a Tokio runtime. This example uses only the byte/text API
//! and no native tools. A same-format request stages a byte-for-byte copy without
//! decoding, validating the claimed format, or choosing a codec.
//!
//! ```rust
//! use convert_rs::{ConversionOptions, Converter};
//!
//! # #[tokio::main(flavor = "current_thread")]
//! # async fn main() -> Result<(), Box<dyn std::error::Error>> {
//! let inputs = tempfile::tempdir()?;
//! let source = inputs.path().join("note.txt");
//! std::fs::write(&source, "Conversion example\n")?;
//! let converter = Converter::new();
//! let report = converter.convert(&[source], "txt", &ConversionOptions::default()).await?;
//! assert!(report.plan.steps.is_empty());
//! assert!(report.warnings.is_empty());
//! assert_eq!(std::fs::read_to_string(&report.artifacts[0].path)?, "Conversion example\n");
//! // Persist output outside the report-owned workspace before dropping it.
//! let saved = inputs.path().join("saved.txt");
//! std::fs::copy(&report.artifacts[0].path, &saved)?;
//! let temporary = report.artifacts[0].path.clone();
//! drop(report);
//! assert!(!temporary.exists());
//! assert!(saved.exists());
//! # Ok(())
//! # }
//! ```
//!
//! # Features and fidelity
//!
//! Default features are `data` (JSON, TOML, CSV), `media` (image and audio
//! operations), `archives` (archive and compression operations), and `external`
//! (native-tool discovery). Without default features, the built-in byte and text
//! operations remain. Registered formats and available routes depend on features
//! and adapters; listing a format does not promise an encoder for every input.
//!
//! Plans use adapter-declared costs and semantic classifications. The default
//! policy permits lossy conversions. There is no lossless-only policy flag:
//! inspect [`Plan::steps`] and, if appropriate, execute a reviewed forced plan.
//! [`ConversionReport::warnings`] describes non-lossless steps of the successful
//! route, not measured quality loss or every possible metadata change. An empty
//! warning list is not proof of complete fidelity. No public option selects
//! codecs, bitrates, or encoder quality settings.
//!
//! # Execution boundaries
//!
//! Inputs are staged separately for each attempted route. Final paths belong to
//! the returned [`ConversionReport`] and are removed when it is dropped. Cloning
//! an [`Artifact`] does not keep its file alive. Returned output paths, declared
//! formats, counts, and byte totals are checked, but file contents are not
//! independently decoded by the engine to verify their declared format.
//!
//! [`Limits`] provides staging checks and adapter-specific budgets, not a hard
//! memory, CPU, disk, or wall-clock sandbox. Custom adapters are trusted code.
//! Native tools require operating-system isolation when processing hostile data;
//! this library does not provide that isolation.
#![warn(missing_docs)]

#[cfg(feature = "archives")]
mod archives;
mod builtin;
mod engine;
#[cfg(feature = "external")]
mod external;
#[cfg(feature = "media")]
mod media;
#[cfg(feature = "external")]
mod process;

pub use convert_rs_core::{Conversion, ConversionKind, Format, Graph, MediaKind, Plan, Policy};
pub use engine::{
    ConversionOptions, ConversionReport, Converter, Identification, IdentificationMethod, Warning,
};

use std::future::Future;
use std::path::PathBuf;
use std::pin::Pin;
use std::time::Duration;

/// A path and declared format for a staged input or adapter output.
///
/// Artifact vectors preserve order. This value neither owns the file nor verifies
/// its contents. Cloning it clones only the path and format; report output files
/// remain owned by the [`ConversionReport`]'s temporary directory.
#[derive(Clone, Debug)]
pub struct Artifact {
    /// Path to the artifact in the report-owned temporary workspace.
    pub path: PathBuf,
    /// Canonical format ID for the artifact.
    pub format: String,
}

/// Resource budgets used during staging, output checks, and adapter execution.
///
/// The engine enforces input count, input bytes per staged file, and returned
/// output count and aggregate bytes per step. Other checks depend on the adapter.
/// The engine does not wrap arbitrary adapter futures in a timeout, preempt
/// blocking work, or enforce process-wide memory and disk quotas. Budgets do not
/// bound unreported files or the cumulative size of all intermediate steps.
/// Identity copies do not pass through adapter output validation.
///
/// Defaults are 120 seconds, 256 MiB input bytes, 512 MiB output bytes, 256
/// artifacts, 16 million pixels and audio samples, 256 archive entries, a 100:1
/// expansion ratio, and a 256 MiB adapter memory budget. Values are not validated
/// by [`Converter::limits`]; zero and small budgets can make operations fail.
#[derive(Clone, Debug)]
pub struct Limits {
    /// Per-invocation time budget used by adapters, not a universal hard deadline.
    ///
    /// Pure-Rust adapters check elapsed time at operation-specific points; native
    /// process execution uses a timeout. Staging and arbitrary custom adapters
    /// are not interrupted by the engine.
    pub timeout: Duration,
    /// Maximum bytes per file during engine staging.
    ///
    /// Some adapters additionally apply this as an aggregate input budget.
    pub max_input_bytes: u64,
    /// Maximum aggregate bytes of returned output files per step.
    ///
    /// Engine validation occurs after the adapter returns; this is not a disk quota.
    pub max_output_bytes: u64,
    /// Maximum input count and maximum returned output count per step.
    pub max_artifacts: usize,
    /// Decoded image pixel budget used by media operations, not by engine validation.
    pub max_pixels: u64,
    /// Decoded audio sample budget used by media operations.
    pub max_audio_samples: u64,
    /// Archive entry budget used by archive operations, including directory entries.
    pub max_archive_entries: usize,
    /// Decompression byte expansion ratio used by archive operations.
    pub max_expansion_ratio: u64,
    /// Memory budget for adapter-specific allocation checks, not process resident memory.
    pub max_memory_bytes: u64,
}

impl Default for Limits {
    fn default() -> Self {
        Self {
            timeout: Duration::from_secs(120),
            max_input_bytes: 256 * 1024 * 1024,
            max_output_bytes: 512 * 1024 * 1024,
            max_artifacts: 256,
            max_pixels: 16_000_000,
            max_audio_samples: 16_000_000,
            max_archive_entries: 256,
            max_expansion_ratio: 100,
            max_memory_bytes: 256 * 1024 * 1024,
        }
    }
}

/// One adapter invocation. Adapters must write only inside `output_dir`, must
/// not change inputs, and must return at least one regular file on success.
///
/// The engine creates the output directory before calling the adapter. The
/// adapter is responsible for validating operation-specific inputs, observing
/// limits, and keeping blocking work off the asynchronous executor. The engine
/// checks returned files but does not enforce every part of this contract; in
/// particular, it currently accepts an empty output vector.
#[derive(Clone, Debug)]
pub struct StepRequest {
    /// Ordered staged inputs or outputs from the preceding step; do not mutate them.
    pub inputs: Vec<Artifact>,
    /// Existing private step directory where the adapter must create outputs.
    pub output_dir: PathBuf,
    /// Concrete source format ID, not a filename extension or wildcard.
    pub from: String,
    /// Concrete target format ID that every returned artifact must declare.
    pub to: String,
    /// Adapter-specific operation ID copied from the selected conversion edge.
    pub operation: String,
    /// Resource budgets for this invocation; see [`Limits`] for enforcement scope.
    pub limits: Limits,
}

/// Result of a library operation, with [`Error`] describing failure.
pub type Result<T> = std::result::Result<T, Error>;
/// A sendable, boxed adapter future yielding ordered artifacts or an [`Error`].
///
/// The lifetime allows the future to borrow both the adapter and its request.
pub type ConversionFuture<'a> = Pin<Box<dyn Future<Output = Result<Vec<Artifact>>> + Send + 'a>>;

/// Output port for conversion backends. Implement this to add formats without
/// changing the planner or orchestrator. Backend names identify registrations;
/// registering the same name replaces its previous implementation.
///
/// Implementations are trusted code, not sandboxed plugins. Advertised metadata
/// should remain stable between calls. Each edge's `backend` must match the
/// adapter's name, and the implementation must reject unsupported operations.
/// Registration validates graph metadata, not this correspondence or fidelity.
///
/// # Register a custom adapter
///
/// This adapter copies one text file into a custom text representation. It uses
/// asynchronous file I/O and checks its input size and timeout explicitly.
///
/// ```rust
/// use std::sync::Arc;
/// use convert_rs::{Adapter, Artifact, Conversion, ConversionFuture, ConversionKind,
///     ConversionOptions, Converter, Error, Format, MediaKind, StepRequest};
///
/// struct NoteAdapter;
/// impl Adapter for NoteAdapter {
///     fn name(&self) -> &str { "note-example" }
///     fn formats(&self) -> Vec<Format> {
///         vec![Format::new("example-note", "Example note", MediaKind::Text)]
///     }
///     fn conversions(&self) -> Vec<Conversion> {
///         vec![Conversion::new("txt", "example-note", self.name(), "copy-note",
///             ConversionKind::Lossless, 1)]
///     }
///     fn convert<'a>(&'a self, request: &'a StepRequest) -> ConversionFuture<'a> {
///         Box::pin(async move {
///             if request.operation != "copy-note" || request.from != "txt"
///                 || request.to != "example-note" || request.inputs.len() != 1 {
///                 return Err(Error::Invalid("unsupported note request".into()));
///             }
///             let job = async {
///                 let size = tokio::fs::metadata(&request.inputs[0].path).await?.len();
///                 if size > request.limits.max_input_bytes
///                     || size > request.limits.max_output_bytes {
///                     return Err(Error::Limit("note bytes".into()));
///                 }
///                 let path = request.output_dir.join("note.example-note");
///                 tokio::fs::copy(&request.inputs[0].path, &path).await?;
///                 Ok(vec![Artifact { path, format: request.to.clone() }])
///             };
///             tokio::time::timeout(request.limits.timeout, job).await
///                 .map_err(|_| Error::Timeout { backend: self.name().into() })?
///         })
///     }
/// }
///
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() -> Result<(), Box<dyn std::error::Error>> {
/// let mut converter = Converter::new();
/// converter.register(Arc::new(NoteAdapter))?;
/// let inputs = tempfile::tempdir()?;
/// let source = inputs.path().join("note.txt");
/// std::fs::write(&source, "custom adapter")?;
/// let report = converter.convert(&[source], "example-note",
///     &ConversionOptions::default()).await?;
/// assert_eq!(report.plan.steps[0].backend, "note-example");
/// assert_eq!(std::fs::read_to_string(&report.artifacts[0].path)?, "custom adapter");
/// # Ok(())
/// # }
/// ```
pub trait Adapter: Send + Sync {
    /// Stable registration name, nonempty and at most 64 bytes.
    fn name(&self) -> &str;
    /// Format declarations contributed to the graph.
    ///
    /// Registration requires 1 through 8,192 declarations. Edges can also refer
    /// to formats supplied by other registered adapters.
    fn formats(&self) -> Vec<Format>;
    /// Supported directed edges, including their semantic kind and relative cost.
    ///
    /// Registration accepts at most 1,000,000 edges from the incoming adapter.
    fn conversions(&self) -> Vec<Conversion>;
    /// Execute one operation using the [`StepRequest`] contract.
    ///
    /// # Errors
    ///
    /// Return an appropriate [`Error`] for invalid requests, unsupported content,
    /// I/O failure, or exceeded limits. A failed route may be retried through
    /// another adapter; do not rely on side effects outside the output directory.
    fn convert<'a>(&'a self, request: &'a StepRequest) -> ConversionFuture<'a>;

    /// Return availability information for this adapter.
    ///
    /// The default reports availability unconditionally with an empty detail.
    /// An implementation may return cached discovery results rather than probe
    /// again. Availability is informational and does not guarantee execution.
    fn status(&self) -> BackendStatus {
        BackendStatus {
            name: self.name().to_owned(),
            available: true,
            detail: String::new(),
        }
    }
}

/// An adapter's reported availability and diagnostic detail.
#[derive(Clone, Debug)]
pub struct BackendStatus {
    /// Registered adapter name.
    pub name: String,
    /// Whether the adapter reports itself available; not a per-file capability check.
    pub available: bool,
    /// Human-readable probe details.
    pub detail: String,
}

/// Failures from registration, identification, planning, and execution.
///
/// Errors raised while attempting routes are rendered into [`Self::RoutesFailed`]
/// by [`Converter::convert`], rather than returned as their original variants.
#[derive(Debug, thiserror::Error)]
pub enum Error {
    /// A filesystem or other I/O operation failed.
    #[error(transparent)]
    Io(
        /// Underlying I/O failure.
        #[from]
        std::io::Error,
    ),
    /// Graph registration or route search failed.
    #[error(transparent)]
    Plan(
        /// Underlying graph or search failure.
        #[from]
        convert_rs_core::PlanError,
    ),
    /// Invalid caller input or unsupported request parameters.
    #[error("invalid request: {0}")]
    Invalid(
        /// Explanation of the rejected request.
        String,
    ),
    /// An adapter operation or artifact format check failed.
    #[error("conversion failed: {0}")]
    Conversion(
        /// Adapter or output-validation failure details.
        String,
    ),
    /// An adapter reported exceeding its time budget.
    #[error("{backend} exceeded its timeout")]
    Timeout {
        /// Name of the adapter whose time budget was exceeded.
        backend: String,
    },
    /// A checked resource budget was exceeded.
    #[error("resource limit exceeded: {0}")]
    Limit(
        /// Description of the exceeded budget.
        String,
    ),
    /// A policy or input/output safety check rejected the operation.
    #[error("security policy rejected input: {0}")]
    Security(
        /// Description of the rejected condition.
        String,
    ),
    /// Every attempted route failed, with route descriptions and rendered errors.
    ///
    /// Only routes returned within the planner's budgets are attempted. This
    /// does not mean every possible graph route was tried.
    #[error("all attempted routes failed: {0}")]
    RoutesFailed(
        /// Semicolon-separated route descriptions and rendered failure messages.
        String,
    ),
}