Crate xberg 

Xberg - High-Performance Document Intelligence Library

Xberg is a Rust-first document extraction library with language-agnostic plugin support. It provides fast, accurate extraction from PDFs, images, Office documents, emails, and more.

Quick Start

use xberg::{extract, ExtractInput, ExtractionConfig};

let config = ExtractionConfig::default();
let output = extract(ExtractInput::from_uri("document.pdf"), &config).await?;
println!("Extracted: {}", output.results[0].content);

Architecture

• Core Module (core): Main extraction orchestration, MIME detection, config loading
• Plugin System: Language-agnostic plugin architecture
• Extractors: Format-specific extraction (PDF, images, Office docs, email, etc.)
• OCR: Multiple OCR backend support (Tesseract, PaddleOCR, VLM)

Features

• Fast parallel processing with async/await
• Priority-based extractor selection
• Comprehensive MIME type detection (141 file extensions)
• Configurable caching and quality processing
• Cross-language plugin support (Python, Node.js planned)

Re-exports

pub use error::Result;

pub use error::XbergError;

pub use core::extract::extract;

pub use core::extract::extract_batch;

pub use core::config::AccelerationConfig;

pub use core::config::BedrockConfig;

pub use core::config::CallMode;

pub use core::config::CaptionAltTextMode;

pub use core::config::CaptioningConfig;

pub use core::config::ChunkClassificationConfig;

pub use core::config::ChunkClassificationDefinition;

pub use core::config::ChunkSizing;

pub use core::config::ChunkerType;

pub use core::config::ChunkingConfig;

pub use core::config::ConcurrencyConfig;

pub use core::config::ContentFilterConfig;

pub use core::config::CredentialProviderConfig;

pub use core::config::CsvConfig;

pub use core::config::EmailConfig;

pub use core::config::EmbeddingConfig;

pub use core::config::EmbeddingModelType;

pub use core::config::ExecutionProviderType;

pub use core::config::ExternalRedactionFinding;

pub use core::config::ExtractInput;

pub use core::config::ExtractInputKind;

pub use core::config::ExtractionConfig;

pub use core::config::ExtractionErrorItem;

pub use core::config::ExtractionResult;

pub use core::config::ExtractionSummary;

pub use core::config::FileExtractionConfig;

pub use core::config::GeoJsonExtractionConfig;

pub use core::config::ImageExtractionConfig;

pub use core::config::JupyterCellRendering;

pub use core::config::LanguageDetectionConfig;

pub use core::config::LlmBudgetConfig;

pub use core::config::LlmCacheConfig;

pub use core::config::LlmConfig;

pub use core::config::LlmProviderConfig;

pub use core::config::LlmRateLimitConfig;

pub use core::config::MergeMode;

pub use core::config::MimeDetectionPolicy;

pub use core::config::NerBackendKind;

pub use core::config::NerConfig;

pub use core::config::OcrConfig;

pub use core::config::OutputFormat;

pub use core::config::PageClassificationConfig;

pub use core::config::PageConfig;

pub use core::config::PostProcessorConfig;

pub use core::config::RedactionConfig;

pub use core::config::RedactionOffsetEncoding;

pub use core::config::RedactionPattern;

pub use core::config::RedactionTerm;

pub use core::config::RerankerConfig;

pub use core::config::RerankerHead;

pub use core::config::RerankerModelType;

pub use core::config::StructuredExtractionConfig;

pub use core::config::SummarizationConfig;

pub use core::config::TableChunkingMode;

pub use core::config::TokenReductionOptions;

pub use core::config::TranslationConfig;

pub use core::config::UrlExtractionConfig;

pub use core::config::UrlExtractionMode;

pub use core::config::LateInteractionConfig;

pub use core::config::LateInteractionModelType;

pub use core::config::SparseEmbeddingConfig;

pub use core::config::SparseEmbeddingModelType;

pub use extractors::security::SecurityLimits;

pub use paddle_ocr::ModelPaths;

pub use paddle_ocr::PaddleInferenceBackend;

pub use paddle_ocr::PaddleLanguage;

pub use paddle_ocr::PaddleOcrConfig;

pub use cache::CacheStats;

pub use core::config::OcrPipelineConfig;

pub use core::config::OcrPipelineStage;

pub use core::config::OcrQualityThresholds;

pub use core::config::OcrStrategy;

pub use core::config::VlmFallbackPolicy;

pub use core::mime::SUPPORTED_EXTENSION_COUNT;

pub use core::mime::SUPPORTED_FORMAT_COUNT;

pub use core::mime::SupportedFormat;

pub use core::mime::detect_mime_type_from_bytes;

pub use core::mime::get_extensions_for_mime;

pub use core::mime::list_supported_formats;

pub use doctor::DoctorCheck;

pub use doctor::DoctorReport;

pub use doctor::ProbeStatus;

pub use doctor::doctor;

pub use plugins::clear_document_extractors;

pub use plugins::clear_embedding_backends;

pub use plugins::clear_ocr_backends;

pub use plugins::clear_post_processors;

pub use plugins::clear_renderers;

pub use plugins::clear_reranker_backends;

pub use plugins::clear_tokenizer_backends;

pub use plugins::clear_validators;

pub use plugins::list_document_extractors;

pub use plugins::list_embedding_backends;

pub use plugins::list_ocr_backend_capabilities;

pub use plugins::list_ocr_backend_capabilities_for;

pub use plugins::list_ocr_backends;

pub use plugins::list_post_processors;

pub use plugins::list_renderers;

pub use plugins::list_reranker_backends;

pub use plugins::list_tokenizer_backends;

pub use plugins::list_validators;

pub use plugins::ocr_backend_supports_language;

pub use plugins::ocr_backend_supports_language_for;

pub use plugins::register_document_extractor;

pub use plugins::register_embedding_backend;

pub use plugins::register_ocr_backend;

pub use plugins::register_post_processor;

pub use plugins::register_renderer;

pub use plugins::register_reranker_backend;

pub use plugins::register_tokenizer_backend;

pub use plugins::register_validator;

pub use plugins::unregister_document_extractor;

pub use plugins::unregister_embedding_backend;

pub use plugins::unregister_ocr_backend;

pub use plugins::unregister_post_processor;

pub use plugins::unregister_renderer;

pub use plugins::unregister_reranker_backend;

pub use plugins::unregister_tokenizer_backend;

pub use plugins::unregister_validator;

pub use plugins::ConfidenceSemantics;

pub use plugins::DocumentExtractor;

pub use plugins::EmbeddingBackend;

pub use plugins::InternalDocumentExtractor;

pub use plugins::OcrBackend;

pub use plugins::OcrBackendCapabilities;

pub use plugins::OcrBackendType;

pub use plugins::PageOrientationHandling;

pub use plugins::Plugin;

pub use plugins::PostProcessor;

pub use plugins::ProcessingStage;

pub use plugins::Renderer;

pub use plugins::RerankerBackend;

pub use plugins::TokenizerBackend;

pub use plugins::Validator;

pub use enrich::enrich;

pub use enrich::EnrichedResult;

pub use enrich::EnrichmentConfig;

pub use types::*;

Modules

cache

Generic cache implementation with lock poisoning recovery.

cancellation

Cancellation token for extraction operations.

core

Core extraction orchestration module.

doctor

Environment diagnostics: probe configured backends and report what will actually execute on this host.

engine

Rust-only extraction engine.

enrich

Unified post-extraction enrichment: classification, chunk classification, NER and captioning in a single composable call. Transcription is not an enrichment stage — it runs at extraction time via core::config::ExtractionConfig. Active post-extraction enrichment pipeline for an ExtractedDocument.

error

Error types for Xberg.

extraction

Format-specific document extraction implementations and office metadata types.

extractors

Built-in document extractors.

paddle_ocr

PaddleOCR backend using ONNX Runtime.

plugins

Plugin system for extending Xberg functionality.

rendering

Unified rendering of document content to output formats.

telemetry

Telemetry and observability for xberg.

text

Text post-processing: NER, summarisation, redaction, token reduction, and translation.

types

Core types for document extraction.

utils

Text utility functions for quality processing and string manipulation.

Structs

LateInteractionMatch

Stub match type for builds without the late-interaction-presets feature.

LateInteractionPreset

Stub preset type for builds without the late-interaction-presets feature.

MultiVectorEmbedding

Stub multi-vector result type for builds without the late-interaction-presets feature.

RerankedDocument

Stub result document type for builds without reranker-presets.

SparseEmbedding

Stub result type for builds without the sparse-embedding-presets feature.

SparseEmbeddingPreset

Stub preset type for builds without the sparse-embedding-presets feature.

Functions

clear_engine_caches

Drop every resident model engine: dense, static, sparse and late-interaction embeddings and rerankers. Returns the number of engines removed.

detect_mime_type

Detect the MIME type of a file at the given path.