Crate blz_core 

blz-core

Core functionality for blz - a fast, local search cache for llms.txt documentation.

This crate provides the foundational components for parsing, storing, and searching llms.txt documentation files locally. It’s designed for speed (sub-10ms search latency), offline-first usage, and exact line citations.

Architecture

The crate is organized around several key components:

• Configuration: Global and per-source settings management
• Parsing: Tree-sitter based markdown parsing with structured output
• Types: Core data structures representing sources, search results, and metadata
• Error Handling: Comprehensive error types with categorization and recovery hints

Quick Start

use blz_core::{Config, MarkdownParser, Result};

// Load global configuration
let config = Config::load()?;

// Parse markdown content
let mut parser = MarkdownParser::new()?;
let result = parser.parse("# Hello World\n\nThis is content.")?;

println!("Found {} heading blocks", result.heading_blocks.len());
println!("Generated TOC with {} entries", result.toc.len());

Performance Characteristics

• Parse time: < 150ms per MB of markdown content
• Memory usage: < 2x source document size during parsing
• Thread safety: All types are Send + Sync where appropriate

Error Handling

All operations return Result<T, Error> with structured error information:

use blz_core::{Error, MarkdownParser};

let mut parser = MarkdownParser::new()?;
match parser.parse("malformed content") {
    Ok(result) => println!("Parsed successfully"),
    Err(Error::Parse(msg)) => eprintln!("Parse error: {}", msg),
    Err(e) if e.is_recoverable() => eprintln!("Recoverable error: {}", e),
    Err(e) => eprintln!("Fatal error: {}", e),
}

Re-exports

pub use config::Config;

pub use config::DefaultsConfig;

pub use config::FetchConfig;

pub use config::FollowLinks;

pub use config::IndexConfig;

pub use config::PathsConfig;

pub use config::ToolConfig;

pub use config::ToolMeta;

pub use error::Error;

pub use error::Result;

pub use fetcher::FetchResult;

pub use fetcher::Fetcher;

pub use index::SearchIndex;

pub use language_filter::FilterStats;

pub use language_filter::LanguageFilter;

pub use mapping::build_anchors_map;

pub use mapping::compute_anchor_mappings;

pub use parser::MarkdownParser;

pub use parser::ParseResult;

pub use profiling::PerformanceMetrics;

pub use profiling::ResourceMonitor;

pub use registry::Registry;

pub use storage::Storage;

pub use types::*;

Modules

config

Configuration management for global and per-source settings Configuration management for blz cache system.

error

Error types and result aliases Error types and handling for blz-core operations.

fetcher

HTTP fetching with conditional requests support

index

Search index implementation using Tantivy

language_filter

Language filtering for multilingual llms.txt files Language filtering for llms.txt documentation entries.

mapping

Anchor remapping utilities between versions

parser

Tree-sitter based markdown parser Markdown parsing using tree-sitter for structured content analysis.

profile

Application profile detection helpers

profiling

Performance profiling utilities

registry

Built-in registry of known documentation sources

storage

Local filesystem storage for cached documentation

types

Core data types and structures Core data structures for blz cache system.