Crate llmtxt_core 

llmtxt-core: Portable primitives for the llmtxt content platform.

This crate is the single source of truth for compression, hashing, signing, and encoding functions used by both the Rust (SignalDock) and TypeScript (npm llmtxt via WASM) consumers.

Features

• wasm (default): Enables wasm-bindgen exports for JavaScript consumption. Disable with default-features = false for native-only usage.

Native

All functions are available as regular Rust APIs regardless of features.

Re-exports

pub use crypto::constant_time_eq_hex;

pub use crypto::sign_webhook_payload;

pub use normalize::l2_normalize;

pub use normalize::l2_normalize_wasm;

pub use rbac::DocumentRole;

pub use rbac::OrgRole;

pub use rbac::Permission;

pub use rbac::role_has_permission;

pub use rbac::role_permissions;

pub use slugify::slugify;

pub use validation::DEFAULT_MAX_CONTENT_BYTES;

pub use validation::DEFAULT_MAX_LINE_BYTES;

pub use validation::contains_binary_content;

pub use validation::default_max_content_bytes;

pub use validation::default_max_line_bytes;

pub use validation::detect_format;

pub use validation::find_overlong_line;

pub use graph::GraphEdge;

pub use graph::GraphNode;

pub use graph::GraphStats;

pub use graph::KnowledgeGraph;

pub use graph::MessageInput;

pub use graph::MessageMetadata;

pub use graph::TopAgent;

pub use graph::TopTopic;

pub use graph::build_graph_native;

pub use graph::build_graph_wasm;

pub use graph::extract_directives;

pub use graph::extract_directives_wasm;

pub use graph::extract_mentions;

pub use graph::extract_mentions_wasm;

pub use graph::extract_tags;

pub use graph::extract_tags_wasm;

pub use graph::top_agents_native;

pub use graph::top_agents_wasm;

pub use graph::top_topics_native;

pub use graph::top_topics_wasm;

pub use similarity::SimilarityResult;

pub use similarity::content_similarity;

pub use similarity::content_similarity_wasm;

pub use similarity::extract_ngrams;

pub use similarity::extract_ngrams_wasm;

pub use similarity::extract_word_shingles;

pub use similarity::extract_word_shingles_wasm;

pub use similarity::fingerprint_similarity;

pub use similarity::jaccard_similarity;

pub use similarity::jaccard_similarity_wasm;

pub use similarity::min_hash_fingerprint;

pub use similarity::min_hash_fingerprint_wasm;

pub use similarity::rank_by_similarity;

pub use similarity::rank_by_similarity_wasm;

pub use similarity::simple_hash;

pub use similarity::text_similarity_jaccard;

pub use disclosure::code::parse_code_sections;

pub use disclosure::json::extract_json_keys;

pub use disclosure::json::parse_json_sections;

pub use disclosure::markdown::extract_markdown_toc;

pub use disclosure::markdown::parse_markdown_sections;

pub use disclosure::search::search_content;

pub use disclosure::text::parse_text_sections;

pub use disclosure::DocumentOverview;

pub use disclosure::JsonKey;

pub use disclosure::LineRangeResult;

pub use disclosure::SearchResult;

pub use disclosure::Section;

pub use disclosure::TocEntry;

pub use disclosure::detect_document_format;

pub use disclosure::generate_overview;

pub use disclosure::get_line_range;

pub use disclosure::get_section;

pub use disclosure::query_json_path;

pub use tfidf::fnv1a_hash;

pub use tfidf::tfidf_embed;

pub use identity::body_hash;

pub use identity::canonical_payload;

pub use identity::keygen;

pub use identity::sign_submission;

pub use identity::verify_submission;

pub use bft::ChainedEvent;

pub use bft::bft_check;

pub use bft::bft_max_faults;

pub use bft::bft_quorum;

pub use bft::hash_chain_extend;

pub use bft::verify_chain;

pub use a2a::A2AMessage;

pub use canonical::FrontmatterMeta;

pub use canonical::canonical_frontmatter;

pub use canonical::canonical_frontmatter_wasm;

pub use blob::BlobNameError;

pub use blob::blob_name_validate;

pub use blob::hash_blob;

pub use merkle::AuditEntry;

pub use merkle::hash_audit_entry;

pub use merkle::merkle_root;

pub use merkle::sign_merkle_root;

pub use merkle::verify_audit_chain;

pub use merkle::verify_merkle_proof;

pub use merkle::verify_merkle_root_signature;

pub use billing::TierDecision;

pub use billing::TierKind;

pub use billing::TierLimits;

pub use billing::UsageSnapshot;

pub use billing::evaluate_tier_limits;

pub use billing::evaluate_tier_limits_wasm;

pub use billing::get_tier_limits_wasm;

pub use billing::tier_limits;

pub use export_archive::ARCHIVE_VERSION;

pub use export_archive::ExportApiKey;

pub use export_archive::ExportArchive;

pub use export_archive::ExportAuditEntry;

pub use export_archive::ExportDocument;

pub use export_archive::ExportVersion;

pub use export_archive::ExportWebhook;

pub use export_archive::RetentionPolicy;

pub use export_archive::deserialize_export_archive;

pub use export_archive::deserialize_retention_policy;

pub use export_archive::serialize_export_archive;

pub use export_archive::serialize_retention_policy;

pub use retention::EvictionSet;

pub use retention::LawfulBasis;

pub use retention::RetentionAction;

pub use retention::RetentionRow;

pub use retention::RetentionTier;

pub use retention::apply_retention;

pub use retention::canonical_policies;

Modules

a2a

Agent-to-Agent (A2A) message envelope primitives.

bft

Byzantine Fault Tolerant (BFT) consensus primitives for LLMtxt.

billing

Tier limit evaluation — Single Source of Truth for billing policy.

blob

Binary blob primitives for content-addressed attachment storage (T428).

canonical

Canonical frontmatter serializer for document export (T427).

classify

Multi-modal document classification.

crypto

Webhook and generic payload HMAC signing primitives.

disclosure

Progressive disclosure: structural analysis, section extraction, line-range access, content search, and JSONPath queries.

export_archive

Export archive primitives for GDPR data portability (T094).

graph

Knowledge graph extraction from message metadata.

identity

Agent identity primitives — Ed25519 key generation, signing, and canonical-payload construction for the LLMtxt Verifiable Agent Identity epic (T147).

merkle

Binary SHA-256 Merkle tree for the tamper-evident audit log (T164).

normalize

Vector normalization primitives.

rbac

Role-Based Access Control primitives for llmtxt documents.

retention

Retention policy DSL — T168.2 (richer than export_archive::RetentionPolicy). Retention policy DSL — T168.2 (T614).

similarity

Lightweight content similarity using n-gram fingerprinting.

slugify

URL-safe slug generation.

tfidf

TF-IDF vectorizer with FNV-1a hashing.

validation

Content validation primitives.

wasm_bindings

WASM binding wrappers for crates/llmtxt-core public API.

Structs

ApprovalPolicy

Policy governing how approvals are evaluated.

ApprovalResult

Result of evaluating reviews against a policy.

Conflict

A single conflicting region where both ours and theirs diverge from base.

DiffResult

Result of computing a line-based diff between two texts.

EmbeddedReview

A single agent review with a pre-computed embedding of its content.

EmbeddedSection

A document section with a pre-computed embedding.

MergeStats

Statistics describing the outcome of a 3-way merge.

MultiDiffLine

One line entry in a multi-way diff result.

MultiDiffResult

Full result of a multi-way diff.

MultiDiffStats

Aggregate statistics for a multi-way diff.

MultiDiffVariant

A single version variant at a divergent line position.

Review

A single review from an agent.

ReviewCluster

A cluster of reviewers whose embeddings are mutually similar.

SectionSimilarity

Per-section similarity record.

SemanticChange

A semantic change annotation for a section pair.

SemanticConsensusResult

Result of semantic consensus evaluation across a set of reviews.

SemanticDiffResult

Full result of a semantic diff between two document versions.

SignedUrlBuildRequest

Input for generating a signed URL in native Rust consumers.

SignedUrlParams

Parameters extracted from a verified signed URL.

StructuredDiffLine

A single line in a structured diff, with type and line numbers.

StructuredDiffResult

Full structured diff result with lines and summary counts.

ThreeWayMergeResult

Result of a 3-way merge operation.

Enums

DocumentState

Lifecycle state of a collaborative document.

SectionAlignment

How a section from version A maps to version B.

VerifyError

Errors that can occur during signed URL verification.

Functions

apply_patch

Apply a unified diff patch to an original string. Returns the updated string on success, or an error if the patch is invalid or fails to apply cleanly.

batch_diff_versions

Compare multiple versions against a base version in a single call.

calculate_compression_ratio

Calculate the compression ratio (original / compressed), rounded to 2 decimals. Returns 1.0 when compressed_size is 0.

calculate_tokens

Estimate token count using the ~4 chars/token heuristic.

cherry_pick_merge

Merge content from multiple versions by cherry-picking specific line ranges and/or markdown sections.

compress

Compress a UTF-8 string using zstd (RFC 8478), level 3.

compute_diff

Compute a line-based diff between two texts.

compute_org_signature

Compute the HMAC-SHA256 signature for org-scoped signed URL parameters. Includes org_id in the HMAC payload for organization-level access control. Returns the first 32 hex characters (128 bits) by default.

compute_org_signature_with_length

Compute org-scoped HMAC-SHA256 signature with configurable output length.

compute_sections_modified

Compute which markdown sections were modified between two document versions.

compute_sections_modified_native

Native version returning a Vec<String> of modified section names.

compute_signature

Compute the HMAC-SHA256 signature for signed URL parameters. Returns the first 16 hex characters of the digest (64 bits). For longer signatures, use compute_signature_with_length.

compute_signature_with_length

Compute the HMAC-SHA256 signature with configurable output length.

cosine_similarity

Cosine similarity between two embedding vectors.

cosine_similarity_wasm

Compute cosine similarity between two embedding vectors supplied as JSON arrays.

create_patch

Create a unified diff patch representing the difference between original and modified.

decode_base62

Decode a base62-encoded string back into an integer.

decompress

Decompress bytes back to a UTF-8 string.

decompress_bytes

Decompress bytes, auto-detecting codec by magic bytes.

derive_signing_key

Derive a per-agent signing key from their API key. Uses HMAC-SHA256(api_key, "llmtxt-signing").

diff_versions

Reconstruct two versions and compute a diff between them.

diff_versions_native

Native version of diff_versions that accepts a slice and returns a struct.

encode_base62

Encode a non-negative integer into a base62 string.

evaluate_approvals

Evaluate reviews against a policy. All inputs and output are JSON strings.

evaluate_approvals_native

Evaluate reviews against an approval policy (native API).

generate_id

Generate an 8-character base62 ID from a UUID v4.

generate_signed_url

Generate a signed URL with an optional resource path prefix. Native Rust API only.

hash_content

Compute the SHA-256 hash of a UTF-8 string, returned as lowercase hex.

is_editable

Check whether a document state allows content modifications.

is_editable_str

Parse a state string and check if it’s editable. Returns false for unrecognized state names.

is_expired

Check whether a timestamp (milliseconds) has expired. Returns false for 0 (no expiration).

is_terminal

Check whether a document state is terminal (no further transitions).

is_terminal_str

Parse a state string and check if it’s terminal. Returns false for unrecognized state names.

is_valid_transition

Check whether a state transition is allowed.

is_valid_transition_str

Parse a state string and check if the transition is valid. Accepts uppercase state names (“DRAFT”, “REVIEW”, etc.). Returns false for unrecognized state names.

mark_stale_reviews

Mark reviews as stale for the given version. JSON I/O for WASM.

mark_stale_reviews_native

Mark reviews as stale when a document version changes.

multi_way_diff

JSON-returning wrapper for multi_way_diff_native.

multi_way_diff_native

Compute a multi-way diff across a base version and up to 4 additional versions.

reconstruct_version

Apply a sequence of patches to base content, returning the content at the target version. This avoids N WASM boundary crossings by performing all patch applications in a single Rust call.

reconstruct_version_native

Native-friendly version of reconstruct_version that accepts a slice directly instead of JSON. Use this from Rust consumers; the JSON variant is for WASM callers.

semantic_consensus

Evaluate semantic consensus from a JSON array of reviews (WASM / backend entry point).

semantic_consensus_native

Evaluate semantic consensus across a set of reviews (native API).

semantic_diff

Compute semantic diff from JSON strings (WASM / backend entry point).

semantic_diff_native

Compute a semantic diff between two sets of pre-embedded sections (native API).

squash_patches

Apply all patches sequentially to base content, then produce a single unified diff from the original base to the final state.

squash_patches_native

Native-friendly version of squash_patches that accepts a slice directly.

structured_diff

Compute a structured line-level diff between two texts.

structured_diff_native

Native version of structured_diff returning a typed struct.

three_way_merge

JSON-serializing wrapper around three_way_merge_native.

three_way_merge_native

Perform a 3-way merge of base, ours, and theirs.

validate_transition

Validate a proposed transition and return a JSON result.

verify_signed_url

Verify a signed URL. Native Rust API only.

zlib_compress

Compress bytes using legacy zlib/deflate (RFC 1950).

zstd_compress

Compress arbitrary bytes using zstd at level 3.

zstd_decompress

Decompress zstd bytes back to raw bytes.