Struct MarkdownIngester 

pub struct MarkdownIngester { /* private fields */ }

Markdown file ingester that parses and chunks markdown documents.

The ingester handles:

• YAML front-matter extraction
• Content chunking by headings
• Code block preservation with language tags
• Recursive directory traversal

Implementations

impl MarkdownIngester

pub fn new() -> Self

Create a new markdown ingester with default configuration.

pub fn with_config(config: IngesterConfig) -> Self

Create a new markdown ingester with custom configuration.

pub async fn ingest_file(&self, path: &Path) -> Result<MarkdownDocument>

Ingest a single markdown file.

This method reads the file, extracts front-matter, parses the markdown, and returns a structured document with chunks.

Arguments

• path - Path to the markdown file

Returns

A MarkdownDocument containing the parsed content and metadata

Errors

Returns an error if the file cannot be read or is not valid UTF-8

pub async fn ingest_directory( &self, dir: &Path, recursive: bool, ) -> Result<Vec<MarkdownDocument>>

Ingest all markdown files in a directory.

Arguments

• dir - Path to the directory
• recursive - Whether to recursively traverse subdirectories

Returns

A vector of MarkdownDocument for each markdown file found

Errors

Returns an error if the directory cannot be read

pub fn extract_frontmatter( content: &str, ) -> (Option<HashMap<String, String>>, &str)

Extract YAML front-matter from markdown content.

Front-matter is expected to be at the very beginning of the file, delimited by --- lines. For example:

---
title: My Document
author: John Doe
date: 2024-01-15
---

# Document content starts here

Arguments

• content - The full markdown file content

Returns

A tuple of (optional metadata HashMap, remaining content after front-matter)

Trait Implementations

impl Clone for MarkdownIngester

fn clone(&self) -> MarkdownIngester

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MarkdownIngester

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MarkdownIngester

fn default() -> Self

Returns the “default value” for a type.