Struct PdfLoaderConfig 

pub struct PdfLoaderConfig {
    pub min_chunk_size: usize,
    pub max_chunk_size: usize,
    pub chunk_overlap: usize,
    pub default_weight: f32,
    pub include_metadata: bool,
    pub split_by_sentence: bool,
    pub split_by_paragraph: bool,
    pub clean_text: bool,
    pub remove_page_numbers: bool,
    pub dehyphenate: bool,
}

Structure for configuring how PDFs are converted to training data

Fields

min_chunk_size: usize

Minimum length of a chunk (in characters)

max_chunk_size: usize

Maximum length of a chunk (in characters)

chunk_overlap: usize

Overlap between chunks (in characters)

default_weight: f32

Default weight for generated training examples

include_metadata: bool

Whether metadata like page number and position in the document should be added

split_by_sentence: bool

Whether chunks should be split at sentence boundaries

split_by_paragraph: bool

Whether to primarily split at paragraph boundaries (\n\n)

clean_text: bool

Whether to apply the text cleaning pipeline

remove_page_numbers: bool

Whether to remove page number lines

dehyphenate: bool

Whether to rejoin hyphenated words across line breaks

Trait Implementations

impl Default for PdfLoaderConfig

fn default() -> Self

Returns the “default value” for a type.