Struct ChunkingConfiguration 

#[non_exhaustive]
pub struct ChunkingConfiguration {
    pub chunking_strategy: ChunkingStrategy,
    pub fixed_size_chunking_configuration: Option<FixedSizeChunkingConfiguration>,
    pub hierarchical_chunking_configuration: Option<HierarchicalChunkingConfiguration>,
    pub semantic_chunking_configuration: Option<SemanticChunkingConfiguration>,
}

Details about how to chunk the documents in the data source. A chunk refers to an excerpt from a data source that is returned when the knowledge base that it belongs to is queried.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

chunking_strategy: ChunkingStrategy

Knowledge base can split your source data into chunks. A chunk refers to an excerpt from a data source that is returned when the knowledge base that it belongs to is queried. You have the following options for chunking your data. If you opt for NONE, then you may want to pre-process your files by splitting them up such that each file corresponds to a chunk.

fixed_size_chunking_configuration: Option<FixedSizeChunkingConfiguration>

Configurations for when you choose fixed-size chunking. If you set the chunkingStrategy as NONE, exclude this field.

hierarchical_chunking_configuration: Option<HierarchicalChunkingConfiguration>

Settings for hierarchical document chunking for a data source. Hierarchical chunking splits documents into layers of chunks where the first layer contains large chunks, and the second layer contains smaller chunks derived from the first layer.

semantic_chunking_configuration: Option<SemanticChunkingConfiguration>

Settings for semantic document chunking for a data source. Semantic chunking splits a document into smaller documents based on groups of similar content derived from the text with natural language processing.

Implementations

impl ChunkingConfiguration

pub fn chunking_strategy(&self) -> &ChunkingStrategy

Knowledge base can split your source data into chunks. A chunk refers to an excerpt from a data source that is returned when the knowledge base that it belongs to is queried. You have the following options for chunking your data. If you opt for NONE, then you may want to pre-process your files by splitting them up such that each file corresponds to a chunk.

pub fn fixed_size_chunking_configuration( &self, ) -> Option<&FixedSizeChunkingConfiguration>

Configurations for when you choose fixed-size chunking. If you set the chunkingStrategy as NONE, exclude this field.

pub fn hierarchical_chunking_configuration( &self, ) -> Option<&HierarchicalChunkingConfiguration>

Settings for hierarchical document chunking for a data source. Hierarchical chunking splits documents into layers of chunks where the first layer contains large chunks, and the second layer contains smaller chunks derived from the first layer.

pub fn semantic_chunking_configuration( &self, ) -> Option<&SemanticChunkingConfiguration>

Settings for semantic document chunking for a data source. Semantic chunking splits a document into smaller documents based on groups of similar content derived from the text with natural language processing.

impl ChunkingConfiguration

pub fn builder() -> ChunkingConfigurationBuilder

Creates a new builder-style object to manufacture ChunkingConfiguration.

Trait Implementations

impl Clone for ChunkingConfiguration

fn clone(&self) -> ChunkingConfiguration

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChunkingConfiguration

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ChunkingConfiguration

fn eq(&self, other: &ChunkingConfiguration) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ChunkingConfiguration