pub struct KnowledgeBase { /* private fields */ }Expand description
Dataset id -> retriever registry.
Clones share their datasets: a dataset added to or removed from one clone
(for example by the management API) is seen by every other clone (for
example the one the workflows’ knowledge-retrieval nodes hold).
Implementations§
Source§impl KnowledgeBase
impl KnowledgeBase
pub fn new() -> Self
Sourcepub fn with_store_dataset(
self,
id: impl Into<String>,
store: Arc<dyn VectorStore>,
embedder: Arc<dyn Embedder>,
reranker: Option<Arc<dyn Reranker>>,
) -> Self
pub fn with_store_dataset( self, id: impl Into<String>, store: Arc<dyn VectorStore>, embedder: Arc<dyn Embedder>, reranker: Option<Arc<dyn Reranker>>, ) -> Self
Add a dataset backed by the built-in store pipeline.
Sourcepub fn add_store_dataset(
&mut self,
info: DatasetInfo,
store: Arc<dyn VectorStore>,
embedder: Arc<dyn Embedder>,
reranker: Option<Arc<dyn Reranker>>,
) -> &mut Self
pub fn add_store_dataset( &mut self, info: DatasetInfo, store: Arc<dyn VectorStore>, embedder: Arc<dyn Embedder>, reranker: Option<Arc<dyn Reranker>>, ) -> &mut Self
Add (or replace) a store-backed dataset with descriptive info.
Sourcepub fn with_retriever(
self,
id: impl Into<String>,
retriever: Arc<dyn Retriever>,
) -> Self
pub fn with_retriever( self, id: impl Into<String>, retriever: Arc<dyn Retriever>, ) -> Self
Add a dataset served by an external retriever.
Sourcepub fn add_retriever(
&mut self,
info: DatasetInfo,
retriever: Arc<dyn Retriever>,
) -> &mut Self
pub fn add_retriever( &mut self, info: DatasetInfo, retriever: Arc<dyn Retriever>, ) -> &mut Self
Add (or replace) a dataset served by an external retriever.
Sourcepub fn with_reranker(self, reranker: Option<Arc<dyn Reranker>>) -> Self
pub fn with_reranker(self, reranker: Option<Arc<dyn Reranker>>) -> Self
Reranker applied to the merged results of multi-dataset retrieval.
Sourcepub fn datasets(&self) -> Vec<DatasetInfo>
pub fn datasets(&self) -> Vec<DatasetInfo>
Every dataset’s id, name and description, in registration order.
Sourcepub fn dataset_details(&self) -> Vec<DatasetDetails>
pub fn dataset_details(&self) -> Vec<DatasetDetails>
Every dataset with what a management UI shows about it.
Sourcepub fn resource_names(&self) -> ResourceNames
pub fn resource_names(&self) -> ResourceNames
The stores, embedders and rerankers KnowledgeBase::create_dataset can use.
Sourcepub fn create_dataset(
&self,
spec: DatasetSpec,
) -> Result<DatasetDetails, RagError>
pub fn create_dataset( &self, spec: DatasetSpec, ) -> Result<DatasetDetails, RagError>
Create a store-backed dataset at runtime from the named resources of the knowledge config. Every clone of this knowledge base sees it at once.
@throws RagError::Input for an invalid or taken id, an unknown store,
embedder or reranker name, or invalid chunking.
Sourcepub async fn remove_dataset(&self, dataset_id: &str) -> Result<usize, RagError>
pub async fn remove_dataset(&self, dataset_id: &str) -> Result<usize, RagError>
Delete a dataset created at runtime together with its documents. Returns the number of chunks removed.
@throws RagError::NotFound for an unknown dataset, RagError::Unsupported
for one declared in the knowledge config (edit the config instead).
Sourcepub fn runtime_datasets(&self) -> Vec<DatasetSpec>
pub fn runtime_datasets(&self) -> Vec<DatasetSpec>
Specs of the datasets created at runtime, for persisting them.
pub fn contains(&self, dataset_id: &str) -> bool
Sourcepub fn ingestor(&self, dataset_id: &str) -> Option<Ingestor>
pub fn ingestor(&self, dataset_id: &str) -> Option<Ingestor>
An ingestor for a store-backed dataset (None for external retrievers or unknown ids),
using the dataset’s chunking settings.
Sourcepub fn set_chunking(
&mut self,
dataset_id: &str,
chunking: ChunkerConfig,
) -> Result<&mut Self, RagError>
pub fn set_chunking( &mut self, dataset_id: &str, chunking: ChunkerConfig, ) -> Result<&mut Self, RagError>
Set how documents ingested into dataset_id are chunked (e.g. parent-child).
@throws RagError::NotFound for an unknown dataset, RagError::Config when
parent_max_chars is not larger than max_chars.
Sourcepub fn chunking(&self, dataset_id: &str) -> Option<ChunkerConfig>
pub fn chunking(&self, dataset_id: &str) -> Option<ChunkerConfig>
The chunking settings of a dataset.
Sourcepub fn is_managed(&self, dataset_id: &str) -> bool
pub fn is_managed(&self, dataset_id: &str) -> bool
Whether documents can be added to, listed in and removed from this dataset
(store-backed datasets; false for external retrievers and unknown ids).
Sourcepub async fn list_documents(
&self,
dataset_id: &str,
limit: usize,
after: Option<&str>,
) -> Result<Vec<DocumentSummary>, RagError>
pub async fn list_documents( &self, dataset_id: &str, limit: usize, after: Option<&str>, ) -> Result<Vec<DocumentSummary>, RagError>
Up to limit documents of a dataset (capped at MAX_DOCUMENT_PAGE),
ordered by id and starting after the after id.
Sourcepub async fn document_chunks(
&self,
dataset_id: &str,
document_id: &str,
) -> Result<Vec<StoredChunk>, RagError>
pub async fn document_chunks( &self, dataset_id: &str, document_id: &str, ) -> Result<Vec<StoredChunk>, RagError>
Chunks of one document in chunk order (vectors included).
Sourcepub async fn set_document_enabled(
&self,
dataset_id: &str,
document_id: &str,
enabled: bool,
) -> Result<usize, RagError>
pub async fn set_document_enabled( &self, dataset_id: &str, document_id: &str, enabled: bool, ) -> Result<usize, RagError>
Enable or disable a document: a disabled document keeps its chunks (and stays disabled across re-ingestion) but no search returns it. Returns the number of chunks of the document.
@throws RagError::NotFound when the document does not exist.
Sourcepub async fn delete_document(
&self,
dataset_id: &str,
document_id: &str,
) -> Result<usize, RagError>
pub async fn delete_document( &self, dataset_id: &str, document_id: &str, ) -> Result<usize, RagError>
Remove a document. Returns the number of chunks removed (0 when it did not exist).
Sourcepub async fn retrieve(
&self,
request: &RetrievalRequest,
) -> Result<Vec<Hit>, RagError>
pub async fn retrieve( &self, request: &RetrievalRequest, ) -> Result<Vec<Hit>, RagError>
Retrieve across all requested datasets and merge by score.
When request.rerank is set and the knowledge base has a cross-dataset
reranker, the merged list is reranked as a whole (making scores from
different datasets comparable). Otherwise each dataset’s own reranker, if
any, is applied before merging.
Sourcepub async fn retrieve_single(
&self,
request: &RetrievalRequest,
) -> Result<Vec<Hit>, RagError>
pub async fn retrieve_single( &self, request: &RetrievalRequest, ) -> Result<Vec<Hit>, RagError>
Retrieve from the single best-matching dataset: every dataset is queried, and only the hits of the dataset whose top hit scores highest are kept.