Struct WebBaseLoader 

pub struct WebBaseLoader { /* private fields */ }

Loads content from a URL via HTTP GET.

Uses reqwest to fetch the URL content and returns a single Document with the URL as id and the response text as content. Metadata includes source (the URL) and content_type (from the response header).

Implementations

impl WebBaseLoader

pub fn new(url: impl Into<String>) -> Self

Trait Implementations

impl Loader for WebBaseLoader

fn load<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<Document>, SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Load all documents from this source.

fn lazy_load( &self, ) -> Pin<Box<dyn Stream<Item = Result<Document, SynapticError>> + Send + '_>>

Stream documents lazily. Default implementation wraps load().