Struct SimulateStreamingMiddleware 

pub struct SimulateStreamingMiddleware;

Middleware that emits a do_stream derived from one do_generate call.

Implementations

impl SimulateStreamingMiddleware

pub fn new() -> Self

Construct the default simulation middleware.

Trait Implementations

impl Clone for SimulateStreamingMiddleware

fn clone(&self) -> SimulateStreamingMiddleware

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SimulateStreamingMiddleware

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SimulateStreamingMiddleware

fn default() -> SimulateStreamingMiddleware

Returns the “default value” for a type.

impl LanguageModelMiddleware for SimulateStreamingMiddleware

fn wrap_stream<'life0, 'life1, 'async_trait>( &'life0 self, next: &'life1 dyn LanguageModel, params: CallOptions, ) -> Pin<Box<dyn Future<Output = Result<StreamResult>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Wrap a streaming generation.

fn override_provider(&self, _inner: &dyn LanguageModel) -> Option<String>

Override the provider id exposed by the wrapped model.

fn override_model_id(&self, _inner: &dyn LanguageModel) -> Option<String>

Override the model id exposed by the wrapped model.

fn override_supported_urls<'life0, 'life1, 'async_trait>( &'life0 self, _inner: &'life1 dyn LanguageModel, ) -> Pin<Box<dyn Future<Output = Option<SupportedUrls>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Override the supported-URL map exposed by the wrapped model.

fn transform_params<'life0, 'life1, 'async_trait>( &'life0 self, _kind: CallKind, params: CallOptions, _inner: &'life1 dyn LanguageModel, ) -> Pin<Box<dyn Future<Output = Result<CallOptions>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Transform the call options before they reach the inner model.

fn wrap_generate<'life0, 'life1, 'async_trait>( &'life0 self, next: &'life1 dyn LanguageModel, params: CallOptions, ) -> Pin<Box<dyn Future<Output = Result<GenerateResult>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Wrap a non-streaming generation.

impl Copy for SimulateStreamingMiddleware