Module middleware 

Model middleware.

A middleware wraps a model: it may rewrite call options, wrap the generate/stream calls, and override identity or supported URLs. Apply with wrap_language_model; the first middleware in the list is the outermost. EmbeddingModelMiddleware / wrap_embedding_model and ImageModelMiddleware / wrap_image_model do the same for embedding and image models, and wrap_provider applies all three kinds to every model a provider resolves.

Modules

builtin

Built-in middleware.

Structs

EmbeddingMiddlewareContext

The embedding model being wrapped.

ImageMiddlewareContext

The image model being wrapped.

MiddlewareContext

The model being wrapped and the kind of call.

ProviderMiddleware

Middleware applied by wrap_provider to the models a provider resolves.

Enums

CallKind

Which model method is being called.

Traits

EmbeddingModelMiddleware

Intercepts embedding model calls. Every method has a pass-through default.

ImageModelMiddleware

Intercepts image model calls. Every method has a pass-through default.

LanguageModelMiddleware

Intercepts language model calls. Every method has a pass-through default.

Functions

wrap_embedding_model

Wraps model with middleware; the first entry becomes the outermost layer. An empty list returns model unchanged.

wrap_image_model

Wraps model with middleware; the first entry becomes the outermost layer. An empty list returns model unchanged.

wrap_language_model

Wraps model with middleware; the first entry becomes the outermost layer. An empty list returns model unchanged.

wrap_provider

Wraps every language, embedding and image model resolved through provider with the matching middleware. Other model kinds, the provider id and the services are delegated unchanged. An empty set returns provider itself.

Type Aliases

EmbedNext

Continuation of a wrapped do_embed.

GenerateNext

Continuation of a wrapped do_generate.

ImageGenerateNext

Continuation of a wrapped image do_generate.

StreamNext

Continuation of a wrapped do_stream.