Structs§
- Response
Stream - Adapter for a
DataStreamto aResponseStream.
Traits§
- Async
Engine - Engine is a trait that defines the interface for a steaming LLM completion engine. The synchronous Engine version is does not need to be awaited.
- Async
Engine Context - The
AsyncEngineContexttrait defines the interface to control the resulting stream produced by the engine. - Async
Engine Context Provider - Async
Engine Controller - Async
Engine Stream - Async
Engine Unary - Data
- All
Send+Sync+'statictypes can be used asAsyncEnginerequest and response types.
Type Aliases§
- Context
- Data
Stream - Data
Unary DataStreamis a type alias for a stream ofDataitems. This can be adapted to aResponseStreamby associating it with aAsyncEngineContext.- Engine
- Engine
Stream - Engine
Unary