Module types

Expand description

Core types for the LLMKit unified LLM API.

This module defines the unified message format that works across all LLM providers.

§Advanced Features

This module supports advanced LLM features:

Prompt Caching: Reduce costs by caching static content (CacheControl)
Extended Thinking: Enable deep reasoning with budget control (ThinkingConfig)
Structured Outputs: Guarantee JSON schema compliance (StructuredOutput)
Predicted Outputs: Speed up generation for known content (PredictionConfig)
Document Support: Process PDFs and other documents (DocumentSource)

BatchError: Error from a batch request.
BatchJob: Information about a batch job.
BatchRequest: A single request within a batch.
BatchRequestCounts: Request counts for a batch job.
BatchResult: Result of a single request in a batch.
CacheBreakpoint: Cache breakpoint marking content to be cached.
CompletionRequest: Request to complete a conversation.
CompletionResponse: Response from a completion request.
JsonSchemaDefinition: JSON schema definition for structured outputs.
Message: A message in a conversation.
PredictionConfig: Configuration for predicted outputs (speculative decoding).
StreamChunk: A chunk from a streaming response.
StructuredOutput: Configuration for structured output / JSON schema enforcement.
ThinkingConfig: Configuration for extended thinking / reasoning mode.
TokenCountRequest: Request to count tokens in content.
TokenCountResult: Result of a token counting request.
ToolDefinition: Definition of a tool that can be called by the LLM.
Usage: Token usage information.