Expand description
Request and response types for inference
Structs§
- Batch
Request - Batch request for processing multiple requests together
- Inference
Request - Inference request
- Inference
Response - Inference response
- Scheduled
Request - Scheduled request with additional state information
- Stream
Chunk - Streaming response chunk
Enums§
- Request
State - Request state in the scheduler