Expand description
Protobuf messages and gRPC clients and servers for vLLM’s Inference and Control APIs.
Modules§
- candidate_
token_ info - Nested message and enum types in
CandidateTokenInfo. - candidate_
tokens - Nested message and enum types in
CandidateTokens. - control_
client - Generated client implementations.
- control_
server - Generated server implementations.
- decoding_
parameters - Nested message and enum types in
DecodingParameters. - finish_
info - Nested message and enum types in
FinishInfo. - generate_
request - Nested message and enum types in
GenerateRequest. - inference_
client - Generated client implementations.
- inference_
server - Generated server implementations.
- media_
item - Nested message and enum types in
MediaItem.
Structs§
- Abort
Request - Abort
Response - Candidate
Token Info - Info for candidate tokens other than the input/sampled token at a given position
- Candidate
Tokens - Controls which extra candidate tokens at each position should be returned
- Decoding
Parameters - Engine
Reasoning Gate - Inputs to the engine’s reasoning gate, which keeps a structured-output constraint inactive until the engine’s reasoning parser detects the end of reasoning.
- Finish
Info - Finish info, returned in the final response
- Finish
Weight Update Request - Finish
Weight Update Response - Generate
Request - Generate
Response - GetKv
Event Sources Request - GetKv
Event Sources Response - GetModel
Info Request - GetServer
Info Request - GetWeight
Version Request - GetWeight
Version Response - Init
Weight Transfer Engine Request - The payloads are backend-specific JSON objects. Tensor data remains on the configured NCCL, IPC, or sparse-NCCL transport rather than crossing gRPC.
- Init
Weight Transfer Engine Response - IsPaused
Request - IsPaused
Response - IsSleeping
Request - IsSleeping
Response - KvCache
Parameters - KvEvent
Source - KvHint
Action - KvHints
Envelope - List
Loras Request - List
Loras Response - Load
Lora Request - Load
Lora Response - Lora
Adapter - Media
Item - Model
Info - Parallelism
Info - Pause
Generation Request - Pause
Generation Response - Preprocessed
Media Features - Engine-ready multimodal features produced by a version-compatible frontend.
kwargsis the MessagePack wire representation of one MmKwargsItem. - Prompt
Info - Prompt info, returned in the first response
- Random
Sampling - Response
Options - Resume
Generation Request - Resume
Generation Response - RlCapabilities
- Sequence
Output - Server
Info - Shutdown
Request - Shutdown
Response - Acknowledges acceptance, not completion; shutdown may close the connection first.
- Sleep
Request - Sleep
Response - Start
Draft Weight Update Request - Start
Draft Weight Update Response - Start
Weight Update Request - Start
Weight Update Response - Stopping
Criteria - Token
Ids - Token ids used for prompt
- Unload
Lora Request - Unload
Lora Response - Update
Weight Version Request - Update
Weight Version Response - Update
Weights Request - Update
Weights Response - Wake
UpRequest - Wake
UpResponse
Enums§
Constants§
- VERSION
- Schema version of this crate, reported by servers as
ServerInfo.proto_version.