Struct NvCreateChatCompletionRequest 

pub struct NvCreateChatCompletionRequest {
    pub inner: CreateChatCompletionRequest,
    pub common: CommonExt,
    pub nvext: Option<NvExt>,
    pub chat_template_args: Option<HashMap<String, Value>>,
}

A request structure for creating a chat completion, extending OpenAI’s CreateChatCompletionRequest with NvExt extensions and common fields.

Fields

• inner: The base OpenAI chat completion request, embedded using serde(flatten).
• common: Common extension fields (ignore_eos, min_tokens) at root level, embedded using serde(flatten).
• nvext: The optional NVIDIA extension field. See NvExt for more details. Note: If ignore_eos is specified in both common and nvext, the common (root-level) value takes precedence.

Fields

inner: CreateChatCompletionRequest

common: CommonExt

nvext: Option<NvExt>

chat_template_args: Option<HashMap<String, Value>>

Extra args to pass to the chat template rendering context

Implementations

impl NvCreateChatCompletionRequest

Provides a method for generating a DeltaGenerator from a chat completion request.

pub fn response_generator(&self, request_id: String) -> DeltaGenerator

Creates a DeltaGenerator instance based on the chat completion request.

Arguments

• request_id - The request ID to use for the chat completion response ID.

Returns

• DeltaGenerator configured with model name and response options.

Trait Implementations

impl AnnotationsProvider for NvCreateChatCompletionRequest

Implements AnnotationsProvider for NvCreateChatCompletionRequest, enabling retrieval and management of request annotations.

fn annotations(&self) -> Option<Vec<String>>

Retrieves the list of annotations from NvExt, if present.

fn has_annotation(&self, annotation: &str) -> bool

Checks whether a specific annotation exists in the request.

Arguments

• annotation - A string slice representing the annotation to check.

Returns

true if the annotation exists, false otherwise.

impl Clone for NvCreateChatCompletionRequest

fn clone(&self) -> NvCreateChatCompletionRequest

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl CommonExtProvider for NvCreateChatCompletionRequest

Implements CommonExtProvider for NvCreateChatCompletionRequest, providing access to common extension fields.

fn common_ext(&self) -> Option<&CommonExt>

Returns a reference to the CommonExt struct.

fn get_guided_json(&self) -> Option<&Value>

Guided Decoding Options

fn get_guided_regex(&self) -> Option<String>

fn get_guided_grammar(&self) -> Option<String>

fn get_guided_choice(&self) -> Option<Vec<String>>

fn get_guided_decoding_backend(&self) -> Option<String>

fn get_top_k(&self) -> Option<i32>

Other sampling Options

fn get_min_p(&self) -> Option<f32>

fn get_repetition_penalty(&self) -> Option<f32>

fn get_include_stop_str_in_output(&self) -> Option<bool>

impl Debug for NvCreateChatCompletionRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for NvCreateChatCompletionRequest

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl NvExtProvider for NvCreateChatCompletionRequest

Implements NvExtProvider for NvCreateChatCompletionRequest, providing access to NVIDIA-specific extensions.

fn nvext(&self) -> Option<&NvExt>

Returns a reference to the optional NvExt extension, if available.

fn raw_prompt(&self) -> Option<String>

Returns None, as raw prompt extraction is not implemented.

impl OAIChatLikeRequest for NvCreateChatCompletionRequest

fn model(&self) -> String

fn messages(&self) -> Value

fn tools(&self) -> Option<Value>

fn tool_choice(&self) -> Option<Value>

fn should_add_generation_prompt(&self) -> bool

fn extract_text(&self) -> Option<TextInput>

fn chat_template_args(&self) -> Option<&HashMap<String, Value>>

Optional additional args to merge into the chat template context

fn prompt_input_type(&self) -> PromptInput

Returns the type of input for the prompt. Default is Text.

fn extract_tokens(&self) -> Option<TokenInput>

Extract tokens if the input is pre-tokenized

impl Serialize for NvCreateChatCompletionRequest

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl TryFrom<NvCreateResponse> for NvCreateChatCompletionRequest

type Error = Error

The type returned in the event of a conversion error.

fn try_from(resp: NvCreateResponse) -> Result<Self, Self::Error>

Performs the conversion.

impl Validate for NvCreateChatCompletionRequest

fn validate(&self) -> Result<(), ValidationErrors>

impl<'v_a> ValidateArgs<'v_a> for NvCreateChatCompletionRequest

type Args = ()

fn validate_with_args(&self, args: Self::Args) -> Result<(), ValidationErrors>

impl ValidateRequest for NvCreateChatCompletionRequest

Implements ValidateRequest for NvCreateChatCompletionRequest, allowing us to validate the data.

fn validate(&self) -> Result<(), Error>