Struct Request 

pub struct Request<'a> { /* private fields */ }

Inference request object.
One can get this item using Server::create_request.

It’s required to add input data and Allocator to this structure before the inference via one of add_input methods via Request::add_allocator or Request::add_default_allocator method.

Implementations

impl Request<'_>

Start inference.

pub fn infer_async(self) -> Result<ResponseFuture, Error>

Perform inference using the metadata and inputs supplied by the Request(self).
If the function returns success, the returned struct can be used to get results (.await) of the inference and to return input buffers after the inference start ResponseFuture::get_input_release.
Note: output buffer will be returned with Response or InferenceError. \

impl<'a> Request<'a>

pub fn add_allocator( &mut self, custom_allocator: Box<dyn Allocator>, ) -> &mut Self

Add custom Allocator to the request.
Check Allocator trait for more info.

pub fn add_default_allocator(&mut self) -> &mut Self

Add DefaultAllocator to the request.
Check Allocator trait and DefaultAllocator for more info.

pub fn add_trace(&mut self, custom_trace: Trace) -> &mut Self

Add custom Trace to the request.
If this method is not invoked, no tracing will be provided.
Check Trace for more info about tracing.

pub fn get_id(&self) -> Result<String, Error>

Get the ID of the request.

pub fn set_id<I: AsRef<str>>(&mut self, id: I) -> Result<&mut Self, Error>

Set the ID of the request.

pub fn get_flags(&self) -> Result<Sequence, Error>

👎Deprecated since 0.4.1: Can’t be used with bit operations as flags. Use Request::get_sequence_flags instead

Get the flag(s) associated with the request.
Check Sequence for available flags.

pub fn get_sequence_flags(&self) -> Result<SequenceFlag, Error>

Get the flag(s) associated with the request.
Check SequenceFlag for available flags.

pub fn set_flags(&mut self, flags: Sequence) -> Result<&mut Self, Error>

👎Deprecated since 0.4.1: Can’t be used with bit operations as flags. Use Request::set_sequence_flags instead

Set the flag(s) associated with a request.
Check Sequence for available flags.

pub fn set_sequence_flags( &mut self, flags: SequenceFlag, ) -> Result<&mut Self, Error>

Set the flag(s) associated with a request.
Check SequenceFlag for available flags.

pub fn get_correlation_id(&self) -> Result<u64, Error>

Get the correlation ID of the inference request.
Default is 0, which indicates that the request has no correlation ID.
If the correlation id associated with the inference request is a string, this function will return a failure.
The correlation ID is used to indicate two or more inference request are related to each other.
How this relationship is handled by the inference server is determined by the model’s scheduling policy.

pub fn get_correlation_id_as_string(&self) -> Result<String, Error>

Get the correlation ID of the inference request as a string.
Default is empty “”, which indicates that the request has no correlation ID.
If the correlation id associated with the inference request is an unsigned integer, then this function will return a failure.
The correlation ID is used to indicate two or more inference request are related to each other.
How this relationship is handled by the inference server is determined by the model’s scheduling policy.

pub fn set_correlation_id(&mut self, id: u64) -> Result<&mut Self, Error>

Set the correlation ID of the inference request to be an unsigned integer.
Default is 0, which indicates that the request has no correlation ID.
The correlation ID is used to indicate two or more inference request are related to each other.
How this relationship is handled by the inference server is determined by the model’s scheduling policy.

pub fn set_correlation_id_as_str<I: AsRef<str>>( &mut self, id: I, ) -> Result<&mut Self, Error>

Set the correlation ID of the inference request to be a string.
The correlation ID is used to indicate two or more inference request are related to each other.
How this relationship is handled by the inference server is determined by the model’s scheduling policy.

pub fn get_priority(&self) -> Result<u32, Error>

Get the priority of the request.
The default is 0 indicating that the request does not specify a priority and so will use the model’s default priority.

pub fn set_priority(&mut self, priority: u32) -> Result<&mut Self, Error>

Set the priority of the request.
The default is 0 indicating that the request does not specify a priority and so will use the model’s default priority.

pub fn get_timeout(&self) -> Result<Duration, Error>

Get the timeout of the request.
The default is 0 which indicates that the request has no timeout.

pub fn set_timeout(&mut self, timeout: Duration) -> Result<&mut Self, Error>

Set the timeout of the request.
The default is 0 which indicates that the request has no timeout.

pub fn add_input<N: AsRef<str>>( &mut self, input_name: N, buffer: Buffer, ) -> Result<&mut Self, Error>

Add an input to the request.
input_name: The name of the input.
buffer: input data containing buffer.
Note: input data will be returned after the inference. Check ResponseFuture::get_input_release for more info.

pub fn add_input_with_dims<N, D>( &mut self, input_name: N, buffer: Buffer, dims: D, ) -> Result<&mut Self, Error>

where N: AsRef<str>, D: AsRef<[i64]>,

Add an input with the specified shape to the request.
input_name: The name of the input.
buffer: input data containing buffer.
dims: Dimensions of the input.
Note: input data will be returned after the inference. Check ResponseFuture::get_input_release for more info.

pub fn add_input_with_policy<N, P>( &mut self, input_name: N, buffer: Buffer, policy: P, ) -> Result<&mut Self, Error>

where N: AsRef<str>, P: AsRef<str>,

Add an input with the specified host policy to the request.
input_name: The name of the input.
buffer: input data containing buffer.
policy: The policy name, all model instances executing with this policy will use this input buffer for execution.
Note: input data will be returned after the inference. Check ResponseFuture::get_input_release for more info.

pub fn add_input_with_policy_and_dims<N, P, D>( &mut self, input_name: N, buffer: Buffer, policy: P, dims: D, ) -> Result<&mut Self, Error>

where N: AsRef<str>, P: AsRef<str>, D: AsRef<[i64]>,

Add an input with the specified host policy and shape to the request. input_name: The name of the input.
buffer: input data containing buffer.
policy: The policy name, all model instances executing with this policy will use this input buffer for execution.
dims: Dimensions of the input.
Note: input data will be returned after the inference. Check ResponseFuture::get_input_release for more info.

pub fn remove_input<N: AsRef<str>>(&mut self, name: N) -> Result<Buffer, Error>

Remove an input from a request. Returns appended to the input data.

name The name of the input. \

pub fn remove_all_inputs(&mut self) -> Result<HashMap<String, Buffer>, Error>

Remove all the inputs from a request. Returns appended to the inputs data.

pub fn set_parameter( &mut self, parameter: Parameter, ) -> Result<&mut Self, Error>

Set a parameter in the request. Does not support ParameterContent::Bytes.

Trait Implementations

impl Drop for Request<'_>

fn drop(&mut self)

Executes the destructor for this type.

impl Send for Request<'_>