Struct DebugOutput 

pub struct DebugOutput {

    pub attempts: i32,
    pub request: String,
    pub prompt: String,
    pub engine_request: String,
    pub responses: Vec<String>,
    pub chunks: Vec<String>,
    pub cache_read_count: i32,
    pub cache_read_input_bytes: i64,
    pub cache_write_count: i32,
    pub cache_write_input_bytes: i64,
    pub lb_address: String,
    pub sampler_tag: String,
    pub sampler_checkpoint_mount: String,
}

Debug output. Only available to trusted testers.

Fields

attempts: i32

Number of attempts made to the model.

request: String

The request received from the user.

prompt: String

The prompt sent to the model in text form.

engine_request: String

JSON-serialized request sent to the inference engine.

responses: Vec<String>

The response(s) received from the model.

chunks: Vec<String>

The individual chunks returned from the pipeline of samplers.

cache_read_count: i32

Number of cache reads.

cache_read_input_bytes: i64

Size of cache read.

cache_write_count: i32

Number of cache writes.

cache_write_input_bytes: i64

Size of cache write.

lb_address: String

The load balancer address.

sampler_tag: String

The tag of the actual engines sitting behind the GTP address.

sampler_checkpoint_mount: String

The underlying checkpoint mount path for the sampler that served this request.

Trait Implementations

impl Clone for DebugOutput

fn clone(&self) -> DebugOutput

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DebugOutput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DebugOutput

fn default() -> DebugOutput

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DebugOutput

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for DebugOutput

fn eq(&self, other: &DebugOutput) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for DebugOutput

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for DebugOutput