Struct InferenceResult 

pub struct InferenceResult {

    pub text: String,
    pub tool_calls: Vec<ToolCall>,
    pub bounding_boxes: Vec<BoundingBox>,
    pub trace_id: String,
    pub model_used: String,
    pub model_identity: InferenceModelIdentity,
    pub latency_ms: u64,
    pub time_to_first_token_ms: Option<u64>,
    pub usage: Option<TokenUsage>,
    pub provider_output_items: Vec<Value>,
    pub thinking: Vec<ThinkingBlock>,
    pub stop_reason: Option<String>,
    pub auth_fallback_from: Option<String>,
    pub local_last_resort: bool,
    pub fallback_from: Vec<FallbackFrom>,
}

Result of an inference call, including trace ID for outcome tracking.

Fields

text: String

The generated text (empty if tool_calls are present).

tool_calls: Vec<ToolCall>

Tool calls returned by the model (when tools were provided in the request).

bounding_boxes: Vec<BoundingBox>

Structured bounding boxes when the model emitted Qwen2.5-VL grounding spans (<|box_*|>, <|object_ref_*|>) in its text. Parsed from the same text field — the raw span markers remain visible in text for callers that need to see them verbatim. Empty vec when the model didn’t ground anything (typical for non-VL models or prompts that only ask for description).

trace_id: String

Trace ID for reporting outcomes back to the tracker.

model_used: String

Which model was used. Exact catalog-id pins report the immutable resolved id; legacy/adaptive routes retain their display-name behavior.

model_identity: InferenceModelIdentity

Immutable catalog identity, flattened onto the v3 inference result so callers can validate the route without another lookup.

latency_ms: u64

Wall-clock latency in ms.

time_to_first_token_ms: Option<u64>

Time to first token in milliseconds. Populated by the local generate paths (Candle/MLX) which observe the prefill→first-decode transition directly. None for paths that can’t measure it honestly without streaming — currently the non-streaming remote paths. Callers needing TTFT on remote models should use InferenceEngine::generate_tracked_stream and time the first text event arrival themselves.

Always serialized (as null when None) so downstream validation harnesses can distinguish “wasn’t measured” from “field doesn’t exist on this client’s protocol version”.

usage: Option<TokenUsage>

Token usage for the call. Populated by the remote providers from their API response, and by the local backends from their own decode loops — the in-process MLX and candle paths report the post-truncation prompt length and the number of tokens they sampled, and the mlx-vlm CLI path reports the counts the CLI prints (image patches included).

None means nobody could report a count, and it is deliberately not a zeroed struct: a consumer summing total_tokens cannot tell a fabricated 0 from a real “this used no tokens”, so an absent count is the honest answer and lets callers fall back to their own estimator (Parslee-ai/car#795). Still None on: FoundationModels (Apple’s on-device framework exposes no token counts), a delegated runner that emits no usage stream event, and an mlx-vlm build whose performance summary doesn’t parse.

TokenUsage::context_window is 0 on the streaming path — the accumulator builds usage from stream events, which carry no model metadata. Non-streaming calls populate it.

provider_output_items: Vec<Value>

Provider-specific output items the protocol emitted alongside the response — currently used by the OpenAI Responses API to return reasoning blobs, encrypted_content, web-search results, etc. as opaque structured items the next request must include verbatim. Empty for protocols that don’t emit them (Chat Completions, Anthropic, Gemini, all local backends).

Callers carry these between turns by emitting them as a tasks::generate::Message::ProviderOutputItems message in the next request. Builder paths that don’t recognize the originating protocol drop the variant — the items are protocol-specific and have no portable rendering.

thinking: Vec<ThinkingBlock>

Extended-thinking blocks the model produced this turn (Anthropic adaptive thinking). Captured verbatim (text + opaque signature) so the caller can attach them to the replayed tasks::generate::Message::Assistant and preserve them on the next turn — Anthropic 400s if prior thinking blocks aren’t sent back unchanged before the tool_use blocks. Empty for providers/models without thinking (Chat Completions, Gemini, all local backends).

stop_reason: Option<String>

Why generation stopped. For remote models this is the raw provider string (OpenAI finish_reason, Anthropic stop_reason, Google finishReason). For local Qwen3 hybrid-thinking models the runtime also sets it for its reasoning-recovery path (car-releases#60): "thinking_recovered" when reasoning consumed the whole token budget inside an unclosed <think> block and the runtime retried with reasoning suppressed to produce a direct answer, or "thinking_truncated" when even that retry was empty. A model decoded in-process also reports "local_decode_timeout" (LOCAL_DECODE_TIMEOUT_STOP_REASON) when the wall-clock ceiling cut the pass short (car#851). None for an ordinary local completion or a provider that didn’t report one. Always serialized (as null when None) so the wire contract is stable — see the inference_result_serializes_* tests. Use InferenceResult::was_truncated to detect a cut-short response.

auth_fallback_from: Option<String>

The candidate that was skipped because its credential was REJECTED (not merely absent), when a later candidate in the fallback chain then succeeded. None on the common path.

Exists so a caller can ANNOUNCE the degrade instead of silently serving a different model: an operator whose Parslee sign-in lapsed otherwise sees a working run on a fallback backbone with no hint that the lane they configured is dead (Parslee-ai/car#888).

local_last_resort: bool

True when this turn was served by the installed on-device model that CAR appended behind an otherwise remote-only fallback chain.

This is distinct from merely using a local model: an explicitly chosen local primary is ordinary routing. Callers should surface this marker so a resilience fallback cannot masquerade as the preferred remote model.

fallback_from: Vec<FallbackFrom>

Every candidate the chain moved past, in order, and WHY. Empty when the first candidate served.

The general form of InferenceResult::auth_fallback_from, which answers only “was a credential rejected”. A run whose backbone changed mid-session because of a rate limit, a timeout, or an absent credential had no reason recorded anywhere at all — so a surprising result could be attributed to the code under test when the real cause was that a different model wrote it (Parslee-ai/car#1351).

Not a superset of auth_fallback_from, even though it holds every hop. FallbackReason::CredentialRejected is deliberately broader than that field’s predicate: it includes a provider refusing an API key (ProviderAccount 401), whose remedy is to fix the key. auth_fallback_from names only the narrower set a person clears by signing in, because the announcement it drives says car auth login — and telling someone to sign in over a bad OpenAI key is the wrong remedy (Parslee-ai/car#888). Recording both keeps the journal general without making the announcement wrong.

Implementations

impl InferenceResult

pub fn served_model_id(&self) -> &str

Canonical immutable id of the model that served this turn.

Older/scripted payloads may not carry the flattened v3 identity fields; retain model_used as their compatibility fallback.

pub fn has_tool_calls(&self) -> bool

Returns true if the model chose to call tools instead of generating text.

pub fn was_truncated(&self) -> bool

Returns true when the response was cut short rather than finished. Matches every provider spelling of an output-token cap: OpenAI chat "length", OpenAI Responses "max_output_tokens", Anthropic "max_tokens", Google "MAX_TOKENS", and the local MLX/Candle "length" — plus the local wall-clock ceiling ("local_decode_timeout", car#851), which produces the same partial text for a different reason and would otherwise read as a complete answer. A truncated response often carries a half-written tool_use argument the validator will reject, so callers (e.g. car-cli run_task) should detect this and ask the model to retry in smaller chunks rather than re-emitting the oversized call.

pub fn append_assistant_history( &self, messages: &mut Vec<Message>, tool_calls: Vec<ToolCall>, )

Append this result to a caller-owned multi-turn history.

Responses continuity items belong immediately before the assistant message they accompanied in the provider’s output sequence. Keeping the ordering here centralized prevents CAR’s agent, coder, bench, and CLI loops from independently dropping or misordering opaque reasoning state. Personal Chat Completions and non-Responses providers leave provider_output_items empty, so their history shape is unchanged.

Trait Implementations

impl Clone for InferenceResult

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceResult

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for InferenceResult

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl JsonSchema for InferenceResult

fn schema_name() -> String

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn is_referenceable() -> bool

Whether JSON Schemas generated for this type should be re-used where possible using the $ref keyword.

impl Serialize for InferenceResult

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.