Skip to main content

call_computer_use_backend

computeruse_computer_use

Function call_computer_use_backend 

Source 
pub async fn call_computer_use_backend(
    base64_image: &str,
    goal: &str,
    previous_actions: Option<&[ComputerUsePreviousAction]>,
) -> Result<ComputerUseResponse>
Expand description

Call the Gemini Computer Use backend to get the next action.

This is the main API for communicating with the vision model backend. It sends a screenshot and goal, optionally with previous actions, and receives either a completion signal or the next action to take.

§Arguments

  • base64_image - Base64 encoded PNG screenshot
  • goal - The task to accomplish
  • previous_actions - History of previous actions with their results

§Returns

  • Ok(ComputerUseResponse) - The model’s response
  • Err(e) - If the backend call fails