Pick actions from an image
Pick the best action from a list of actions given an image and context.
This can be used for other AI models, but currently only has a model for ChatGPT-4V.
Usage
Define an ImagePath
:
// Picture of a cute dog
let url = parse.unwrap;
let image_path = Url;
Files can also be used:
let path = from;
let image_path = File;
Then provide context for how actions should be picked. This can be a question or a statement.
let context = "What would make them happiest? They haven't eaten."; // Your question
Then define a list of actions to pick from. The description helps the AI understand what the action is and when it should be picked.
let actions = vec!;
Then create an AI instance. Currently only ChatGPT-4V is implemented.
dotenv.ok;
let api_key = var.expect;
let chat_gpt4v = ChatGpt4v ;
And finally, ask the AI about the image. You'll get back the best action as a String
.
// Ask the AI about the image
let picked_action = pick_action_from_image
.await
.unwrap;
println!;
You can see a full example in the repository's examples
folder.
Setup
Make sure that you have Rust installed.
Then you should setup your OPENAI_API_KEY
environment variable.
To do this, you can copy the .env.example
file to .env
and fill in the value.
You can find your API key in your User Settings.