pub enum Modality<'a> {
Text {
data: Cow<'a, str>,
},
Image {
data: Cow<'a, str>,
},
}
Expand description
The prompt for models can be a combination of different modalities (Text and Image). The type of modalities which are supported depend on the Model in question.
Variants§
Text
The only type of prompt which can be used with pure language models
Image
An image input into the model. See Modality::from_image_path
.
Implementations§
Source§impl<'a> Modality<'a>
impl<'a> Modality<'a>
Sourcepub fn from_image_path(path: impl AsRef<Path>) -> Result<Self, LoadImageError>
pub fn from_image_path(path: impl AsRef<Path>) -> Result<Self, LoadImageError>
Image input for model, from file path.
The model can only see squared pictures. Images are centercropped.
use aleph_alpha_client::{Client, How, Modality, Prompt, Sampling, Stopping, TaskCompletion,
Task, Logprobs};
use dotenvy::dotenv;
use std::path::PathBuf;
#[tokio::main(flavor = "current_thread")]
async fn main() {
// Create client
let client = Client::from_env().unwrap();
// Define task
let task = TaskCompletion {
prompt: Prompt::from_vec(vec![
Modality::from_image_path("cat.png").unwrap(),
Modality::from_text("A picture of "),
]),
stopping: Stopping::from_maximum_tokens(10),
sampling: Sampling::MOST_LIKELY,
special_tokens: false,
logprobs: Logprobs::No,
echo: false,
};
// Execute
let model = "luminous-base";
let job = task.with_model(model);
let response = client.output_of(&job, &How::default()).await.unwrap();
// Show result
println!("{}", response.completion);
}
Sourcepub fn from_image(image: &DynamicImage) -> Result<Self, LoadImageError>
pub fn from_image(image: &DynamicImage) -> Result<Self, LoadImageError>
Image input for model
The model can only see squared pictures. Images are centercropped. You may want to use this
method instead of Self::from_image_path
in case you have the image in memory already
and do not want to load it from a file again.
Sourcepub fn borrow(&self) -> Modality<'_>
pub fn borrow(&self) -> Modality<'_>
Create a semantically idetical entry of modality which borrows the contents of this one.
It is very practical to allow Modality of e.g. Text to take both ownership of the string it contains as well as borrow a slice. However then we are creating a body from the user input we want to avoid copying everything and needing to allocate for that modality again. This is there this borrow function really shines.
Trait Implementations§
impl<'a> Eq for Modality<'a>
impl<'a> StructuralPartialEq for Modality<'a>
Auto Trait Implementations§
impl<'a> Freeze for Modality<'a>
impl<'a> RefUnwindSafe for Modality<'a>
impl<'a> Send for Modality<'a>
impl<'a> Sync for Modality<'a>
impl<'a> Unpin for Modality<'a>
impl<'a> UnwindSafe for Modality<'a>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more