Expand description
Asking the AI engine for an answer.
Four operations read something and answer with it — extract pulls
structured data out against a schema you give, summarize writes prose,
transcribe turns audio or video into words, and transcribe_pages
reads the pages of a PDF that have no text of their own from their images.
Three more hold the tenant’s face collection: enroll_face,
search_face and delete_face.
#[derive(Deserialize)]
struct Invoice {
number: String,
total: f64,
}
let schema = json!({
"type": "object",
"properties": {
"number": { "type": "string" },
"total": { "type": "number" }
},
"required": ["number", "total"]
});
let read: Execution<Invoice> = simple::ai::extract(
json!({ "file_hash": "9f2c…", "mime_type": "application/pdf" }),
"Read the invoice number and the total.",
schema,
Options::default(),
)?;
assert_eq!(read.data.number, "INV-8");
assert_eq!(read.metadata.input_tokens, 812);§The shape, and why it is this one
What an operation must have is an argument; what it may have is a field.
An input, a prompt and a schema are the operation — leave one out and there
is nothing to run — so they are arguments, and the type system asks for them.
Everything else has a working default, so it lives in Options, which
derives one:
let options = Options {
model: Some(Model::Large),
timeout: Some(Duration::from_secs(90)),
..Default::default()
};Nobody passes six Nones, and a member added to Options later does not
disturb a call that was written with ..Default::default().
The answer type is yours. extract is generic over what it decodes
into, exactly as crate::graphql::query is, so an extraction lands in your
own struct rather than in a Value you then have to pick apart. Where the
answer’s shape is settled it is named for you instead: summarize answers
with prose, and transcribe answers with a Transcript, because this
module wrote the schema that produced it.
Every answer carries what it cost. An operation answers with an
Execution — the data and the Metadata beside it — so token counts are
there to record without a second call.
§Asking for part of a file, or for its text
A file reference may say how the file should travel and which of its pages
should. DocumentInput is that reference with those two beside it, and
Delivery and Pages are what they say. A plain handle asks for nothing
and sends the whole file the way its type is sent by default.
What was asked for is not always what happened, so every answer says how
each file travelled, in Metadata::delivery: as the document, as its text
or as an image, which pages were read from their images, and, when text was
asked for and the document went instead, which pages could not be read.
§Files that have not been uploaded yet
A document handle that is still pending is uploaded to ephemeral storage before the operation runs, and the operation is given the handle that upload answered with. A handle that is already stored is passed straight through, and so is a list of handles, item by item. Nothing in a call site changes: pass the handle you were given.
Structs§
- Delivery
Fallback - Why a file asked for as text travelled as the document instead.
- Document
Input - A stored file handed to an AI operation, together with what should be done with it.
- Execution
- What an operation answered, and what it cost.
- Face
Search - How far a face search reaches.
- File
Delivery - How one file an operation carried reached the model.
- Metadata
- What a run spent, how it reasoned, and how its files reached the model.
- Options
- What any operation may be told, beyond the inputs that define it.
- Pages
- The pages of a PDF to use, counted from 1 with both ends included.
- Transcribe
Options - What a transcription is to produce.
- Transcribe
Pages Options - What
transcribe_pagesmay be told, beyond the PDF and the pages. - Transcript
- What a transcription answered with.
Enums§
- Delivered
As - How a file reached the model.
- Delivery
- How a document reaches the model.
- Model
- The size of model an operation runs on.
- Page
Transcription - One page’s transcription, or why it has none.
- Participants
- Whether to tell the speakers apart, and what to call them.
- Transcription
- The reader used for scanned pages; omission or
Standardpreserves the default.Precisecosts more and suits exact characters (codes, part numbers); text pages are unaffected.
Functions§
- delete_
face - Remove faces from the tenant’s collection, by the ids they were enrolled under.
- enroll_
face - Add a face to the tenant’s collection, under the subject it belongs to.
- extract
- Read structured data out of an input, against a schema you give.
- search_
face - Find the faces in the tenant’s collection that match an image.
- summarize
- Write prose about an input.
- transcribe
- Turn audio or video into words.
- transcribe_
pages - Read the pages of a PDF that have no usable text of their own — scans, image-only exhibits, text that reads as noise — from their images.