Skip to main content

Module ai

Module ai 

Source
Expand description

Asking the AI engine for an answer.

Four operations read something and answer with it — extract pulls structured data out against a schema you give, summarize writes prose, transcribe turns audio or video into words, and transcribe_pages reads the pages of a PDF that have no text of their own from their images. Three more hold the tenant’s face collection: enroll_face, search_face and delete_face.

#[derive(Deserialize)]
struct Invoice {
    number: String,
    total: f64,
}

let schema = json!({
    "type": "object",
    "properties": {
        "number": { "type": "string" },
        "total": { "type": "number" }
    },
    "required": ["number", "total"]
});

let read: Execution<Invoice> = simple::ai::extract(
    json!({ "file_hash": "9f2c…", "mime_type": "application/pdf" }),
    "Read the invoice number and the total.",
    schema,
    Options::default(),
)?;

assert_eq!(read.data.number, "INV-8");
assert_eq!(read.metadata.input_tokens, 812);

§The shape, and why it is this one

What an operation must have is an argument; what it may have is a field. An input, a prompt and a schema are the operation — leave one out and there is nothing to run — so they are arguments, and the type system asks for them. Everything else has a working default, so it lives in Options, which derives one:

let options = Options {
    model: Some(Model::Large),
    timeout: Some(Duration::from_secs(90)),
    ..Default::default()
};

Nobody passes six Nones, and a member added to Options later does not disturb a call that was written with ..Default::default().

The answer type is yours. extract is generic over what it decodes into, exactly as crate::graphql::query is, so an extraction lands in your own struct rather than in a Value you then have to pick apart. Where the answer’s shape is settled it is named for you instead: summarize answers with prose, and transcribe answers with a Transcript, because this module wrote the schema that produced it.

Every answer carries what it cost. An operation answers with an Execution — the data and the Metadata beside it — so token counts are there to record without a second call.

§Asking for part of a file, or for its text

A file reference may say how the file should travel and which of its pages should. DocumentInput is that reference with those two beside it, and Delivery and Pages are what they say. A plain handle asks for nothing and sends the whole file the way its type is sent by default.

What was asked for is not always what happened, so every answer says how each file travelled, in Metadata::delivery: as the document, as its text or as an image, which pages were read from their images, and, when text was asked for and the document went instead, which pages could not be read.

§Files that have not been uploaded yet

A document handle that is still pending is uploaded to ephemeral storage before the operation runs, and the operation is given the handle that upload answered with. A handle that is already stored is passed straight through, and so is a list of handles, item by item. Nothing in a call site changes: pass the handle you were given.

Structs§

DeliveryFallback
Why a file asked for as text travelled as the document instead.
DocumentInput
A stored file handed to an AI operation, together with what should be done with it.
Execution
What an operation answered, and what it cost.
FaceSearch
How far a face search reaches.
FileDelivery
How one file an operation carried reached the model.
Metadata
What a run spent, how it reasoned, and how its files reached the model.
Options
What any operation may be told, beyond the inputs that define it.
Pages
The pages of a PDF to use, counted from 1 with both ends included.
TranscribeOptions
What a transcription is to produce.
TranscribePagesOptions
What transcribe_pages may be told, beyond the PDF and the pages.
Transcript
What a transcription answered with.

Enums§

DeliveredAs
How a file reached the model.
Delivery
How a document reaches the model.
Model
The size of model an operation runs on.
PageTranscription
One page’s transcription, or why it has none.
Participants
Whether to tell the speakers apart, and what to call them.
Transcription
The reader used for scanned pages; omission or Standard preserves the default. Precise costs more and suits exact characters (codes, part numbers); text pages are unaffected.

Functions§

delete_face
Remove faces from the tenant’s collection, by the ids they were enrolled under.
enroll_face
Add a face to the tenant’s collection, under the subject it belongs to.
extract
Read structured data out of an input, against a schema you give.
search_face
Find the faces in the tenant’s collection that match an image.
summarize
Write prose about an input.
transcribe
Turn audio or video into words.
transcribe_pages
Read the pages of a PDF that have no usable text of their own — scans, image-only exhibits, text that reads as noise — from their images.