Skip to main content

Client

Trait Client 

Source
pub trait Client: Send + Sync {
    type Error: Error + Send + Sync + 'static;
    type Stream: Stream<Item = Result<Event, Self::Error>> + Send;

    // Required methods
    fn info(&self) -> Info;
    fn generate(
        &self,
        request: Request,
    ) -> impl Future<Output = Result<Self::Stream, Self::Error>> + Send;
}
Expand description

Narrow, statically dispatched model provider boundary.

This is the one trait reef cannot ship an implementation of: it is the adapter between reef’s Request and whatever actually answers it. Two methods, one of which is a constant.

The explicit RPITIT future follows ADR-013 and avoids async-trait. The returned stream remains provider-owned; callers must apply StreamGrammar.

§Implementing it

Client::Stream is bounded by Stream, which lives in futures-core. That crate is deliberately combinator-free — there is no stream::iter and no StreamExt — so the associated type is a type you write, with a poll_next, and it is about ten lines. Reach for the bound through reef_core::futures_core rather than adding the crate yourself, so the two versions cannot disagree.

This is the whole of a provider that replays a fixed script, which is what a test double looks like:

use std::future::{Future, ready};
use std::pin::Pin;
use std::task::{Context, Poll};

use reef_core::futures_core::Stream;
use reef_core::runtime::model;
use reef_core::value::content;

/// The events of one call, oldest first.
#[derive(Debug)]
struct Scripted(std::collections::VecDeque<model::Event>);

impl Stream for Scripted {
    type Item = Result<model::Event, Unreachable>;

    fn poll_next(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Option<Self::Item>> {
        // A script is always ready, so this never registers a waker. A real
        // adapter polls its transport here and returns `Poll::Pending` when
        // that transport has registered one.
        Poll::Ready(self.0.pop_front().map(Ok))
    }
}

#[derive(Debug)]
struct Unreachable;

impl std::fmt::Display for Unreachable {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str("a scripted provider cannot fail")
    }
}

impl std::error::Error for Unreachable {}

#[derive(Debug)]
struct ScriptedModel;

impl model::Client for ScriptedModel {
    type Error = Unreachable;
    type Stream = Scripted;

    fn info(&self) -> model::Info {
        model::Info::new("scripted/v1".to_owned(), 8_192).with_tools(true)
    }

    fn generate(
        &self,
        request: model::Request,
    ) -> impl Future<Output = Result<Self::Stream, Self::Error>> + Send {
        // `request.messages()` is already projected: every item in it is
        // model-visible and every tool exchange in it is closed.
        let _turns = request.messages().len();
        ready(Ok(Scripted(
            [
                model::Event::ContentFinal {
                    id: "answer".to_owned(),
                    kind: model::ContentKind::Text,
                    content: "Hello.".to_owned(),
                    extensions: content::SegmentExtensions::new(String::new(), None, None),
                },
                model::Event::Usage(model::TokenUsage::new(12, 3)),
                model::Event::Done {
                    finish_reason: "stop".to_owned(),
                    completion: model::CompletionReason::Stop,
                    provider_data: Vec::new(),
                },
            ]
            .into(),
        )))
    }
}

Every event that stream yields is checked by StreamGrammar before reef acts on it, so an adapter can validate its own output against the same rules in its own tests.

Required Associated Types§

Source

type Error: Error + Send + Sync + 'static

Provider-specific call failure.

Source

type Stream: Stream<Item = Result<Event, Self::Error>> + Send

Normalized event stream for one call.

Required Methods§

Source

fn info(&self) -> Info

Returns the immutable model capability snapshot.

Source

fn generate( &self, request: Request, ) -> impl Future<Output = Result<Self::Stream, Self::Error>> + Send

Starts one model call from an owned request.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§