Skip to main content

Provider

skilltest_core::provider

Trait Provider 

Source 
pub trait Provider {
    // Required methods
    fn respond(
        &self,
        platform: &str,
        model: &str,
        skill: &SkillRef<'_>,
        messages: &[Message],
        session: Option<&str>,
    ) -> Result<AssistantTurn>;
    fn simulate_user(
        &self,
        model: &str,
        persona: &str,
        messages: &[Message],
    ) -> Result<UserTurn>;
    fn judge(
        &self,
        model: &str,
        query: &JudgeQuery<'_>,
        messages: &[Message],
    ) -> Result<JudgeVerdict>;

    // Provided methods
    fn respond_streaming(
        &self,
        platform: &str,
        model: &str,
        skill: &SkillRef<'_>,
        messages: &[Message],
        session: Option<&str>,
        on_event: &mut dyn FnMut(&ToolEvent) -> ControlFlow<()>,
    ) -> Result<AssistantTurn> { ... }
    fn supports_resume(&self, _platform: &str) -> bool { ... }
}
Expand description

The provider boundary.

Required Methods§

Source

fn respond( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, ) -> Result<AssistantTurn>

Run one assistant/skill turn given the conversation so far. session, when Some, is a handle returned by a previous respond call on this run that the provider may use to continue the same harness session (e.g. via oneharness run --resume); providers that don’t support continuation should ignore it.

§Errors

Error::Provider if the command fails or returns malformed output.

Source

fn simulate_user( &self, model: &str, persona: &str, messages: &[Message], ) -> Result<UserTurn>

Produce one simulated-user turn.

§Errors

Error::Provider if the command fails or returns malformed output.

Source

fn judge( &self, model: &str, query: &JudgeQuery<'_>, messages: &[Message], ) -> Result<JudgeVerdict>

Score a criterion against the conversation.

§Errors

Error::Provider if the command fails or returns malformed output.

Provided Methods§

Source

fn respond_streaming( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, on_event: &mut dyn FnMut(&ToolEvent) -> ControlFlow<()>, ) -> Result<AssistantTurn>

Like Provider::respond, but delivers each normalized tool event to on_event as it is observed, so a caller can stream events live and short-circuit. on_event returns ControlFlow::Break to abort the turn — the provider tears down the harness and returns the partial turn.

The default implementation runs the buffered Provider::respond and replays the finished turn’s events once; providers that can stream (like OneharnessProvider, via oneharness --stream) override it so events arrive — and an abort takes effect — mid-turn.

§Errors

Error::Provider if the command fails or returns malformed output.

Source

fn supports_resume(&self, _platform: &str) -> bool

True iff respond on platform will faithfully continue a prior session when given its session_id. The default is false; providers that support resume override this so the runner knows to thread the session id through.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§