Skip to main content

Provider

skilltest_core::provider

Trait Provider 

Source 
pub trait Provider {
    // Required methods
    fn respond(
        &self,
        platform: &str,
        model: &str,
        skill: &SkillRef<'_>,
        messages: &[Message],
        session: Option<&str>,
    ) -> Result<AssistantTurn>;
    fn simulate_user(
        &self,
        model: &str,
        persona: &str,
        messages: &[Message],
    ) -> Result<UserTurn>;
    fn judge(
        &self,
        model: &str,
        query: &JudgeQuery<'_>,
        messages: &[Message],
    ) -> Result<JudgeVerdict>;

    // Provided method
    fn supports_resume(&self, _platform: &str) -> bool { ... }
}
Expand description

The provider boundary.

Required Methods§

Source

fn respond( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, ) -> Result<AssistantTurn>

Run one assistant/skill turn given the conversation so far. session, when Some, is a handle returned by a previous respond call on this run that the provider may use to continue the same harness session (e.g. via oneharness run --resume); providers that don’t support continuation should ignore it.

§Errors

Error::Provider if the command fails or returns malformed output.

Source

fn simulate_user( &self, model: &str, persona: &str, messages: &[Message], ) -> Result<UserTurn>

Produce one simulated-user turn.

§Errors

Error::Provider if the command fails or returns malformed output.

Source

fn judge( &self, model: &str, query: &JudgeQuery<'_>, messages: &[Message], ) -> Result<JudgeVerdict>

Score a criterion against the conversation.

§Errors

Error::Provider if the command fails or returns malformed output.

Provided Methods§

Source

fn supports_resume(&self, _platform: &str) -> bool

True iff respond on platform will faithfully continue a prior session when given its session_id. The default is false; providers that support resume override this so the runner knows to thread the session id through.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§