pub struct SplitProvider { /* private fields */ }Expand description
A Provider that runs skills with one provider and judges with another:
respond (and supports_resume) go to the skill-running provider; judge
and simulate_user go to the judge. This keeps harness fidelity for the
thing under test while letting the judge run on a fast, cheap, deterministic
backend (typically ApiJudgeProvider).
Implementations§
Source§impl SplitProvider
impl SplitProvider
Sourcepub fn new(responder: Box<dyn Provider>, judge: ApiJudgeProvider) -> Self
pub fn new(responder: Box<dyn Provider>, judge: ApiJudgeProvider) -> Self
Compose a skill-running responder with an API judge.
Trait Implementations§
Source§impl Provider for SplitProvider
impl Provider for SplitProvider
Source§fn respond(
&self,
platform: &str,
model: &str,
skill: &SkillRef<'_>,
messages: &[Message],
session: Option<&str>,
) -> Result<AssistantTurn>
fn respond( &self, platform: &str, model: &str, skill: &SkillRef<'_>, messages: &[Message], session: Option<&str>, ) -> Result<AssistantTurn>
Run one assistant/skill turn given the conversation so far.
session,
when Some, is a handle returned by a previous respond call on this
run that the provider may use to continue the same harness session
(e.g. via oneharness run --resume); providers that don’t support
continuation should ignore it. Read moreSource§fn simulate_user(
&self,
model: &str,
persona: &str,
messages: &[Message],
) -> Result<UserTurn>
fn simulate_user( &self, model: &str, persona: &str, messages: &[Message], ) -> Result<UserTurn>
Produce one simulated-user turn. Read more
Source§fn judge(
&self,
model: &str,
query: &JudgeQuery<'_>,
messages: &[Message],
) -> Result<JudgeVerdict>
fn judge( &self, model: &str, query: &JudgeQuery<'_>, messages: &[Message], ) -> Result<JudgeVerdict>
Score a criterion against the conversation. Read more
Source§fn supports_resume(&self, platform: &str) -> bool
fn supports_resume(&self, platform: &str) -> bool
True iff
respond on platform will faithfully continue a prior
session when given its session_id. The default is false; providers
that support resume override this so the runner knows to thread the
session id through.Auto Trait Implementations§
impl !RefUnwindSafe for SplitProvider
impl !Send for SplitProvider
impl !Sync for SplitProvider
impl !UnwindSafe for SplitProvider
impl Freeze for SplitProvider
impl Unpin for SplitProvider
impl UnsafeUnpin for SplitProvider
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more