Expand description
Conversations evaluated by simulated users (ADR-021): a model plays a person with a goal and a manner, talks to the runtime until the goal or a turn limit, and code scores each conversation from the stores and the turns.
| Module | What it owns |
|---|---|
goal | what the person wants, loaded strictly from .toml, and the state that proves it |
user | what the person sees and does next: scripted, or played by a model |
converse | one conversation held with a harness’s runtime, and a run over many |
score | the classes a conversation is scored on, and a run’s rates |
The run and its classes are described in docs/evaluation.md.
Re-exports§
pub use converse::Simulation;pub use converse::converse;pub use converse::turn_at;pub use goal::Goal;pub use goal::Reached;pub use score::Conversation;pub use score::ConversationScore;pub use score::Ending;pub use score::Exchange;pub use score::SimulationReport;pub use score::score;pub use user::CardOnScreen;pub use user::ModelUser;pub use user::Screen;pub use user::ScriptedUser;pub use user::SimulatedUser;pub use user::UserMove;
Modules§
- converse
- One conversation held with the runtime, turn by turn, until the goal or the limit.
- goal
- What a simulated user wants, and the state that proves it got it.
- score
- A conversation scored by code, and a run reported as rates over its conversations.
- user
- The person a conversation is held with: what they see, and what they do next.