Expand description
Task definition types for benchmark harness.
Defines the JSON schema for benchmark tasks, including setup, verification,
and metrics collection. Tasks run in isolated workspaces with their own
.git directories to avoid polluting the main repository.
§Example
use ralph_core::task_definition::{TaskDefinition, TaskSuite, Verification};
let task = TaskDefinition::builder("hello-world", "tasks/hello-world/PROMPT.md", "TASK_COMPLETE")
.verification_command("python hello.py | grep -q 'Hello, World!'")
.max_iterations(5)
.expected_iterations(1)
.complexity("simple")
.build();
assert_eq!(task.name, "hello-world");
assert!(task.verification.command.contains("Hello, World!"));Structs§
- Suite
Metadata - Suite-level metadata.
- Task
Definition - A single benchmark task definition.
- Task
Definition Builder - Builder for constructing task definitions.
- Task
Setup - Setup configuration for task workspace.
- Task
Suite - A suite of benchmark tasks loaded from a JSON file.
- Verification
- Verification configuration for a task.
Enums§
- Task
Definition Error - Errors that can occur when working with task definitions.