Expand description
Benchmark event protocol for Smith agent optimization
This module defines the complete event schema for collecting agent performance data to optimize Smith’s performance on coding benchmarks like SWE-bench.
Structs§
- Benchmark
Event - Core benchmark event with required tracking fields
- Config
Suggestion - Optimizer configuration suggestions
- Error
State - Final error state classification
- Evidence
Footprint - Evidence of tool’s impact on the codebase
- Failure
Analysis - Structured failure analysis for learning
- Pruning
Decision - Context pruning decisions for optimization
- Recovery
Attempt - Recovery attempt during failure handling
- Retry
Policy - Retry policy configuration
- RunConfig
- Configuration for a benchmark run
- RunResult
- Results from a completed benchmark run
- Sandbox
Limits - Sandbox resource limits
- Step
Data - Individual reasoning/action step data
- Task
Features - Task features for contextual optimization
- Tool
Performance - Tool performance metrics for optimization
Enums§
- Benchmark
Event Type - All benchmark event types for agent optimization
- Exit
Kind - Failure
Root - Segment
Type - Step
Type