Crate llm_test_bench_datasets 

LLM Test Bench Datasets

This crate provides comprehensive dataset management and built-in benchmark datasets for the LLM Test Bench framework.

Features

• Schema validation: Comprehensive dataset schema with serde_valid
• Multiple formats: Load datasets from JSON and YAML files
• Template engine: Variable substitution in prompts using {{variable}} syntax
• Built-in datasets: 5+ ready-to-use benchmark datasets
• Type safety: Strongly-typed Rust structures with validation

Modules

• schema: Dataset schema definitions with validation
• loader: Dataset loading and saving (JSON/YAML)
• template: Template engine for variable substitution
• builtin: Built-in benchmark datasets

Example

use llm_test_bench_datasets::loader::DatasetLoader;
use llm_test_bench_datasets::template::TemplateEngine;
use std::path::Path;

// Load a dataset
let loader = DatasetLoader::new();
let dataset = loader.load(Path::new("datasets/coding-tasks.json")).unwrap();

// Render a templated prompt
let test_case = &dataset.test_cases[0];
if let Some(ref vars) = test_case.variables {
    let prompt = TemplateEngine::render(&test_case.prompt, vars).unwrap();
    println!("Rendered prompt: {}", prompt);
}

Re-exports

pub use schema::Dataset;

pub use schema::TestCase;

pub use schema::DefaultConfig;

pub use schema::TestConfig;

Modules

builtin

Built-in benchmark datasets.

loader

Dataset loader implementation

schema

Dataset schema definitions with validation.

template

Template engine for prompt variable substitution

Enums

DatasetError

Dataset errors