Module benchmark 

Benchmark comparison module Benchmark comparison module for UniLLM vs llama.cpp

This module provides infrastructure for comparing inference performance between UniLLM and llama.cpp using the same GGUF model files.

Structs

BenchmarkComparison

Comparison between two backends

BenchmarkConfig

Configuration for benchmark runs

BenchmarkResults

Aggregated benchmark results for a backend

BenchmarkRunner

Benchmark runner that executes benchmarks on backends

GenerationResult

Result of a single generation

InferenceMetrics

Metrics collected per inference run

UniLLMBackend

UniLLM inference backend

Traits

InferenceBackend

Trait for inference backends that can be benchmarked

Functions

get_process_memory

Get current process memory usage in bytes

print_comparison_table

Print comparison results as a formatted table

print_single_results

Print results for a single backend