llmsim 0.4.0

LLM Traffic Simulator - A lightweight, high-performance LLM API simulator
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

//! # LLMSim - LLM Traffic Simulator
//!
//! A lightweight, high-performance LLM API simulator that replicates
//! the traffic shape of real LLM APIs without running actual models.
//!
//! ## Features
//!
//! - Realistic latency simulation (time-to-first-token, inter-token delays)
//! - Streaming support (Server-Sent Events)
//! - Accurate token counting using tiktoken-rs
//! - Error injection for testing error handling
//! - Multiple response generators (lorem, echo, fixed, random)
//! - Realistic model profiles from [models.dev](https://models.dev) with context windows
//!
//! ## Usage
//!
//! ### As a CLI
//!
//! ```bash
//! # Start the server
//! llmsim serve --port 8080
//! ```
//!
//! ### As a Library
//!
//! ```rust,no_run
//! use llmsim::{
//!     openai::{ChatCompletionRequest, Message},
//!     generator::LoremGenerator,
//!     latency::LatencyProfile,
//! };
//!
//! // Create a generator
//! let generator = LoremGenerator::new(100);
//!
//! // Create a latency profile
//! let latency = LatencyProfile::gpt5();
//!
//! // Count tokens
//! let tokens = llmsim::tokens::count_tokens("Hello, world!", "gpt-5").unwrap();
//! ```

// Core library modules
pub mod errors;
pub mod generator;
mod ids;
pub mod latency;
pub mod openai;
pub mod openresponses;
pub mod responses_stream;
pub mod script;
pub mod script_stream;
pub mod stats;
pub mod stream;
pub mod tokens;

// CLI module (for `llmsim serve` command)
pub mod cli;

// TUI module (for `llmsim serve --tui`)
#[cfg(feature = "tui")]
pub mod tui;

// Re-export commonly used types
pub use errors::{ErrorConfig, ErrorInjector, SimulatedError};
pub use generator::{
    create_generator, EchoGenerator, FixedGenerator, LoremGenerator, RandomWordGenerator,
    ResponseGenerator, SequenceGenerator,
};
pub use latency::LatencyProfile;
pub use responses_stream::{ResponsesTokenStream, ResponsesTokenStreamBuilder};
pub use script::{
    OnExhausted, Script, ScriptError, ScriptSpec, ScriptedResponse, SimError, SimToolCall, SimTurn,
};
pub use stats::{new_shared_stats, EndpointType, SharedStats, Stats, StatsSnapshot};
pub use stream::{TokenStream, TokenStreamBuilder};
pub use tokens::{count_tokens, count_tokens_default, TokenCounter, TokenError};