1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
/// Default average custom_id length for JSONL overhead estimation.
///
/// This is a conservative estimate used when actual custom_id data isn't available.
/// Based on typical UUID format (36 chars) or short identifiers (10-20 chars).
///
/// Why 20? It's a middle ground that works well for most use cases:
/// - Short IDs (e.g., "req-123"): ~10 bytes
/// - UUIDs (e.g., "550e8400-e29b-41d4-a716-446655440000"): 36 bytes
/// - Custom strings: typically 10-30 bytes
pub const DEFAULT_AVG_CUSTOM_ID_LENGTH: usize = 20;
/// Default status code for JSONL overhead estimation.
///
/// Uses 200 (OK) as the default since:
/// - Most successful API calls return 2xx status codes
/// - 200 is 3 digits, same as most status codes (100-599)
/// - Overhead difference between codes is minimal (1-3 bytes)
pub const DEFAULT_AVG_STATUS_CODE: i16 = 200;
/// Calculate the raw size of a response body in bytes.
/// This is the actual response content size, not the JSONL-formatted size.
/// Returns None if the size exceeds i64::MAX.
/// Calculate the raw size of an error message in bytes.
/// This is the actual error content size, not the JSONL-formatted size.
/// Returns None if the size exceeds i64::MAX.
/// Calculate JSONL formatting overhead per line for output file.
/// This is the extra bytes added when wrapping a response in JSONL format.
/// Format: {"id":"batch_req_UUID","custom_id":"...","response":{"status_code":200,"body":{...}}}\n
///
/// # Overhead Breakdown
/// - Base structure: ~70 bytes (includes id, custom_id field name, response wrapper)
/// - custom_id value: Varies (included in parameter, already JSON-escaped by caller)
/// - Error margin: 50 bytes fixed overhead to account for:
/// - request_id field if present (~45 bytes)
/// - Additional JSON escaping in response body (~10% handled separately by caller)
/// - Minor variations in formatting
///
/// Note: This is an estimate. Actual overhead may vary based on:
/// - Length of custom_id (if provided, should already be JSON-escaped)
/// - Status code length (1-3 digits typically)
/// - Presence of optional fields like request_id
/// - JSON escaping requirements in the response body
/// Calculate JSONL formatting overhead per line for error file.
/// Format: {"id":"batch_req_UUID","custom_id":"...","response":null,"error":{"code":null,"message":"..."}}\n
///
/// # Overhead Breakdown
/// - Base structure: ~100 bytes (includes id, custom_id, error wrapper)
/// - custom_id value: Varies (included in parameter, already JSON-escaped by caller)
/// - Error margin: 20 bytes fixed overhead to account for:
/// - Minor variations in formatting
/// - Additional escaping needs
///
/// Note: For error messages with extensive JSON escaping (e.g., messages with many quotes,
/// backslashes, or control characters), this may underestimate the actual size. The error
/// message size should already include JSON serialization overhead from the caller.
/// Estimate total JSONL file size for output file from raw response body sizes.
/// Takes the sum of raw body sizes and adds estimated overhead per request.
///
/// Returns None if the calculation would overflow i64 (extremely unlikely in practice,
/// would require petabytes of data).
/// Estimate total JSONL file size for error file from raw error message sizes.
///
/// Returns None if the calculation would overflow i64 (extremely unlikely in practice,
/// would require petabytes of data).