wbi-rs 0.1.11

A Rust library + CLI to fetch, store, visualize, and summarize World Bank indicator data.
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485

[![Crates.io](https://img.shields.io/crates/v/wbi-rs?label=Crates.io)](https://crates.io/crates/wbi-rs)
[![rust-clippy analyze](https://img.shields.io/github/actions/workflow/status/ardentempiricist/wbi-rs/rust-clippy.yml?label=Rust%20Clippy)](https://github.com/ArdentEmpiricist/wbi-rs/actions/workflows/rust-clippy.yml)
[![Deploy](https://github.com/ArdentEmpiricist/wbi-rs/actions/workflows/deploy.yml/badge.svg)](https://github.com/ArdentEmpiricist/wbi-rs/actions/workflows/deploy.yml)
[![Documentation](https://docs.rs/wbi-rs/badge.svg)](https://docs.rs/wbi-rs/)
[![Crates.io](https://img.shields.io/crates/d/wbi-rs?color=darkblue&label=Downloads)](https://crates.io/crates/wbi-rs)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Rust Edition](https://img.shields.io/badge/rust-2024-orange)](https://doc.rust-lang.org/edition-guide/rust-2024/index.html)

# wbi-rs 🦀📊📈

<p align="center">
  <img src="https://raw.githubusercontent.com/ArdentEmpiricist/wbi-rs/refs/heads/main/assets/logo.png?raw=true" alt="wbi-rs Logo" width="200"/>
</p>

Fetch, analyze, and visualize World Bank data from Rust.  
This project provides a **CLI**, **GUI**, and **library API** to retrieve time series from the World Bank API, export them safely (CSV/JSON), compute grouped statistics, and render charts (SVG/PNG) with Plotters.

---

## Table of contents

- [Features]#features
  - [What it can do]#what-it-can-do
  - [Under the hood]#under-the-hood
- [Install]#install
- [Quick start (CLI)]#quick-start-cli
- [Country-Consistent Styling]#country-consistent-styling
- [GUI Application]#gui-application
  - [Installation and Usage]#installation-and-usage
  - [Features (GUI)]#features-gui
  - [Platform Support]#platform-support
  - [Example Workflow]#example-workflow
- [CLI usage]#cli-usage
  - [Format inference for `--out`]#format-inference-for---out
  - [Examples]#examples
- [Library usage]#library-usage
  - [Add to `Cargo.toml`]#add-to-cargotoml
  - [Fetch data]#fetch-data
  - [Export data (atomic CSV/JSON)]#export-data-atomic-csvjson
  - [Compute grouped summaries]#compute-grouped-summaries
  - [Plot charts]#plot-charts
  - [Data model]#data-model
- [Data formats]#data-formats
  - [CSV]#csv
  - [JSON]#json
- [Security & reliability]#security--reliability
- [Testing]#testing
- [Contributing]#contributing
- [License]#license
- [Disclaimer]#disclaimer

## Features

### What it can do

- **Retrieve data** from the World Bank by country/countries, indicator(s), and optional date range.
- **Multi-indicator requests** work without specifying a World Bank `source`; the client transparently fans out per indicator when `--source` is omitted, while still supporting the single-call path when `--source` is provided.
- **Show short stats in the terminal** (grouped min / max / mean / median per (indicator, country)).
- **Export datasets** to **CSV** or **JSON** (format inferred from `--out` extension or set via `--format`).  
  Exports are **atomic** and CSV is **spreadsheet-safe**.
- **Export plots** as **SVG** or **PNG** (backend inferred from `--plot` file extension).
- **Country-consistent styling**: when enabled via `--country-styles`, series from the same country share consistent base colors while indicators are differentiated by shades and patterns.

### Under the hood

- **Hardened HTTPS client** (rustls, connect/request timeouts, limited redirects, descriptive `User-Agent`: `wbi_rs`).
- **Robust URL handling**
- **Transient error resilience** (small retry/backoff) and **page caps**.
- **Valid JSON** under all inputs (`NaN`/`±∞``null`).
- **Numerical stability** (non-finite values treated as missing; safe float sorting).
- **Portable plotting** with an embedded TTF font for CI/headless environments.

---

## Install

Download a prebuilt binary (GitHub Releases):

1) Go to **GitHub → Releases**: [https://github.com/ArdentEmpiricist/wbi-rs/releases/latest](https://github.com/ArdentEmpiricist/wbi-rs/releases/latest)
2) Download the binary for your platform

Via cargo/crates.io:

```bash
# Install both CLI and GUI binaries (default)
cargo install wbi-rs
wbi --help    # CLI binary
wbi-gui       # GUI binary

# Install CLI only
cargo install wbi-rs --bin wbi
wbi --help

# Install GUI only  
cargo install wbi-rs --bin wbi-gui
wbi-gui
```

From source:

```bash
git clone <this-repo>
cd wbi-rs

# Build both CLI and GUI binaries (default)
cargo build --release

# Build CLI only
cargo build --release --bin wbi

# Build GUI only
cargo build --release --bin wbi-gui
```

As a library (example):

```toml
[dependencies]
wbi-rs = "0.1" # or your path
anyhow = "1"
```

---

## Quick start (CLI)

Fetch population for Germany & France (2000–2023), write to CSV (format inferred from extension) and show quick stats in terminal:

```bash
wbi get \
  --countries DEU,FRA \
  --indicators SP.POP.TOTL \
  --date 2000:2023 \
  --out pop.csv \
  --stats
```

Output:
<p align="center">
  <img src="https://github.com/ArdentEmpiricist/wbi-rs/blob/main/assets/example_stats.png?raw=true" alt="example terminal output" style='width: 90%; object-fit: contain'/>
</p>

Fetch GDP (in current US$) for the US, China, Germany and India and render a plot (backend inferred from extension):

```bash
wbi get \
  --countries USA,CHN,DEU,IND \
  --indicators NY.GDP.MKTP.CD \
  --date 1970:2025 \
  --plot pop.svg \
  --plot-kind line-points
```

Output:
<p align="center">
  <img src="https://raw.githubusercontent.com/ArdentEmpiricist/wbi-rs/bf42ecfa7f9a4a559b886aa5bf48d397fb41233d/assets/example_plot.svg" alt="example plot" style='width: 90%; object-fit: contain'/>
</p>

---

### Country-Consistent Styling

You can enable country-consistent styling where all series for the same country share one base hue from the MS Office palette, while indicators within that country are differentiated by shades:

```bash
# Use country-consistent styling
wbi get \
  --countries USA,CHN,IND,DEU \
  --indicators SP.POP.TOTL,SP.POP.TOTL.FE.IN \
  --date 2010:2020 \
  --plot multi_indicator.svg \
  --plot-kind line-points \
  --country-styles
```

This feature ensures that:

- All series from the same country use the same base color hue
- Different indicators within a country are differentiated by brightness variations
- The styling is deterministic and consistent across runs

Output:
<p align="center">
  <img src="https://raw.githubusercontent.com/ArdentEmpiricist/wbi-rs/refs/heads/main/assets/example_multi_indicator.svg" alt="example plot" style='width: 90%; object-fit: contain'/>
</p>

---

## GUI Application

For users who prefer a graphical interface, `wbi-rs` includes a modern, cross-platform desktop application.

<p align="center">
  <img src="https://raw.githubusercontent.com/ArdentEmpiricist/wbi-rs/refs/heads/main/assets/gui.png" alt="gui example" style='width: 90%; object-fit: contain'/>
</p>

### Installation and Usage

Build and run the GUI:

```bash
cargo build --release --bin wbi-gui
cargo run --release --bin wbi-gui
```

### Features-GUI

The GUI provides an intuitive interface for:

- **Data Selection**: Easy input fields for countries and indicators
- **Date Range**: Visual controls for selecting start and end years
- **Export Options**: Choose CSV, JSON, or both formats
- **Chart Creation**: Generate professional PNG or SVG charts
- **Advanced Configuration**: Access all CLI features through collapsible advanced options
- **File Management**: Native file browser integration with home directory defaults
- **Real-time Feedback**: Progress indicators and clear error messages

> Advanced options for plots only visible if "Create chart" checkbox is active.

### Platform Support

The GUI application works on:

- **Windows**: Native look and feel
- **macOS**: Cocoa interface
- **Linux**: X11 and Wayland support

### Example Workflow

1. Enter countries: `USA,DEU,CHN`
2. Enter indicators: `SP.POP.TOTL,NY.GDP.MKTP.CD`
3. Set date range: 2010 to 2020
4. Choose export format: CSV
5. Enable chart creation: PNG format
6. Click "Fetch Data"

The application will download the data, save it to your chosen location, and create a chart—all with a single click.

---

## CLI usage

Subcommand `get` accepts at least:

- `--countries` ISO2/ISO3 codes separated by `,` or `;` (e.g., `DEU,FRA` or (attention: `" "` are required using `;`) `"DEU;FRA"`)
- `--indicators` World Bank indicator IDs (e.g., `SP.POP.TOTL`)
- `--date` optional year or range (e.g., `2020` or `2000:2023`)
- `--source` optional source ID (e.g., `2` for WDI). Recommended for efficiency when querying multiple indicators, but optional.
- `--out <PATH>` optional export (CSV/JSON); **atomic**
- `--plot <PATH>` optional chart output (SVG/PNG), using Plotters
- `--country-styles` enable country-consistent styling with optional mode (on|symbols|off)

### Format inference for `--out`

- If `--format` is **not** provided, the format is **inferred**:
  - `.csv` → CSV
  - `.json` → JSON
  - no extension → defaults to CSV
  - unknown extension (with no `--format`) → error
- If `--format` **is** provided:
  - It must **match** known extensions; conflicting combinations (e.g., `--out data.csv --format json`) **error** early
  - For unknown extensions, the explicit format wins

### Examples

```bash
# CSV via extension inference
wbi get --countries DEU --indicators SP.POP.TOTL --out data.csv

# JSON via extension inference
wbi get --countries DEU --indicators SP.POP.TOTL --out data.json

# Unknown extension allowed when --format is explicit
wbi get --countries DEU --indicators SP.POP.TOTL --out dump.xyz --format csv

# Error on conflict
wbi get --countries DEU --indicators SP.POP.TOTL --out data.csv --format json
```

---

## Library usage

The crate exposes modules for API access, models, storage, statistics, and plotting.

### Add to `Cargo.toml`

```toml
[dependencies]
wbi-rs = "0.1" # or "{ path = "." } to your source"
anyhow = "1"
```

### Fetch data

```rust
use anyhow::Result;
use wbi_rs::api::Client;
use wbi_rs::models::DateSpec;

fn main() -> Result<()> {
    // Hardened blocking client (rustls, timeouts, redirect policy, UA)
    let api = Client::default();

    // Countries & indicators can be given as lists; date is optional
    let points = api.fetch(
        &["DEU".into(), "FRA".into()],
        &["SP.POP.TOTL".into()],
        Some(DateSpec::Range { start: 2000, end: 2023 }),
        None, // source id
    )?;

    println!("rows: {}", points.len());
    Ok(())
}
```

For multi-indicator requests when `source` is `None`, the client automatically handles fallback behavior by making separate requests per indicator to ensure data retrieval succeeds.

The `fetch` method automatically enriches `DataPoint.unit` values when observation rows lack a unit by fetching metadata from the World Bank indicator endpoint. This ensures that visualization and analysis code has access to appropriate unit information for axis labeling and scaling decisions.

If you need to manually fetch indicator units for specific indicators:

```rust
let units = api.fetch_indicator_units(&["SP.POP.TOTL".into(), "NY.GDP.MKTP.CD".into()])?;
// Returns HashMap<String, String> mapping indicator ID to unit
```

### Export data (atomic CSV/JSON)

```rust
use wbi_rs::storage::{save_csv, save_json};

save_csv(&points, "pop.csv")?;   // spreadsheet-safe + atomic
save_json(&points, "pop.json")?; // non-finite -> null + atomic
```

- **CSV**: fixed header order; cells beginning with `=`, `+`, `-`, `@` are prefixed with `'`.
- **JSON**: pretty-printed; non-finite floats are serialized as `null`.

Both writers use a tempfile in the destination directory and atomically replace the target file.

### Compute grouped summaries

```rust
use wbi_rs::stats::{grouped_summary, Summary};

let summaries: Vec<Summary> = grouped_summary(&points);
// Summary contains: key (indicator_id, country_iso3), count, missing, min, max, mean, median.
// Non-finite values are counted as missing; sorting avoids panics on floats.
```

### Plot charts

```rust
use wbi_rs::viz::plot_chart;

// `plot_chart` filters non-finite values and sorts by integer year.
// The backend is selected from the output extension (.svg, .png).
plot_chart(&points, "pop.svg")?;
```

Charts automatically derive appropriate units for axis labeling using a two-tier approach:

1. **Prefer units from DataPoint.unit**: When all points have the same non-empty unit, use it directly for axis labeling
2. **Fallback to indicator name parsing**: For single-indicator plots without consistent units, extract unit information from parentheses in the indicator name (e.g., "GDP (current US$)" → "current US$")

This approach ensures that both API-provided units and legacy indicator naming conventions are properly handled for visualization.

### Data model

```rust
// Simplified view
pub struct DataPoint {
    pub indicator_id: String,
    pub indicator_name: String,
    pub country_id: String,
    pub country_name: String,
    pub country_iso3: String,
    pub year: i32,
    pub value: Option<f64>,   // may be None for missing
    pub unit: Option<String>,
    pub obs_status: Option<String>,
    pub decimal: Option<i64>,
}
```

---

## Data formats

### CSV

- **Header:** `indicator_id, indicator_name, country_id, country_name, country_iso3, year, value, unit, obs_status, decimal`
- **Quoting/escaping:** handled by the `csv` crate (RFC-4180)
- **Missing values:** `None` → empty cell
- **Safety:** cells beginning with `=`, `+`, `-`, `@` are prefixed with `'` (prevents formula execution)

### JSON

- **Shape:** array of objects mirroring the CSV fields
- **Numbers:** non-finite floats serialized as `null`
- **Formatting:** pretty-printed for readability

---

## Security & reliability

- **Networking**
  - TLS via `rustls`
  - Request + connect timeouts, limited redirects
  - Descriptive `User-Agent`
  - Percent-encoding for user-supplied path segments
  - Small retry/backoff on transient failures
  - Hard cap on pages to avoid runaway jobs
- **Exports**
  - **Atomic writes** for CSV/JSON
  - **CSV formula guard**
  - **Valid JSON** under all numeric inputs
- **Numerics**
  - Non-finite values filtered/treated as missing
  - Safe sorting; integer year ordering for plots
- **Rendering**
  - Embedded font registration avoids “FontUnavailable” in headless/CI

---

## Testing

Run all tests:

```bash
# use without --feature online to test local only
cargo test --features online
```

Coverage includes:

- CSV formula guard (prefix `'`)
- Output format logic in `cmd_get` (inference, explicit flags, conflicts)
- Basic numeric guards (non-finite handling)

---

## Contributing

Contributions are welcome! Please follow these guidelines:

1. **Discuss major changes first**: open an issue to align on scope/design before large features or public API changes.
2. **Keep PRs focused**: small, single-purpose PRs are easier to review and merge.
3. **Code quality**: ensure all of the following pass locally or supply arguments why some parts do not pass:

   ```bash
   cargo fmt --all
   cargo clippy --all-targets --all-features -- -D warnings
   cargo test --all -- --nocapture
   ```

4. **Commit messages**: using [Conventional Commits]https://www.conventionalcommits.org/ is appreciated (e.g., `feat:`, `fix:`, `docs:`).
5. **License**: by contributing, you agree your changes are dual MIT and Apache-2.-licensed.

---

## License

Licensed under either of

- [Apache License, Version 2.0]https://www.apache.org/licenses/LICENSE-2.0.txt
- [MIT license]LICENSE

at your option.

Any contribution intentionally submitted for inclusion in this work shall be
dual licensed as above, without any additional terms or conditions.

## Disclaimer

This project is an independent, community-maintained library and CLI. It is not affiliated with, endorsed by, or sponsored by The World Bank Group or any of its institutions. “The World Bank” name and any related trademarks are the property of The World Bank Group and are used here solely for identification purposes.

This software accesses publicly available data via the World Bank Indicators API. Your use of any World Bank data is governed by the World Bank’s terms and applicable data licenses. No warranty is provided; use at your own risk.

- World Bank Indicators API: <https://datahelpdesk.worldbank.org/knowledgebase/articles/889392>
- World Bank Terms of Use: <https://www.worldbank.org/terms>
- World Bank Data Terms (incl. licensing): <https://datacatalog.worldbank.org/terms-and-conditions>