thrust-rl 0.4.0

High-performance reinforcement learning in Rust with the Burn tensor backend
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
# ๐Ÿค Contributing to Thrust

Thank you for your interest in contributing to Thrust! This document provides guidelines and information for contributors.

---

## ๐ŸŒŸ Ways to Contribute

### ๐Ÿ› Report Bugs
Found a bug? Please [open an issue](https://github.com/yourusername/thrust/issues/new) with:
- Clear description of the problem
- Steps to reproduce
- Expected vs actual behavior
- System information (OS, Rust version, GPU)
- Relevant logs or error messages

### ๐Ÿ’ก Suggest Features
Have an idea? We'd love to hear it! Open an issue with:
- Description of the feature
- Use case / motivation
- Proposed API (if applicable)
- Implementation notes (optional)

### ๐Ÿ“ Improve Documentation
Documentation improvements are always welcome:
- Fix typos or unclear explanations
- Add examples or tutorials
- Improve API documentation
- Create diagrams or visualizations

### ๐Ÿ”ง Write Code
Ready to contribute code? Great! See below for the development workflow.

---

## ๐Ÿš€ Getting Started

### Prerequisites
- **Rust nightly:** Install via [rustup]https://rustup.rs/; the
  workspace pins a nightly toolchain in `rust-toolchain.toml`.
- **Git:** For version control.

No external system libraries are required โ€” the default `training`
feature uses Burn's pure-Rust NdArray backend. Optional GPU backends
(wgpu, cuda) are documented in `docs/BURN_BACKENDS.md`.
  ```bash
  # On Ubuntu
  sudo apt install nvidia-cuda-toolkit
  ```

### Setup
1. **Fork the repository**
   ```bash
   # Via GitHub UI, then:
   git clone https://github.com/YOUR_USERNAME/thrust.git
   cd thrust
   ```

2. **Install dependencies**
   ```bash
   cargo build
   ```

3. **Run tests**
   ```bash
   cargo test
   ```

4. **Check code style**
   ```bash
   cargo fmt --check
   cargo clippy -- -D warnings
   ```

---

## ๐Ÿ”„ Development Workflow

### 1. Create a Branch
```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/issue-number-description
```

Branch naming conventions:
- `feature/` - New features
- `fix/` - Bug fixes
- `docs/` - Documentation changes
- `perf/` - Performance improvements
- `refactor/` - Code refactoring

### 2. Make Changes
- Write clear, self-documenting code
- Follow Rust conventions and idioms
- Add tests for new functionality
- Update documentation as needed
- Keep commits focused and atomic

### 3. Test Your Changes
```bash
# Run all tests
cargo test

# Run specific test
cargo test test_name

# Run with output
cargo test -- --nocapture

# Run benchmarks (when available)
cargo bench
```

### 4. Format and Lint
```bash
# Format code
cargo fmt

# Run clippy
cargo clippy --all-targets --all-features -- -D warnings

# Check documentation
cargo doc --no-deps --open
```

### 5. Commit Changes
Write clear commit messages following [Conventional Commits](https://www.conventionalcommits.org/):

```bash
# Format: <type>(<scope>): <description>

git commit -m "feat(env): add CartPole environment"
git commit -m "fix(buffer): resolve memory leak in rollout storage"
git commit -m "docs(readme): add installation instructions"
git commit -m "perf(train): optimize minibatch sampling"
```

Types:
- `feat` - New feature
- `fix` - Bug fix
- `docs` - Documentation
- `perf` - Performance improvement
- `refactor` - Code refactoring
- `test` - Adding tests
- `chore` - Maintenance tasks

### 6. Push and Create PR
```bash
git push origin your-branch-name
```

Then create a Pull Request on GitHub with:
- Clear title and description
- Link to related issues
- Screenshots/videos (for UI changes)
- Checklist of changes
- Notes for reviewers

---

## ๐Ÿ“‹ Pull Request Checklist

Before submitting, ensure:
- [ ] Code compiles without errors
- [ ] All tests pass
- [ ] New tests added for new functionality
- [ ] Code is formatted (`cargo fmt`)
- [ ] No clippy warnings (`cargo clippy`)
- [ ] Documentation is updated
- [ ] Commit messages are clear
- [ ] PR description is complete
- [ ] Changes are focused and minimal

---

## ๐ŸŽจ Code Style Guidelines

### General Rust Style
- Follow [Rust API Guidelines]https://rust-lang.github.io/api-guidelines/
- Use `rustfmt` for consistent formatting
- Prefer explicit types in public APIs
- Use descriptive variable names
- Add comments for complex logic

### Documentation
```rust
/// Brief description of function
///
/// More detailed explanation if needed. Can include examples:
///
/// # Examples
///
/// ```
/// use thrust_rl::prelude::*;
/// let env = CartPoleEnv::new();
/// ```
///
/// # Errors
///
/// Returns `Err` if...
///
/// # Panics
///
/// Panics if...
pub fn example_function() -> Result<()> {
    // Implementation
}
```

### Error Handling
- Use `Result<T, E>` for recoverable errors
- Use `anyhow::Result` for application code
- Use `thiserror` for library errors
- Document error conditions

### Testing
```rust
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_something() {
        let result = function_under_test();
        assert_eq!(result, expected_value);
    }

    #[test]
    #[should_panic(expected = "specific error message")]
    fn test_panic_case() {
        // Test code that should panic
    }
}
```

---

## ๐Ÿ—๏ธ Project Structure

```
thrust/
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ lib.rs           # Main library entry
โ”‚   โ”œโ”€โ”€ env/             # Environment implementations
โ”‚   โ”‚   โ”œโ”€โ”€ mod.rs       # Environment trait
โ”‚   โ”‚   โ”œโ”€โ”€ cartpole.rs  # CartPole environment
โ”‚   โ”‚   โ””โ”€โ”€ snake.rs     # Snake environment
โ”‚   โ”œโ”€โ”€ policy/          # Neural network policies
โ”‚   โ”‚   โ”œโ”€โ”€ mod.rs       # Policy trait
โ”‚   โ”‚   โ””โ”€โ”€ mlp.rs       # MLP policy
โ”‚   โ”œโ”€โ”€ buffer/          # Experience buffers
โ”‚   โ”‚   โ”œโ”€โ”€ mod.rs       # Buffer trait
โ”‚   โ”‚   โ””โ”€โ”€ rollout.rs   # Rollout buffer
โ”‚   โ”œโ”€โ”€ train/           # Training algorithms
โ”‚   โ”‚   โ”œโ”€โ”€ mod.rs       # Trainer trait
โ”‚   โ”‚   โ””โ”€โ”€ ppo.rs       # PPO implementation
โ”‚   โ””โ”€โ”€ utils/           # Utilities
โ”‚       โ”œโ”€โ”€ mod.rs
โ”‚       โ””โ”€โ”€ logging.rs
โ”œโ”€โ”€ examples/            # Example programs
โ”œโ”€โ”€ benches/            # Benchmarks
โ”œโ”€โ”€ tests/              # Integration tests
โ””โ”€โ”€ docs/               # Additional documentation
```

---

## ๐ŸŽฏ Current Priorities

See [ROADMAP.md](ROADMAP.md) for the current roadmap and
[CHANGELOG.md](CHANGELOG.md) for what has already shipped. As of v0.3.0 the
foundation phases are complete; active work is multi-agent communication
(Phase 3), multi-host distributed training, and mixed-precision support.

### Help Wanted
Check issues labeled `good-first-issue` or `help-wanted` for specific tasks.

---

## ๐Ÿงช Testing Guidelines

### Unit Tests
- Test individual functions and methods
- Mock external dependencies
- Cover edge cases and error conditions
- Keep tests fast (<1s each)

### Integration Tests
- Test complete workflows
- Use real environments and policies
- Verify end-to-end functionality
- Can be slower but should complete in <30s

### Benchmarks
```rust
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn benchmark_function(c: &mut Criterion) {
    c.bench_function("function_name", |b| {
        b.iter(|| {
            // Code to benchmark
            black_box(expensive_function())
        })
    });
}

criterion_group!(benches, benchmark_function);
criterion_main!(benches);
```

---

## ๐Ÿ› Debugging Tips

### Enable Logging
```bash
RUST_LOG=debug cargo run
RUST_LOG=thrust_rl=trace cargo run
```

### Use rust-gdb or rust-lldb
```bash
rust-gdb target/debug/thrust-rl
rust-lldb target/debug/thrust-rl
```

### Profiling
```bash
# CPU profiling
cargo flamegraph --bin your-binary

# Memory profiling
cargo valgrind --bin your-binary
```

---

## ๐Ÿ“š Resources

### Rust Learning
- [The Rust Book]https://doc.rust-lang.org/book/
- [Rust by Example]https://doc.rust-lang.org/rust-by-example/
- [Rustlings]https://github.com/rust-lang/rustlings - Interactive exercises

### RL Background
- [Spinning Up in Deep RL]https://spinningup.openai.com/
- [Sutton & Barto Book]http://incompleteideas.net/book/the-book.html
- [PPO Paper]https://arxiv.org/abs/1707.06347

### Related Projects
- [PufferLib]https://github.com/PufferAI/PufferLib - Python RL library
- [Burn]https://burn.dev - Pure-Rust deep learning framework (Thrust's tensor backend)
- [Border]https://github.com/laboroai/border - Rust RL library
- [tch-rs]https://github.com/LaurentMazare/tch-rs - PyTorch bindings (Thrust's pre-v0.1.0 backend; since replaced by Burn)

---

## ๐Ÿ’ฌ Communication

### GitHub Discussions
Use for:
- General questions
- Feature discussions
- Architecture decisions
- Showcase your projects

### Issues
Use for:
- Bug reports
- Feature requests
- Specific technical problems

### Discord (Coming Soon)
- Real-time chat
- Community support
- Development updates

---

## ๐Ÿ“œ Code of Conduct

We are committed to providing a welcoming and inclusive environment for everyone. Please be:
- **Respectful:** Treat others with respect and consideration
- **Constructive:** Provide helpful, actionable feedback
- **Collaborative:** Work together toward common goals
- **Patient:** Remember that everyone was a beginner once

Unacceptable behavior includes:
- Harassment, discrimination, or personal attacks
- Trolling or deliberate disruption
- Publishing others' private information
- Other conduct inappropriate in a professional setting

Violations may result in temporary or permanent ban from the project.

---

## ๐ŸŽ‰ Recognition

Contributors will be:
- Credited in release notes
- Mentioned in blog posts (for major contributions)
- Given credit in relevant documentation

---

## โ“ Questions?

Not sure where to start? Have questions?
- Check existing [issues]https://github.com/yourusername/thrust/issues and [discussions]https://github.com/yourusername/thrust/discussions
- Open a new discussion
- Join our Discord (coming soon!)

---

**Thank you for contributing to Thrust! ๐Ÿš€**

*Built with ๐Ÿฆ€ Rust and โค๏ธ for reinforcement learning*