tarantool-test 0.3.1

CLI test framework for tarantool-based Rust applications
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144

# tarantool-test

`tarantool-test` is a CLI test framework, written for tarantool-based Rust applications.

Powered by [tarantool-runner](https://git.picodata.io/picodata/picodata/tarantool-runner), it eliminates huge boilerplate associated with functional tests.

You have a tarantool-oriented Rust application, and you have tests that depend on tarantool symbols and you find yourself writing your own test suite? Then you've found needed utility. All you need is to write business-logic of your test-suite, bind it through `tarantool-test` macro and use supplied CLI-application.

Quick glance at CLI usage:

```bash
# use default entrypoint:
tarantool-test -p ./target/debug/mypackage.so

# use custom entrypoint(for example, for secondary test suite):
tarantool-test -p ./target/debug/mypackage.so -e custom_entrypoint
```

If you want a quick steps to setup tests at your own crate, read [installation](#installation) and [tutorial](#tutorial) sections.

For more detailed explanation, see [details](#details) and [features](#features) sections.

## Installation

`tarantool-test` consists of two parts, merged into one crate:

1) library, which contains utilities you must use in order to prepare your own crate for tests;
2) binary utility which you install and run just like `cargo test`

You can install binary utility from `crates.io`: `cargo install --features="bin" tarantool-test`. Take a note that we activated feature `bin` in order to install the binary. **Do not** activate this feature when you use `tarantool-test` as a library.

You can also install binary utility directly from the repository:

```bash
git clone https://git.picodata.io/picodata/picodata/tarantool-test
cd tarantool-test
cargo install --features="bin" --path tarantool-test
```

In order to add library part to your project, you simply use `tarantool-test` as a dependency: `cargo add tarantool-test`

## Tutorial

You can see an example of usage in `Makefile`, target `run-example` executes test suite against `example` crate, which can be found in `example` subdir.

Now you want to use `tarantool-test` with your own crate. Following steps will use default test suite:

1. Add `tarantool-test` to your `Cargo.toml` as an ordinary dependency: `cargo add tarantool-test`;
2. Ensure your package could be used by the `tarantool-test`. Include those lines in the `Cargo.toml`:

```toml
[lib]
crate-type = ["lib", "cdylib"]
```

3. Mark desired tests with `#[tarantool_test::test]` attribute macro. This would allow `tarantool-test` to collect and run them;
4. Bind test suite you want to use by using `bind_test_suite` macro - add this line to `lib.rs`: `tarantool_test::bind_test_suite!()`
5. Build your crate: `cargo build -p example`. Check `./target/debug` directory to find out the actual path of shared object you've built. It would be named something like `libexample.so` for Linux or `libexample.dylib` for Macos.
6. Launch your tests by running: `tarantool-test -p ./debug/tarantool/libexample.so`. Tests would be executed, and you are done! No need to cleanup anything.

If you got confused by some of the points above, check `example` crate or read more details below.

## Features

### Multiple test suites

You are not limited to the single or default test suite - you can use as many custom suites as you want, or even bind same test suite to the many entrypoints. The important thing is that `tarantool-test` binary can only run single entrypoint - so if you would like to run many entrypoints, you'll have to use `tarantool-test` multiple times, at least for now.

In order to introduce new test suite, you implement `TestSuite` trait for some type. Then you decide which entrypoint you'd like to bind it to, for example `new_test_entrypoint`, and bind test suite to it. Then your setup would look like this:

```rust
struct MyTestSuite;

impl tarantool_test::TestSuite for MyTestSuite {
    fn before_all() {}
    fn after_all() {}
    fn before_each() {}
    fn after_each() {}
}

bind_test_suite!(new_test_entrypoint, MyTestSuite);
```

Then you run your setup by specifying concrete entrypoint you've used: `tarantool-test -p ./debug/target/myproject.so -e new_test_entrypoint`.

You can also omit entrypoint name for your custom test suite while binding, then it would be bind to the default entrypoint, which is named "default_test_entrypoint": `bind_test_suite!(MyTestSuite)`

You can omit test suite type reference, then the default test suite would be bind to the specified entrypoint: `bind_test_suite!(new_test_entrypoint)`.

Take a note, that default test suite is implemented for `()`, so the line above would have the same effect as this one: `bind_test_suite!(new_test_entrypoint, ())`.

*NOTE*: You can't use the same entrypoint twice. For example, this setup fails:

```rust
// DO NOT COMPILE!
bind_test_suite!(first_test_entrypoint);
bind_test_suite!(first_test_entrypoint);
```

..But you surely can reuse test suites many times! For example, this setup works:

```rust
bind_test_suite!(first_test_entrypoint, MyTestSuite);
bind_test_suite!(second_test_entrypoint, MyTestSuite);
```

### Override tarantool-runner args

If you want, you can override some of the arguments that are passed to `tarantool-runner`. Just supply them after `--`.

For example, you've written your own tarantool initialization script and want `tarantool-test` to use it. You can forward it as an argument to `tarantool-runner` like this:
`tarantool-test -p my_package.so -e my_entrypoint -- -i my_init_script.lua`

However, there are limitations: you can't override `-p` and `-e` arguments and override input(which goes after `--`). So these usages are **NOT** allowed:

1. `tarantool-test -p my_package.so -e my_entrypoint -- -e entrypoint`
1. `tarantool-test -p my_package.so -e my_entrypoint -- -p package`
1. `tarantool-test -p my_package.so -e my_entrypoint -- -- "my input"`

## Details

### Problem and solution

**Problem**: `tarantool-module`, which is actively used to develop tarantool applications, depends on tarantool symbols as it is, essentially, a wrapper over tarantool API. Tarantool symbols are supplied only if an application is loaded via tarantool runtime. Therefore, in order to run tests that depend on `tarantool-module`, we have to develop some tarantool runtime and run test suite there. As the number of projects grows, such boilerplate setups migrate from one project to another, keeping bugs and undocumented behavior.

**Solution**: `tarantool-test` introduces CLI application that runs test suite in the tarantool environment. The only boilerplate you need to write is a few lines of code to setup your test suite(or use the default one if you ain't interested in `before_all` or any other callbacks).

### Implementation

`tarantool-test` is backed by [tarantool-runner](https://git.picodata.io/picodata/picodata/tarantool-runner). It executes test setup in the provided by runner tarantool environment, so anything related to `tarantool-runner` is applicable here.
*NOTE*: you don't need to install `tarantool-runner` yourself - it is builtin.

Currently, `tarantool-test` highly depends on the [tarantool-module](https://git.picodata.io/picodata/picodata/tarantool-module). Namely, `tarantool_test::test` macro is just a reexport of `tarantool::test`, which might change in the future.
Despite the fact that marking tests with `tarantool::test` would also work, it is **discouraged**, as `tarantool_test::test` would someday be separated to include special features.

What `tarantool-test` does: whenever you use `bind_tarantool_test!` macro, it generates tarantool procedure with needed entrypoint name. Newly created entrypoint deserializes input and launches tests with specified test suite.

Binary utility simply wraps `tarantool-runner`, introducing cleaner interface.

## Contributing

If you have question, problem or suggestion, feel free to reach me directly at `f.telnov@picodata.io` or at [telegram](https://t.me/ftelnov).

You can also join Picodata chat at [telegram](https://t.me/picodataru) and tag me with your question here as well. We'll be glad to see you there!