pyo3_asyncio/testing.rs
1//! # PyO3 Asyncio Testing Utilities
2//!
3//! This module provides some utilities for parsing test arguments as well as running and filtering
4//! a sequence of tests.
5//!
6//! As mentioned [here](crate#pythons-event-loop), PyO3 Asyncio tests cannot use the default test
7//! harness since it doesn't allow Python to gain control over the main thread. Instead, we have to
8//! provide our own test harness in order to create integration tests.
9//!
10//! Running `pyo3-asyncio` code in doc tests _is_ supported however since each doc test has its own
11//! `main` function. When writing doc tests, you may use the
12//! [`#[pyo3_asyncio::async_std::main]`](crate::async_std::main) or
13//! [`#[pyo3_asyncio::tokio::main]`](crate::tokio::main) macros on the test's main function to run
14//! your test.
15//!
16//! If you don't want to write doc tests, you're unfortunately stuck with integration tests since
17//! lib tests do not offer the same level of flexibility for the `main` fn. That being said,
18//! overriding the default test harness can be quite different from what you're used to doing for
19//! integration tests, so these next sections will walk you through this process.
20//!
21//! ## Main Test File
22//! First, we need to create the test's main file. Although these tests are considered integration
23//! tests, we cannot put them in the `tests` directory since that is a special directory owned by
24//! Cargo. Instead, we put our tests in a `pytests` directory.
25//!
26//! > The name `pytests` is just a convention. You can name this folder anything you want in your own
27//! > projects.
28//!
29//! We'll also want to provide the test's main function. Most of the functionality that the test harness needs is packed in the [`pyo3_asyncio::testing::main`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/testing/fn.main.html) function. This function will parse the test's CLI arguments, collect and pass the functions marked with [`#[pyo3_asyncio::async_std::test]`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/attr.test.html) or [`#[pyo3_asyncio::tokio::test]`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/tokio/attr.test.html) and pass them into the test harness for running and filtering.
30//!
31//! `pytests/test_example.rs` for the `tokio` runtime:
32//! ```rust
33//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
34//! #[pyo3_asyncio::tokio::main]
35//! async fn main() -> pyo3::PyResult<()> {
36//! pyo3_asyncio::testing::main().await
37//! }
38//! # #[cfg(not(all(feature = "tokio-runtime", feature = "attributes")))]
39//! # fn main() {}
40//! ```
41//!
42//! `pytests/test_example.rs` for the `async-std` runtime:
43//! ```rust
44//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
45//! #[pyo3_asyncio::async_std::main]
46//! async fn main() -> pyo3::PyResult<()> {
47//! pyo3_asyncio::testing::main().await
48//! }
49//! # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
50//! # fn main() {}
51//! ```
52//!
53//! ## Cargo Configuration
54//! Next, we need to add our test file to the Cargo manifest by adding the following section to the
55//! `Cargo.toml`
56//!
57//! ```toml
58//! [[test]]
59//! name = "test_example"
60//! path = "pytests/test_example.rs"
61//! harness = false
62//! ```
63//!
64//! Also add the `testing` and `attributes` features to the `pyo3-asyncio` dependency and select your preferred runtime:
65//!
66//! ```toml
67//! pyo3-asyncio = { version = "0.13", features = ["testing", "attributes", "async-std-runtime"] }
68//! ```
69//!
70//! At this point, you should be able to run the test via `cargo test`
71//!
72//! ### Adding Tests to the PyO3 Asyncio Test Harness
73//!
74//! We can add tests anywhere in the test crate with the runtime's corresponding `#[test]` attribute:
75//!
76//! For `async-std` use the [`pyo3_asyncio::async_std::test`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/attr.test.html) attribute:
77//! ```rust
78//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
79//! mod tests {
80//! use std::{time::Duration, thread};
81//!
82//! use pyo3::prelude::*;
83//!
84//! // tests can be async
85//! #[pyo3_asyncio::async_std::test]
86//! async fn test_async_sleep() -> PyResult<()> {
87//! async_std::task::sleep(Duration::from_secs(1)).await;
88//! Ok(())
89//! }
90//!
91//! // they can also be synchronous
92//! #[pyo3_asyncio::async_std::test]
93//! fn test_blocking_sleep() -> PyResult<()> {
94//! thread::sleep(Duration::from_secs(1));
95//! Ok(())
96//! }
97//! }
98//!
99//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
100//! #[pyo3_asyncio::async_std::main]
101//! async fn main() -> pyo3::PyResult<()> {
102//! pyo3_asyncio::testing::main().await
103//! }
104//! # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
105//! # fn main() {}
106//! ```
107//!
108//! For `tokio` use the [`pyo3_asyncio::tokio::test`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/tokio/attr.test.html) attribute:
109//! ```rust
110//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
111//! mod tests {
112//! use std::{time::Duration, thread};
113//!
114//! use pyo3::prelude::*;
115//!
116//! // tests can be async
117//! #[pyo3_asyncio::tokio::test]
118//! async fn test_async_sleep() -> PyResult<()> {
119//! tokio::time::sleep(Duration::from_secs(1)).await;
120//! Ok(())
121//! }
122//!
123//! // they can also be synchronous
124//! #[pyo3_asyncio::tokio::test]
125//! fn test_blocking_sleep() -> PyResult<()> {
126//! thread::sleep(Duration::from_secs(1));
127//! Ok(())
128//! }
129//! }
130//!
131//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
132//! #[pyo3_asyncio::tokio::main]
133//! async fn main() -> pyo3::PyResult<()> {
134//! pyo3_asyncio::testing::main().await
135//! }
136//! # #[cfg(not(all(feature = "tokio-runtime", feature = "attributes")))]
137//! # fn main() {}
138//! ```
139//!
140//! ## Lib Tests
141//!
142//! Unfortunately, as we mentioned at the beginning, these utilities will only run in integration
143//! tests and doc tests. Running lib tests are out of the question since we need control over the
144//! main function. You can however perform compilation checks for lib tests. This is much more
145//! useful in doc tests than it is for lib tests, but the option is there if you want it.
146//!
147//! `my-crate/src/lib.rs`
148//! ```
149//! # #[cfg(all(
150//! # any(feature = "async-std-runtime", feature = "tokio-runtime"),
151//! # feature = "attributes"
152//! # ))]
153//! mod tests {
154//! use pyo3::prelude::*;
155//!
156//! # #[cfg(feature = "async-std-runtime")]
157//! #[pyo3_asyncio::async_std::test]
158//! async fn test_async_std_async_test_compiles() -> PyResult<()> {
159//! Ok(())
160//! }
161//! # #[cfg(feature = "async-std-runtime")]
162//! #[pyo3_asyncio::async_std::test]
163//! fn test_async_std_sync_test_compiles() -> PyResult<()> {
164//! Ok(())
165//! }
166//!
167//! # #[cfg(feature = "tokio-runtime")]
168//! #[pyo3_asyncio::tokio::test]
169//! async fn test_tokio_async_test_compiles() -> PyResult<()> {
170//! Ok(())
171//! }
172//! # #[cfg(feature = "tokio-runtime")]
173//! #[pyo3_asyncio::tokio::test]
174//! fn test_tokio_sync_test_compiles() -> PyResult<()> {
175//! Ok(())
176//! }
177//! }
178//!
179//! # fn main() {}
180//! ```
181
182use std::{future::Future, pin::Pin};
183
184use clap::{Arg, Command};
185use futures::stream::{self, StreamExt};
186use pyo3::prelude::*;
187
188/// Args that should be provided to the test program
189///
190/// These args are meant to mirror the default test harness's args.
191/// > Currently only `--filter` is supported.
192pub struct Args {
193 filter: Option<String>,
194}
195
196impl Default for Args {
197 fn default() -> Self {
198 Self { filter: None }
199 }
200}
201
202/// Parse the test args from the command line
203///
204/// This should be called at the start of your test harness to give the CLI some
205/// control over how our tests are run.
206///
207/// Ideally, we should mirror the default test harness's arguments exactly, but
208/// for the sake of simplicity, only filtering is supported for now. If you want
209/// more features, feel free to request them
210/// [here](https://github.com/awestlake87/pyo3-asyncio/issues).
211///
212/// # Examples
213///
214/// Running the following function:
215/// ```
216/// # use pyo3_asyncio::testing::parse_args;
217/// let args = parse_args();
218/// ```
219///
220/// Produces the following usage string:
221///
222/// ```bash
223/// Pyo3 Asyncio Test Suite
224/// USAGE:
225/// test_example [TESTNAME]
226///
227/// FLAGS:
228/// -h, --help Prints help information
229/// -V, --version Prints version information
230///
231/// ARGS:
232/// <TESTNAME> If specified, only run tests containing this string in their names
233/// ```
234pub fn parse_args() -> Args {
235 let matches = Command::new("PyO3 Asyncio Test Suite")
236 .arg(
237 Arg::new("TESTNAME")
238 .help("If specified, only run tests containing this string in their names"),
239 )
240 .get_matches();
241
242 Args {
243 filter: matches
244 .get_one::<String>("TESTNAME")
245 .map(|name| name.clone()),
246 }
247}
248
249type TestFn = dyn Fn() -> Pin<Box<dyn Future<Output = PyResult<()>> + Send>> + Send + Sync;
250
251/// The structure used by the `#[test]` macros to provide a test to the `pyo3-asyncio` test harness.
252#[derive(Clone)]
253pub struct Test {
254 /// The fully qualified name of the test
255 pub name: &'static str,
256 /// The function used to create the task that runs the test.
257 pub test_fn: &'static TestFn,
258}
259
260impl Test {
261 /// Create the task that runs the test
262 pub fn task(
263 &self,
264 ) -> std::pin::Pin<Box<dyn std::future::Future<Output = pyo3::PyResult<()>> + Send>> {
265 (self.test_fn)()
266 }
267}
268
269inventory::collect!(Test);
270
271/// Run a sequence of tests while applying any necessary filtering from the `Args`
272pub async fn test_harness(tests: Vec<Test>, args: Args) -> PyResult<()> {
273 stream::iter(tests)
274 .for_each_concurrent(Some(4), |test| {
275 let mut ignore = false;
276
277 if let Some(filter) = args.filter.as_ref() {
278 if !test.name.contains(filter) {
279 ignore = true;
280 }
281 }
282
283 async move {
284 if !ignore {
285 test.task().await.unwrap();
286
287 println!("test {} ... ok", test.name);
288 }
289 }
290 })
291 .await;
292
293 Ok(())
294}
295
296/// Parses test arguments and passes the tests to the `pyo3-asyncio` test harness
297///
298/// This function collects the test structures from the `inventory` boilerplate and forwards them to
299/// the test harness.
300///
301/// # Examples
302///
303/// ```
304/// # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
305/// use pyo3::prelude::*;
306///
307/// # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
308/// #[pyo3_asyncio::async_std::main]
309/// async fn main() -> PyResult<()> {
310/// pyo3_asyncio::testing::main().await
311/// }
312/// # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
313/// # fn main() { }
314/// ```
315pub async fn main() -> PyResult<()> {
316 let args = parse_args();
317
318 test_harness(
319 inventory::iter::<Test>().map(|test| test.clone()).collect(),
320 args,
321 )
322 .await
323}
324
325#[cfg(test)]
326#[cfg(all(
327 feature = "testing",
328 feature = "attributes",
329 any(feature = "async-std-runtime", feature = "tokio-runtime")
330))]
331mod tests {
332 use pyo3::prelude::*;
333
334 use crate as pyo3_asyncio;
335
336 #[cfg(feature = "async-std-runtime")]
337 #[pyo3_asyncio::async_std::test]
338 async fn test_async_std_async_test_compiles() -> PyResult<()> {
339 Ok(())
340 }
341 #[cfg(feature = "async-std-runtime")]
342 #[pyo3_asyncio::async_std::test]
343 fn test_async_std_sync_test_compiles() -> PyResult<()> {
344 Ok(())
345 }
346
347 #[cfg(feature = "tokio-runtime")]
348 #[pyo3_asyncio::tokio::test]
349 async fn test_tokio_async_test_compiles() -> PyResult<()> {
350 Ok(())
351 }
352 #[cfg(feature = "tokio-runtime")]
353 #[pyo3_asyncio::tokio::test]
354 fn test_tokio_sync_test_compiles() -> PyResult<()> {
355 Ok(())
356 }
357}