pyo3_asyncio/
testing.rs

1//! # PyO3 Asyncio Testing Utilities
2//!
3//! This module provides some utilities for parsing test arguments as well as running and filtering
4//! a sequence of tests.
5//!
6//! As mentioned [here](crate#pythons-event-loop), PyO3 Asyncio tests cannot use the default test
7//! harness since it doesn't allow Python to gain control over the main thread. Instead, we have to
8//! provide our own test harness in order to create integration tests.
9//!
10//! Running `pyo3-asyncio` code in doc tests _is_ supported however since each doc test has its own
11//! `main` function. When writing doc tests, you may use the
12//! [`#[pyo3_asyncio::async_std::main]`](crate::async_std::main) or
13//! [`#[pyo3_asyncio::tokio::main]`](crate::tokio::main) macros on the test's main function to run
14//! your test.
15//!
16//! If you don't want to write doc tests, you're unfortunately stuck with integration tests since
17//! lib tests do not offer the same level of flexibility for the `main` fn. That being said,
18//! overriding the default test harness can be quite different from what you're used to doing for
19//! integration tests, so these next sections will walk you through this process.
20//!
21//! ## Main Test File
22//! First, we need to create the test's main file. Although these tests are considered integration
23//! tests, we cannot put them in the `tests` directory since that is a special directory owned by
24//! Cargo. Instead, we put our tests in a `pytests` directory.
25//!
26//! > The name `pytests` is just a convention. You can name this folder anything you want in your own
27//! > projects.
28//!
29//! We'll also want to provide the test's main function. Most of the functionality that the test harness needs is packed in the [`pyo3_asyncio::testing::main`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/testing/fn.main.html) function. This function will parse the test's CLI arguments, collect and pass the functions marked with [`#[pyo3_asyncio::async_std::test]`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/attr.test.html) or [`#[pyo3_asyncio::tokio::test]`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/tokio/attr.test.html) and pass them into the test harness for running and filtering.
30//!
31//! `pytests/test_example.rs` for the `tokio` runtime:
32//! ```rust
33//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
34//! #[pyo3_asyncio::tokio::main]
35//! async fn main() -> pyo3::PyResult<()> {
36//!     pyo3_asyncio::testing::main().await
37//! }
38//! # #[cfg(not(all(feature = "tokio-runtime", feature = "attributes")))]
39//! # fn main() {}
40//! ```
41//!
42//! `pytests/test_example.rs` for the `async-std` runtime:
43//! ```rust
44//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
45//! #[pyo3_asyncio::async_std::main]
46//! async fn main() -> pyo3::PyResult<()> {
47//!     pyo3_asyncio::testing::main().await
48//! }
49//! # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
50//! # fn main() {}
51//! ```
52//!
53//! ## Cargo Configuration
54//! Next, we need to add our test file to the Cargo manifest by adding the following section to the
55//! `Cargo.toml`
56//!
57//! ```toml
58//! [[test]]
59//! name = "test_example"
60//! path = "pytests/test_example.rs"
61//! harness = false
62//! ```
63//!
64//! Also add the `testing` and `attributes` features to the `pyo3-asyncio` dependency and select your preferred runtime:
65//!
66//! ```toml
67//! pyo3-asyncio = { version = "0.13", features = ["testing", "attributes", "async-std-runtime"] }
68//! ```
69//!
70//! At this point, you should be able to run the test via `cargo test`
71//!
72//! ### Adding Tests to the PyO3 Asyncio Test Harness
73//!
74//! We can add tests anywhere in the test crate with the runtime's corresponding `#[test]` attribute:
75//!
76//! For `async-std` use the [`pyo3_asyncio::async_std::test`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/attr.test.html) attribute:
77//! ```rust
78//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
79//! mod tests {
80//!     use std::{time::Duration, thread};
81//!
82//!     use pyo3::prelude::*;
83//!
84//!     // tests can be async
85//!     #[pyo3_asyncio::async_std::test]
86//!     async fn test_async_sleep() -> PyResult<()> {
87//!         async_std::task::sleep(Duration::from_secs(1)).await;
88//!         Ok(())
89//!     }
90//!
91//!     // they can also be synchronous
92//!     #[pyo3_asyncio::async_std::test]
93//!     fn test_blocking_sleep() -> PyResult<()> {
94//!         thread::sleep(Duration::from_secs(1));
95//!         Ok(())
96//!     }
97//! }
98//!
99//! # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
100//! #[pyo3_asyncio::async_std::main]
101//! async fn main() -> pyo3::PyResult<()> {
102//!     pyo3_asyncio::testing::main().await
103//! }
104//! # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
105//! # fn main() {}
106//! ```
107//!
108//! For `tokio` use the [`pyo3_asyncio::tokio::test`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/tokio/attr.test.html) attribute:
109//! ```rust
110//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
111//! mod tests {
112//!     use std::{time::Duration, thread};
113//!
114//!     use pyo3::prelude::*;
115//!
116//!     // tests can be async
117//!     #[pyo3_asyncio::tokio::test]
118//!     async fn test_async_sleep() -> PyResult<()> {
119//!         tokio::time::sleep(Duration::from_secs(1)).await;
120//!         Ok(())
121//!     }
122//!
123//!     // they can also be synchronous
124//!     #[pyo3_asyncio::tokio::test]
125//!     fn test_blocking_sleep() -> PyResult<()> {
126//!         thread::sleep(Duration::from_secs(1));
127//!         Ok(())
128//!     }
129//! }
130//!
131//! # #[cfg(all(feature = "tokio-runtime", feature = "attributes"))]
132//! #[pyo3_asyncio::tokio::main]
133//! async fn main() -> pyo3::PyResult<()> {
134//!     pyo3_asyncio::testing::main().await
135//! }
136//! # #[cfg(not(all(feature = "tokio-runtime", feature = "attributes")))]
137//! # fn main() {}
138//! ```
139//!
140//! ## Lib Tests
141//!
142//! Unfortunately, as we mentioned at the beginning, these utilities will only run in integration
143//! tests and doc tests. Running lib tests are out of the question since we need control over the
144//! main function. You can however perform compilation checks for lib tests. This is much more
145//! useful in doc tests than it is for lib tests, but the option is there if you want it.
146//!
147//! `my-crate/src/lib.rs`
148//! ```
149//! # #[cfg(all(
150//! #     any(feature = "async-std-runtime", feature = "tokio-runtime"),
151//! #     feature = "attributes"
152//! # ))]
153//! mod tests {
154//!     use pyo3::prelude::*;
155//!
156//! #   #[cfg(feature = "async-std-runtime")]
157//!     #[pyo3_asyncio::async_std::test]
158//!     async fn test_async_std_async_test_compiles() -> PyResult<()> {
159//!         Ok(())
160//!     }
161//! #   #[cfg(feature = "async-std-runtime")]
162//!     #[pyo3_asyncio::async_std::test]
163//!     fn test_async_std_sync_test_compiles() -> PyResult<()> {
164//!         Ok(())
165//!     }
166//!
167//! #   #[cfg(feature = "tokio-runtime")]
168//!     #[pyo3_asyncio::tokio::test]
169//!     async fn test_tokio_async_test_compiles() -> PyResult<()> {
170//!         Ok(())
171//!     }
172//! #   #[cfg(feature = "tokio-runtime")]
173//!     #[pyo3_asyncio::tokio::test]
174//!     fn test_tokio_sync_test_compiles() -> PyResult<()> {
175//!         Ok(())
176//!     }
177//! }
178//!
179//! # fn main() {}
180//! ```
181
182use std::{future::Future, pin::Pin};
183
184use clap::{Arg, Command};
185use futures::stream::{self, StreamExt};
186use pyo3::prelude::*;
187
188/// Args that should be provided to the test program
189///
190/// These args are meant to mirror the default test harness's args.
191/// > Currently only `--filter` is supported.
192pub struct Args {
193    filter: Option<String>,
194}
195
196impl Default for Args {
197    fn default() -> Self {
198        Self { filter: None }
199    }
200}
201
202/// Parse the test args from the command line
203///
204/// This should be called at the start of your test harness to give the CLI some
205/// control over how our tests are run.
206///
207/// Ideally, we should mirror the default test harness's arguments exactly, but
208/// for the sake of simplicity, only filtering is supported for now. If you want
209/// more features, feel free to request them
210/// [here](https://github.com/awestlake87/pyo3-asyncio/issues).
211///
212/// # Examples
213///
214/// Running the following function:
215/// ```
216/// # use pyo3_asyncio::testing::parse_args;
217/// let args = parse_args();
218/// ```
219///
220/// Produces the following usage string:
221///
222/// ```bash
223/// Pyo3 Asyncio Test Suite
224/// USAGE:
225/// test_example [TESTNAME]
226///
227/// FLAGS:
228/// -h, --help       Prints help information
229/// -V, --version    Prints version information
230///
231/// ARGS:
232/// <TESTNAME>    If specified, only run tests containing this string in their names
233/// ```
234pub fn parse_args() -> Args {
235    let matches = Command::new("PyO3 Asyncio Test Suite")
236        .arg(
237            Arg::new("TESTNAME")
238                .help("If specified, only run tests containing this string in their names"),
239        )
240        .get_matches();
241
242    Args {
243        filter: matches
244            .get_one::<String>("TESTNAME")
245            .map(|name| name.clone()),
246    }
247}
248
249type TestFn = dyn Fn() -> Pin<Box<dyn Future<Output = PyResult<()>> + Send>> + Send + Sync;
250
251/// The structure used by the `#[test]` macros to provide a test to the `pyo3-asyncio` test harness.
252#[derive(Clone)]
253pub struct Test {
254    /// The fully qualified name of the test
255    pub name: &'static str,
256    /// The function used to create the task that runs the test.
257    pub test_fn: &'static TestFn,
258}
259
260impl Test {
261    /// Create the task that runs the test
262    pub fn task(
263        &self,
264    ) -> std::pin::Pin<Box<dyn std::future::Future<Output = pyo3::PyResult<()>> + Send>> {
265        (self.test_fn)()
266    }
267}
268
269inventory::collect!(Test);
270
271/// Run a sequence of tests while applying any necessary filtering from the `Args`
272pub async fn test_harness(tests: Vec<Test>, args: Args) -> PyResult<()> {
273    stream::iter(tests)
274        .for_each_concurrent(Some(4), |test| {
275            let mut ignore = false;
276
277            if let Some(filter) = args.filter.as_ref() {
278                if !test.name.contains(filter) {
279                    ignore = true;
280                }
281            }
282
283            async move {
284                if !ignore {
285                    test.task().await.unwrap();
286
287                    println!("test {} ... ok", test.name);
288                }
289            }
290        })
291        .await;
292
293    Ok(())
294}
295
296/// Parses test arguments and passes the tests to the `pyo3-asyncio` test harness
297///
298/// This function collects the test structures from the `inventory` boilerplate and forwards them to
299/// the test harness.
300///
301/// # Examples
302///
303/// ```
304/// # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
305/// use pyo3::prelude::*;
306///
307/// # #[cfg(all(feature = "async-std-runtime", feature = "attributes"))]
308/// #[pyo3_asyncio::async_std::main]
309/// async fn main() -> PyResult<()> {
310///     pyo3_asyncio::testing::main().await
311/// }
312/// # #[cfg(not(all(feature = "async-std-runtime", feature = "attributes")))]
313/// # fn main() { }
314/// ```
315pub async fn main() -> PyResult<()> {
316    let args = parse_args();
317
318    test_harness(
319        inventory::iter::<Test>().map(|test| test.clone()).collect(),
320        args,
321    )
322    .await
323}
324
325#[cfg(test)]
326#[cfg(all(
327    feature = "testing",
328    feature = "attributes",
329    any(feature = "async-std-runtime", feature = "tokio-runtime")
330))]
331mod tests {
332    use pyo3::prelude::*;
333
334    use crate as pyo3_asyncio;
335
336    #[cfg(feature = "async-std-runtime")]
337    #[pyo3_asyncio::async_std::test]
338    async fn test_async_std_async_test_compiles() -> PyResult<()> {
339        Ok(())
340    }
341    #[cfg(feature = "async-std-runtime")]
342    #[pyo3_asyncio::async_std::test]
343    fn test_async_std_sync_test_compiles() -> PyResult<()> {
344        Ok(())
345    }
346
347    #[cfg(feature = "tokio-runtime")]
348    #[pyo3_asyncio::tokio::test]
349    async fn test_tokio_async_test_compiles() -> PyResult<()> {
350        Ok(())
351    }
352    #[cfg(feature = "tokio-runtime")]
353    #[pyo3_asyncio::tokio::test]
354    fn test_tokio_sync_test_compiles() -> PyResult<()> {
355        Ok(())
356    }
357}