Struct pyo3::Python[−][src]

pub struct Python<'p>(_);
Expand description

Marker type that indicates that the GIL is currently held.

The Python struct is a zero-sized marker struct that is required for most Python operations. This is used to indicate that the operation accesses/modifies the Python interpreter state, and thus can only be called if the Python interpreter is initialized and the Python global interpreter lock (GIL) is acquired. The lifetime 'p represents the lifetime of holding the lock.

Note that the GIL can be temporarily released by the Python interpreter during a function call (e.g. importing a module), even when you’re holding a GILGuard. In general, you don’t need to worry about this because the GIL is reacquired before returning to the Rust code:

`Python` exists   |=====================================|
GIL actually held |==========|         |================|
Rust code running |=======|                |==|  |======|

This behaviour can cause deadlocks when trying to lock a Rust mutex while holding the GIL:

  • Thread 1 acquires the GIL
  • Thread 1 locks a mutex
  • Thread 1 makes a call into the Python interpreter which releases the GIL
  • Thread 2 acquires the GIL
  • Thread 2 tries to locks the mutex, blocks
  • Thread 1’s Python interpreter call blocks trying to reacquire the GIL held by thread 2

To avoid deadlocking, you should release the GIL before trying to lock a mutex, e.g. with Python::allow_threads.

Implementations

Acquires the global interpreter lock, which allows access to the Python runtime. The provided closure F will be executed with the acquired Python marker token.

If the auto-initialize feature is enabled and the Python runtime is not already initialized, this function will initialize it. See prepare_freethreaded_python() for details.

Panics

  • If the auto-initialize feature is not enabled and the Python interpreter is not initialized.

Examples

use pyo3::prelude::*;
Python::with_gil(|py| -> PyResult<()> {
    let x: i32 = py.eval("5", None, None)?.extract()?;
    assert_eq!(x, 5);
    Ok(())
});

Like Python::with_gil except Python interpreter state checking is skipped.

Normally when the GIL is acquired, we check that the Python interpreter is an appropriate state (e.g. it is fully initialized). This function skips those checks.

Safety

If Python::with_gil would succeed, it is safe to call this function.

In most cases, you should use Python::with_gil.

A justified scenario for calling this function is during multi-phase interpreter initialization when Python::with_gil would fail before _Py_InitializeMain() is called because the interpreter is only partially initialized.

Behavior in other scenarios is not documented.

Acquires the global interpreter lock, which allows access to the Python runtime.

If the auto-initialize feature is enabled and the Python runtime is not already initialized, this function will initialize it. See prepare_freethreaded_python() for details.

Most users should not need to use this API directly, and should prefer one of two options:

  1. When implementing #[pymethods] or #[pyfunction] add a function argument py: Python to receive access to the GIL context in which the function is running.
  2. Use Python::with_gil to run a closure with the GIL, acquiring only if needed.

Note: This return type from this function, GILGuard, is implemented as a RAII guard around the C-API Python_EnsureGIL. This means that multiple acquire_gil() calls are allowed, and will not deadlock. However, GILGuards must be dropped in the reverse order to acquisition. If PyO3 detects this order is not maintained, it may be forced to begin an irrecoverable panic.

Panics

  • If the auto-initialize feature is not enabled and the Python interpreter is not initialized.

Temporarily releases the GIL, thus allowing other Python threads to run.

Examples

use pyo3::exceptions::PyRuntimeError;
use std::sync::Arc;
use std::thread;
#[pyfunction]
fn parallel_count(py: Python<'_>, strings: Vec<String>, query: String) -> PyResult<usize> {
    let query = query.chars().next().unwrap();
    py.allow_threads(move || {
        let threads: Vec<_> = strings
            .into_iter()
            .map(|s| thread::spawn(move || s.chars().filter(|&c| c == query).count()))
            .collect();
        let mut sum = 0;
        for t in threads {
            sum += t.join().map_err(|_| PyRuntimeError::new_err(()))?;
        }
        Ok(sum)
    })
}

Python::with_gil(|py| {
    let m = PyModule::new(py, "pcount").unwrap();
    m.add_function(wrap_pyfunction!(parallel_count, m).unwrap()).unwrap();
    let locals = [("pcount", m)].into_py_dict(py);
    pyo3::py_run!(py, *locals, r#"
        s = ["Flow", "my", "tears", "the", "Policeman", "Said"]
        assert pcount.parallel_count(s, "a") == 3
    "#);
});

Note: PyO3 types that represent objects with a lifetime tied to holding the GIL cannot be used in the closure. This includes &PyAny and all the concrete-typed siblings, like &PyString.

This is achieved via the Send bound on the closure and the return type. This is slightly more restrictive than necessary, but it’s the most fitting solution available in stable Rust. In the future this bound may be relaxed by a new “auto-trait”, if auto-traits become a stable feature of the Rust language.

You can convert such references to e.g. PyObject or Py<PyString>, which makes them independent of the GIL lifetime. However, you cannot do much with those without a Python<'p> token, for which you’d need to reacquire the GIL.

Examples

ⓘ
fn parallel_print(py: Python<'_>) {
    let s = PyString::new(py, "This object should not be shared >_<");
    py.allow_threads(move || {
        println!("{:?}", s); // This causes a compile error.
    });
}

Evaluates a Python expression in the given context and returns the result.

If globals is None, it defaults to Python module __main__. If locals is None, it defaults to the value of globals.

Examples

let result = py.eval("[i * 10 for i in range(5)]", None, None).unwrap();
let res: Vec<i64> = result.extract().unwrap();
assert_eq!(res, vec![0, 10, 20, 30, 40])

Executes one or more Python statements in the given context.

If globals is None, it defaults to Python module __main__. If locals is None, it defaults to the value of globals.

Examples

use pyo3::{types::{PyBytes, PyDict}, prelude::*};
Python::with_gil(|py| {
    let locals = PyDict::new(py);
    py.run(
        r#"
import base64
s = 'Hello Rust!'
ret = base64.b64encode(s.encode('utf-8'))
"#,
      None,
      Some(locals),
    )
    .unwrap();
    let ret = locals.get_item("ret").unwrap();
    let b64: &PyBytes = ret.downcast().unwrap();
    assert_eq!(b64.as_bytes(), b"SGVsbG8gUnVzdCE=");
});

You can use py_run! for a handy alternative of run if you don’t need globals and unwrapping is OK.

Gets the Python type object for type T.

Imports the Python module with the specified name.

Gets the Python builtin value None.

Gets the Python builtin value NotImplemented.

Gets the running Python interpreter version as a string.

This is a wrapper around the ffi call Py_GetVersion.

Examples

Python::with_gil(|py| {
    // The full string could be, for example:
    // "3.0a5+ (py3k:63103M, May 12 2008, 00:53:55) \n[GCC 4.2.3]"
    assert!(py.version().starts_with("3."));
});

Gets the running Python interpreter version as a struct similar to sys.version_info.

Examples

Python::with_gil(|py| {
    // PyO3 supports Python 3.6 and up.
    assert!(py.version_info() >= (3, 6));
    assert!(py.version_info() >= (3, 6, 0));
});

Registers the object in the release pool, and tries to downcast to specific type.

Registers the object in the release pool, and does an unchecked downcast to the specific type.

Safety

Callers must ensure that ensure that the cast is valid.

Registers the object pointer in the release pool, and does an unchecked downcast to the specific type.

Safety

Callers must ensure that ensure that the cast is valid.

Registers the owned object pointer in the release pool.

Returns Err(PyErr) if the pointer is NULL. Does an unchecked downcast to the specific type.

Safety

Callers must ensure that ensure that the cast is valid.

Registers the owned object pointer in release pool.

Returns None if the pointer is NULL. Does an unchecked downcast to the specific type.

Safety

Callers must ensure that ensure that the cast is valid.

Does an unchecked downcast to the specific type.

Panics if the pointer is NULL.

Safety

Callers must ensure that ensure that the cast is valid.

Does an unchecked downcast to the specific type.

Returns Err(PyErr) if the pointer is NULL.

Safety

Callers must ensure that ensure that the cast is valid.

Does an unchecked downcast to the specific type.

Returns None if the pointer is NULL.

Safety

Callers must ensure that ensure that the cast is valid.

Lets the Python interpreter check and handle any pending signals. This will invoke the corresponding signal handlers registered in Python (if any).

Returns Err(PyErr) if any signal handler raises an exception.

These signals include SIGINT (normally raised by CTRL + C), which by default raises KeyboardInterrupt. For this reason it is good practice to call this function regularly as part of long-running Rust functions so that users can cancel it.

Example

use pyo3::prelude::*;
#[pyfunction]
fn loop_forever(py: Python) -> PyResult<()> {
    loop {
        // As this loop is infinite it should check for signals every once in a while.
        // Using `?` causes any `PyErr` (potentially containing `KeyboardInterrupt`) to break out of the loop.
        py.check_signals()?;

        // do work here
    }
}

Note

This function calls PyErr_CheckSignals() which in turn may call signal handlers. As Python’s signal API allows users to define custom signal handlers, calling this function allows arbitary Python code inside signal handlers to run.

Retrieves a Python instance under the assumption that the GIL is already acquired at this point, and stays acquired for the lifetime 'p.

Because the output lifetime 'p is not connected to any input parameter, care must be taken that the compiler infers an appropriate lifetime for 'p when calling this function.

Safety

The lifetime 'p must be shorter than the period you assume that you have GIL. I.e., Python<'static> is always really unsafe.

Create a new pool for managing PyO3’s owned references.

When this GILPool is dropped, all PyO3 owned references created after this GILPool will all have their Python reference counts decremented, potentially allowing Python to drop the corresponding Python objects.

Typical usage of PyO3 will not need this API, as Python::acquire_gil automatically creates a GILPool where appropriate.

Advanced uses of PyO3 which perform long-running tasks which never free the GIL may need to use this API to clear memory, as PyO3 usually does not clear memory until the GIL is released.

Examples

Python::with_gil(|py| {
    // Some long-running process like a webserver, which never releases the GIL.
    loop {
        // Create a new pool, so that PyO3 can clear memory at the end of the loop.
        let pool = unsafe { py.new_pool() };

        // It is recommended to *always* immediately set py to the pool's Python, to help
        // avoid creating references with invalid lifetimes.
        let py = unsafe { pool.python() };

        // do stuff...
    }
});

Safety

Extreme care must be taken when using this API, as misuse can lead to accessing invalid memory. In addition, the caller is responsible for guaranteeing that the GIL remains held for the entire lifetime of the returned GILPool.

Two best practices are required when using this API:

  • From the moment new_pool() is called, only the Python token from the returned GILPool (accessible using .python()) should be used in PyO3 APIs. All other older Python tokens with longer lifetimes are unsafe to use until the GILPool is dropped, because they can be used to create PyO3 owned references which have lifetimes which outlive the GILPool.
  • Similarly, methods on existing owned references will implicitly refer back to the Python token which that reference was originally created with. If the returned values from these methods are owned references they will inherit the same lifetime. As a result, Rust’s lifetime rules may allow them to outlive the GILPool, even though this is not safe for reasons discussed above. Care must be taken to never access these return values after the GILPool is dropped, unless they are converted to Py<T> before the pool is dropped.

Trait Implementations

Returns a copy of the value. Read more

Performs copy-assignment from source. Read more

Auto Trait Implementations

Blanket Implementations

Gets the TypeId of self. Read more

Immutably borrows from an owned value. Read more

Mutably borrows from an owned value. Read more

Performs the conversion.

Performs the conversion.

The resulting type after obtaining ownership.

Creates owned data from borrowed data, usually by cloning. Read more

🔬 This is a nightly-only experimental API. (toowned_clone_into)

recently added

Uses borrowed data to replace owned data, usually by cloning. Read more

The type returned in the event of a conversion error.

Performs the conversion.

The type returned in the event of a conversion error.

Performs the conversion.