Thread states and the global interpreter lock

Unless on a free-threaded build of CPython, the Python interpreter is generally not thread-safe. In order to support multi-threaded Python programs, there’s a global lock, called the global interpreter lock or GIL, that must be held by a thread before accessing Python objects. Without the lock, even the simplest operations could cause problems in a multi-threaded program: for example, when two threads simultaneously increment the reference count of the same object, the reference count could end up being incremented only once instead of twice.

As such, only a thread that holds the GIL may operate on Python objects or invoke Python’s C API.

In order to emulate concurrency, the interpreter regularly tries to switch threads between bytecode instructions (see sys.setswitchinterval()). This is why locks are also necessary for thread-safety in pure-Python code.

Additionally, the global interpreter lock is released around blocking I/O operations, such as reading or writing to a file. From the C API, this is done by detaching the thread state.

The Python interpreter keeps some thread-local information inside a data structure called PyThreadState, known as a thread state. Each thread has a thread-local pointer to a PyThreadState; a thread state referenced by this pointer is considered to be attached.

A thread can only have one attached thread state at a time. An attached thread state is typically analogous with holding the GIL, except on free-threaded builds. On builds with the GIL enabled, attaching a thread state will block until the GIL can be acquired. However, even on builds with the GIL disabled, it is still required to have an attached thread state, as the interpreter needs to keep track of which threads may access Python objects.

Note

Even on the free-threaded build, attaching a thread state may block, as the GIL can be re-enabled or threads might be temporarily suspended (such as during a garbage collection).

Generally, there will always be an attached thread state when using Python’s C API, including during embedding and when implementing methods, so it’s uncommon to need to set up a thread state on your own. Only in some specific cases, such as in a Py_BEGIN_ALLOW_THREADS block or in a fresh thread, will the thread not have an attached thread state. If uncertain, check if PyThreadState_GetUnchecked() returns NULL.

If it turns out that you do need to create a thread state, it is recommended to use PyThreadState_Ensure() or PyThreadState_EnsureFromView(), which will manage the thread state for you.

Detaching the thread state from extension code

Most extension code manipulating the thread state has the following simple structure:

Save the thread state in a local variable.
... Do some blocking I/O operation ...
Restore the thread state from the local variable.

This is so common that a pair of macros exists to simplify it:

Py_BEGIN_ALLOW_THREADS
... Do some blocking I/O operation ...
Py_END_ALLOW_THREADS

The Py_BEGIN_ALLOW_THREADS macro opens a new block and declares a hidden local variable; the Py_END_ALLOW_THREADS macro closes the block.

The block above expands to the following code:

PyThreadState *_save;

_save = PyEval_SaveThread();
... Do some blocking I/O operation ...
PyEval_RestoreThread(_save);

Here is how these functions work:

The attached thread state implies that the GIL is held for the interpreter. To detach it, PyEval_SaveThread() is called and the result is stored in a local variable.

By detaching the thread state, the GIL is released, which allows other threads to attach to the interpreter and execute while the current thread performs blocking I/O. When the I/O operation is complete, the old thread state is reattached by calling PyEval_RestoreThread(), which will wait until the GIL can be acquired.

Note

Performing blocking I/O is the most common use case for detaching the thread state, but it is also useful to call it over long-running native code that doesn’t need access to Python objects or Python’s C API. For example, the standard zlib and hashlib modules detach the thread state when compressing or hashing data.

On a free-threaded build, the GIL is usually out of the question, but detaching the thread state is still required, because the interpreter periodically needs to block all threads to get a consistent view of Python objects without the risk of race conditions. For example, CPython currently suspends all threads for a short period of time while running the garbage collector.

Warning

Detaching the thread state can lead to unexpected behavior during interpreter finalization. See Cautions regarding interpreter finalization for more details.

APIs

The following macros are normally used without a trailing semicolon; look for example usage in the Python source distribution.

Note

These macros are still necessary on the free-threaded build to prevent deadlocks.

Py_BEGIN_ALLOW_THREADS

Part of the Stable ABI.

This macro expands to { PyThreadState *_save; _save = PyEval_SaveThread();. Note that it contains an opening brace; it must be matched with a following Py_END_ALLOW_THREADS macro. See above for further discussion of this macro.

Py_END_ALLOW_THREADS

Part of the Stable ABI.

This macro expands to PyEval_RestoreThread(_save); }. Note that it contains a closing brace; it must be matched with an earlier Py_BEGIN_ALLOW_THREADS macro. See above for further discussion of this macro.

Py_BLOCK_THREADS

Part of the Stable ABI.

This macro expands to PyEval_RestoreThread(_save);: it is equivalent to Py_END_ALLOW_THREADS without the closing brace.

Py_UNBLOCK_THREADS

Part of the Stable ABI.

This macro expands to _save = PyEval_SaveThread();: it is equivalent to Py_BEGIN_ALLOW_THREADS without the opening brace and variable declaration.

Using the C API from foreign threads

When threads are created using the dedicated Python APIs (such as the threading module), a thread state is automatically associated with them, However, when a thread is created from native code (for example, by a third-party library with its own thread management), it doesn’t hold an attached thread state.

If you need to call Python code from these threads (often this will be part of a callback API provided by the aforementioned third-party library), you must first register these threads with the interpreter by creating a new thread state and attaching it.

The easiest way to do this is through PyThreadState_Ensure() or PyThreadState_EnsureFromView().

Note

These functions require an argument pointing to the desired interpreter; such a pointer can be acquired via a call to PyInterpreterGuard_FromCurrent() (for PyThreadState_Ensure) or PyInterpreterView_FromCurrent() (for PyThreadState_EnsureFromView) from the function that creates the thread. If no pointer is available (such as when the given native thread library doesn’t provide a data argument), PyInterpreterView_FromMain() can be used to get a view for the main interpreter, but note that this will make the code incompatible with subinterpreters.

For example:

// The return value of PyInterpreterGuard_FromCurrent() from the
// function that created this thread.
PyInterpreterGuard *guard = thread_data->guard;

// Create a new thread state for the interpreter.
PyThreadStateToken *token = PyThreadState_Ensure(guard);
if (token == NULL) {
   PyInterpreterGuard_Close(guard);
   return;
}

// We have a valid thread state -- perform Python actions here.
result = CallSomeFunction();
// Evaluate result or handle exceptions.

// Release the thread state. No calls to the C API are allowed beyond this
// point.
PyThreadState_Release(token);
PyInterpreterGuard_Close(guard);

Keep in mind that calling PyThreadState_Ensure might not always create a new thread state, and calling PyThreadState_Release might not always detach it. These functions may reuse an existing attached thread state, or may re-attach a thread state that was previously attached for the current thread.

See also

PEP 788

Attaching/detaching thread states

PyThreadStateToken *PyThreadState_Ensure(PyInterpreterGuard *guard)

Part of the Stable ABI since version 3.15.

Ensure that the thread has an attached thread state for the interpreter protected by guard, and thus can safely invoke that interpreter.

It is OK to call this function if the thread already has an attached thread state, as long as there is a subsequent call to PyThreadState_Release() that matches this one (meaning that “nested” calls to this function are permitted).

The function’s effect (if any) will be reversed by the matching call to PyThreadState_Release().

On error, this function returns NULL without an exception set. Do not call PyThreadState_Release() in this case.

On success, this function returns a pointer value that must be passed to the matching call to PyThreadState_Release().

The conditions in which this function creates a new thread state are considered unstable and implementation-dependent. If you need to control the exact lifetime of a thread state, consider using PyThreadState_New(). However, do not avoid this function solely on the basis that the lifetime of the thread state may be inconsistent across versions; changes to this function will be done with caution and in a backwards-compatible manner. In particular, the saving of thread-local variables and similar state will be retained across Python versions.

CPython implementation detail: The exact behavior of whether this function creates a new thread state is described below, but be aware that this may change in the future.

First, this function checks if an attached thread state is present. If there is, this function then checks if the interpreter of that thread state matches the interpreter guarded by guard. If that is the case, this function simply marks the thread state as being used by a PyThreadState_Ensure call and returns.

If there is no attached thread state, then this function checks if any thread state has been used by the current OS thread. (This is returned by PyGILState_GetThisThreadState().) If there was, then this function checks if that thread state’s interpreter matches guard. If it does, it is re-attached and marked as used.

Otherwise, if both of the above cases fail, a new thread state is created for guard. It is then attached and marked as owned by PyThreadState_Ensure.

Added in version 3.15.

PyThreadStateToken *PyThreadState_EnsureFromView(PyInterpreterView *view)

Part of the Stable ABI since version 3.15.

Get an attached thread state for the interpreter referenced by view.

The behavior and return value are the same as for PyThreadState_Ensure(); additionally, if the function succeeds, the interpreter referenced by view will be implicitly guarded. The guard will be released upon the corresponding PyThreadState_Release() call.

Added in version 3.15.

void PyThreadState_Release(PyThreadStateToken *token)

Part of the Stable ABI since version 3.15.

Undo a PyThreadState_Ensure() or PyThreadState_EnsureFromView() call.

This must be called exactly once for each successful Ensure call, with token set to that call’s return value.

The state that was attached before the corresponding Ensure call (if any) will be attached when PyThreadState_Release() returns.

The exact behavior of whether this function deletes a thread state is considered unstable and implementation-dependent.

CPython implementation detail: Currently, this function will decrement an internal counter on the attached thread state. If this counter ever reaches below zero, this function emits a fatal error (via Py_FatalError()).

If the attached thread state is owned by PyThreadState_Ensure, then the attached thread state will be deallocated and deleted upon the internal counter reaching zero. Otherwise, nothing happens when the counter reaches zero.

Added in version 3.15.

type PyThreadStateToken

Part of the Stable ABI (as an opaque struct) since version 3.15.

An opaque token retrieved from a PyThreadState_Ensure() call and passed to a corresponding PyThreadState_Release() call.

GIL-state APIs

The following APIs are generally not compatible with subinterpreters and will hang the process during interpreter finalization (see Cautions regarding interpreter finalization). As such, these APIs were soft deprecated in Python 3.15 in favor of the new APIs.

type PyGILState_STATE

Part of the Stable ABI.

The type of the value returned by PyGILState_Ensure() and passed to PyGILState_Release().

enumerator PyGILState_LOCKED

The GIL was already held when PyGILState_Ensure() was called.

enumerator PyGILState_UNLOCKED

The GIL was not held when PyGILState_Ensure() was called.