The Complete Guide to Concurrency and Parallelism in Python: The Evolution from One Thread to Multiple Cores
First, Answer a Fundamental Question: Why Does Python Have So Many Concurrency Options?
If you come from Java, you may find this confusing. Java can take the Thread + ExecutorService + CompletableFuture path all the way through, so why has Python produced threading, multiprocessing, asyncio, concurrent.futures, and even concurrent.interpreters?
The reason is that Python carries historical baggage that Java does not: the GIL (Global Interpreter Lock). This lock prevents Python threads from truly executing CPU computations in parallel, so the Python community has had to develop several approaches to work around or resolve this constraint. Each approach has an appropriate use case. They complement one another rather than replace one another.
This article begins with the lowest-level concepts and gradually explains what each approach is, why it exists, where it fits, and how the ecosystem will evolve.
Chapter 1: Concurrency and Parallelism—Two Different Things
These two terms are often used interchangeably in everyday conversation, but they have precise meanings in programming. Confusing them makes everything that follows unclear.
Concurrency
Concurrency means that multiple tasks make progress during overlapping periods of time, although they are not necessarily executing at the exact same instant.
Continuing the chef metaphor from the previous article, one chef prepares three dishes at once. The chef is not literally chopping three kinds of ingredients simultaneously. Instead, the chef finishes cutting the fish and leaves it to marinate, starts stir-frying vegetables, then uses a spare moment after putting them in the pan to prepare a sauce. All three dishes make progress during the same period, but the chef performs only one task at any instant.
The core value of concurrency is using waiting time efficiently. It is especially suitable for I/O-bound tasks such as network requests, file operations, and database queries. These tasks spend most of their time waiting for an external response, leaving the CPU largely idle.
Parallelism
Parallelism means that multiple tasks truly execute at the same instant, which requires multiple execution units, such as multiple CPU cores.
Using the kitchen metaphor again, you hire four chefs and give each one a stove. They genuinely stir-fry four dishes at the same time. This is simultaneous execution in the physical sense.
The core value of parallelism is reducing computation time. It is especially suitable for CPU-bound tasks such as data processing, scientific computing, and image rendering. These tasks involve almost no waiting and keep the CPU busy, so the only way to accelerate them is to compute on multiple cores at once.
Comparison with Java
Java’s Thread supports parallelism by nature. JVM threads are operating-system threads, so multiple threads can run on different cores and genuinely execute Java bytecode at the same time. Therefore, multithreading in Java is a tool for both concurrency and parallelism.
The GIL makes the situation much more complicated in Python. For Python code, threading can provide concurrency but not parallelism. Parallel execution requires multiprocessing or another approach. This is why Python’s concurrency toolbox is more complex than Java’s.
Chapter 2: The GIL—The Key to Understanding Every Python Concurrency Problem
The GIL is a global mutex in the CPython interpreter. Its rule is extremely simple: at any given moment, only one thread in an entire Python process may execute Python bytecode.
Why the GIL Exists
CPython uses reference counting to manage the memory lifecycle of objects. Every Python object contains a counter that records how many references point to it. When that count reaches zero, the object is released immediately.
a = [1, 2, 3] # 列表对象的引用计数 = 1
b = a # 引用计数 = 2
del a # 引用计数 = 1
del b # 引用计数 = 0 → 对象被释放
Incrementing and decrementing this reference count is not atomic. If two threads modify the same object’s reference count simultaneously—for example, thread A executes del a and decrements the count while thread B executes c = a and increments it—a race condition occurs. In the worst case, the count incorrectly reaches zero and the object is released prematurely. A thread that still holds a reference then accesses freed memory, causing a segmentation fault.
The GIL solves this problem in the most blunt way possible. If modifying reference counts is unsafe, it ensures that only one thread can run Python code at a time. Reference-count updates are therefore serialized by nature, without requiring a separate lock on every object.
Consequences of the GIL
It has almost no effect on I/O-bound tasks. When a thread performs an I/O operation such as socket.recv(), CPython actively releases the GIL before entering the system call, giving other threads a chance to run. It reacquires the GIL after the I/O completes. Consequently, multithreaded web crawlers, concurrent HTTP requests, and similar workloads behave as expected.
It is disastrous for CPU-bound tasks. When several threads all perform pure-Python computations such as arithmetic or list operations, they continually contend for the GIL and become serialized at a fine-grained level. Worse still, acquiring and releasing the GIL has its own overhead, so a multithreaded CPU-bound workload can sometimes be slower than a single-threaded one.
From a Java perspective, imagine that the JVM had one global lock and that every thread had to acquire it before executing any line of Java code. Even with ten threads running on ten cores, only one thread could execute Java code at a time. That is the effect of the GIL on Python.
An Additional Note About the GIL
One frequently overlooked detail is that the GIL protects only the execution of Python bytecode. If a C extension explicitly releases the GIL while running pure C code, that code can truly execute in parallel with other threads. This is why NumPy matrix operations can use multiple cores even when invoked from Python: NumPy performs its core computation at the C level and releases the GIL.
Chapter 3: Python’s Four Concurrency and Parallelism Options
Once you understand the GIL, each of Python’s four approaches falls naturally into place.
Option 1: threading—Cooperative Multithreading Under the GIL
The threading module provides operating-system threads. Each thread is a real OS thread scheduled by the operating system. Because of the GIL, however, only one thread can execute Python bytecode at a time.
Appropriate use case: I/O-bound concurrency. Examples include sending 100 HTTP requests concurrently, reading or writing several files concurrently, or waiting for several database queries at once. While a thread waits for I/O, it releases the GIL so another thread can run.
Inappropriate use case: CPU-bound tasks. Running pure-Python computations on multiple threads is no faster than using one thread and may even be slower.
import threading
import requests
def fetch_url(url):
response = requests.get(url)
print(f"{url}: {response.status_code}")
urls = ["https://httpbin.org/delay/1"] * 5
# 创建五个线程，每个线程发一个请求
threads = [threading.Thread(target=fetch_url, args=(url,)) for url in urls]
for t in threads:
t.start() # 启动线程
for t in threads:
t.join() # 等待所有线程完成
# 五个请求并发执行，总耗时约 1 秒而不是 5 秒
# 因为在等待 HTTP 响应时，GIL 被释放，其他线程可以运行
The key difference from Java multithreading: Java threads can genuinely execute Java code in parallel. Python’s threading provides only apparent parallelism while running Python code. Their behavior is similar, however, when handling I/O waits.
Option 2: multiprocessing—Bypass the GIL with Multiple Processes
Because the GIL is scoped to a process, the most direct way to bypass it is to start multiple processes. Each process has its own Python interpreter, its own GIL, and its own memory space. Multiple processes can truly execute Python code in parallel.
Appropriate use case: CPU-bound parallel computation. Examples include data processing, scientific computing, and batch image processing.
Cost: Processes do not share memory. Transferring data requires serialization with pickle and subsequent deserialization, which costs much more than communication between threads. Creating a process also costs more than creating a thread.
from multiprocessing import Pool
def heavy_computation(n):
"""一个 CPU 密集的计算"""
return sum(i * i for i in range(n))
# 创建一个包含 4 个 worker 进程的进程池
with Pool(4) as pool:
# 把任务分发到 4 个进程并行执行
results = pool.map(heavy_computation, [10_000_000] * 4)
# 4 个进程各自有独立的 GIL，可以真正并行计算
# 总耗时约为单进程的 1/4（在 4 核 CPU 上）
Java analogy: This is somewhat like using Java’s ProcessBuilder to start child processes, although Python’s multiprocessing provides a friendlier abstraction. Its API closely resembles threading, so the cost of switching is low.
Option 3: asyncio—Single-Threaded, Event-Driven Concurrency
asyncio is the system discussed in the previous article: one thread, an event loop, coroutines, and await. It does not inherently involve multiple threads or processes. Instead, it implements concurrency through cooperative scheduling within one thread.
Appropriate use case: large-scale I/O concurrency, especially network I/O. When you need to manage thousands or tens of thousands of network connections simultaneously, asyncio is more efficient than threading because it avoids the cost of creating and switching threads as well as contention for the GIL.
Inappropriate use case: CPU-bound tasks, unless it is combined with a thread pool or process pool.
import asyncio
import aiohttp
async def fetch_url(session, url):
async with session.get(url) as response:
return await response.text()
async def main():
async with aiohttp.ClientSession() as session:
# asyncio.gather 让多个协程并发执行
tasks = [fetch_url(session, f"https://httpbin.org/delay/1") for _ in range(100)]
results = await asyncio.gather(*tasks)
# 100 个请求并发执行，总耗时约 1-2 秒
asyncio.run(main())
Java analogy: Conceptually, asyncio resembles Java NIO (non-blocking I/O) and CompletableFuture; more precisely, it resembles Netty’s EventLoop model. Java 21 Virtual Threads from Project Loom follow a similar line of thought: large numbers of lightweight concurrency units provide efficient I/O concurrency through cooperative scheduling.
The essential difference between asyncio and threading:
threading is preemptive: the operating system decides when to switch threads, and your code can be interrupted at any position. This creates a risk of race conditions and requires locks to protect shared data.
asyncio is cooperative: control is yielded only at the points where you explicitly write await. Between two await expressions, no other coroutine can interrupt your code. This greatly reduces the cognitive burden of concurrent programming; you do not need to worry that another coroutine will intervene halfway through a line of code.
This advantage is also a limitation. If one coroutine performs a large amount of computation between two await expressions, every other coroutine must wait.
Option 4: concurrent.futures—A Unified High-Level Interface
concurrent.futures is Python’s high-level abstraction for multithreading and multiprocessing. It provides one unified Executor interface, making it easy to switch between a ThreadPoolExecutor and a ProcessPoolExecutor, sometimes by changing only a single line.
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def task(n):
return sum(i * i for i in range(n))
# I/O 密集型 → 用线程池
with ThreadPoolExecutor(max_workers=10) as executor:
futures = [executor.submit(task, 1000) for _ in range(10)]
results = [f.result() for f in futures]
# CPU 密集型 → 改一行就切换到进程池
with ProcessPoolExecutor(max_workers=4) as executor:
futures = [executor.submit(task, 10_000_000) for _ in range(4)]
results = [f.result() for f in futures]
Direct Java counterpart: concurrent.futures is Python’s version of java.util.concurrent. Python’s Executor corresponds to Java’s ExecutorService, Future corresponds to Java’s Future<T>, and ThreadPoolExecutor and ProcessPoolExecutor correspond to factory methods such as Java’s Executors.newFixedThreadPool(). If you already know Java’s ExecutorService, you can learn this module very quickly.
Chapter 4: How to Choose—A Decision Process
How should you choose an approach for a particular task? Use the following reasoning process.
Step 1: Determine whether the task is I/O-bound or CPU-bound.
I/O-bound means that a task spends most of its time waiting for external responses, such as network requests, file operations, and database queries. CPU-bound means that it spends most of its time computing, such as data processing, mathematical operations, and image processing. Many real workloads are mixed, so you need to identify the actual bottleneck.
Step 2: For I/O-bound tasks.
If concurrency is modest, such as a few dozen tasks, threading or ThreadPoolExecutor is sufficient and straightforward. If concurrency is high, such as hundreds or thousands of tasks, use asyncio, which consumes fewer resources for highly concurrent I/O. Note, however, that choosing asyncio also requires the libraries you call to support asynchronous operation. You need aiohttp instead of requests, and asyncpg instead of psycopg2. If a dependency has only a synchronous version, threading may be more practical.
Step 3: For CPU-bound tasks.
Use multiprocessing or ProcessPoolExecutor; these are currently the most mature approaches. For numerical computation, consider NumPy or Pandas first. Their core operations run at the C level and release the GIL, so they can naturally work with multiple threads.
Step 4: For mixed tasks.
You can combine the approaches. For example, request handling in a web application is I/O-bound because it waits for databases and downstream APIs, so use asyncio for the main loop. If one step performs CPU-intensive computation, send it to a thread pool with asyncio.to_thread(), or to a process pool with loop.run_in_executor(ProcessPoolExecutor(), ...).
Chapter 5: How asyncio and threading Work Together in Practice
The previous article explained the principle behind iterate_in_threadpool. Here is a more general scenario: calling synchronous blocking code from an asyncio program.
asyncio.to_thread(): The Simplest Bridge
Python 3.9 introduced asyncio.to_thread() specifically for calling synchronous functions safely from an asyncio program.
import asyncio
import time
def blocking_io():
"""模拟一个同步阻塞的 I/O 操作"""
time.sleep(2)
return "data from slow API"
async def main():
# 错误做法：直接在协程里调用阻塞函数
# result = blocking_io() # 这会冻结事件循环 2 秒！
# 正确做法：把阻塞函数丢到线程池
result = await asyncio.to_thread(blocking_io)
# 事件循环不会被阻塞，其他协程可以正常运行
asyncio.run(main())
The principle is exactly the same as iterate_in_threadpool: the blocking operation runs in a thread pool, while await waits for the thread pool to return a result without blocking the event-loop thread.
loop.run_in_executor(): The More Flexible Version
If you need to control the concrete configuration of a thread pool or process pool, or use a process pool for CPU-bound work, choose run_in_executor().
import asyncio
from concurrent.futures import ProcessPoolExecutor
def cpu_heavy(n):
"""CPU 密集任务"""
return sum(i * i for i in range(n))
async def main():
loop = asyncio.get_running_loop()
# 用进程池执行 CPU 密集任务
with ProcessPoolExecutor(max_workers=4) as pool:
result = await loop.run_in_executor(pool, cpu_heavy, 10_000_000)
print(result)
asyncio.run(main())
This combination is extremely common in production, particularly in asynchronous web frameworks such as FastAPI and Starlette.
Chapter 6: Comparing the Internal Mechanisms of the Three Approaches
To deepen your understanding, let us compare the underlying execution mechanisms.
The Execution Model of threading
When you create a threading.Thread, CPython creates a real operating-system thread through POSIX pthreads or the Windows Thread API. The operating system genuinely schedules multiple OS threads onto different CPU cores. Before a thread executes Python bytecode, however, it must acquire the GIL. The GIL is acquired and released on a rotation of roughly 5 milliseconds in implementations since Python 3.2, ensuring that each thread gets a chance to run.
From the operating system’s perspective, therefore, Python multithreading is genuine multithreading. From the perspective of Python bytecode execution, however, the GIL serializes the threads. This explains why I/O is unaffected, because the GIL is released while waiting for I/O, whereas CPU computation cannot run in parallel.
The Execution Model of multiprocessing
When you create a multiprocessing.Process, CPython forks a new operating-system process on Linux or spawns one on Windows and macOS. The new process has a complete Python interpreter instance, its own GIL, and its own memory space. Multiple processes can truly execute Python bytecode in parallel on different cores.
The cost is that interprocess communication must use an IPC mechanism such as a pipe, queue, or shared memory. Data sent between processes must be serialized, usually with pickle, and then deserialized by the recipient. This overhead can be substantial when transferring large quantities of data.
The Execution Model of asyncio
asyncio does not create additional threads or processes at all. It runs an event loop in one thread. Coroutines are lightweight Python-level objects, not OS threads. Switching between coroutines requires only saving and restoring Python stack frames and has extremely low, microsecond-scale overhead. Switching threads is an operating-system context switch and usually takes anywhere from a few to tens of microseconds.
This means asyncio can easily manage tens of thousands of concurrent coroutines, whereas creating tens of thousands of threads places a heavy burden on the operating system.
Chapter 7: Thread Safety and Common Pitfalls
The GIL Does Not Mean Thread Safety
An extremely common misconception is that “because Python has the GIL, multithreaded Python programs do not need to consider thread safety.” This is wrong.
The GIL guarantees that only one thread executes bytecode at a time, but a single Python statement can correspond to multiple bytecode instructions. A thread switch may occur between any two such instructions.
import threading
counter = 0
def increment():
global counter
for _ in range(1_000_000):
counter += 1
# counter += 1 实际上是三步：
# 1. 读取 counter 的值（LOAD_GLOBAL）
# 2. 加 1（BINARY_ADD）
# 3. 写回 counter（STORE_GLOBAL）
# 线程切换可能发生在这三步的任何间隙
threads = [threading.Thread(target=increment) for _ in range(2)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter)
# 结果几乎不可能是 2,000,000
# 因为两个线程会互相覆盖对方的写入
The correct approach is to use a lock:
lock = threading.Lock()
def increment_safe():
global counter
for _ in range(1_000_000):
with lock: # 获取锁
counter += 1 # 这三步字节码在锁的保护下原子执行
# 离开 with 块时自动释放锁
asyncio’s “Illusion of Safety”
Cooperative scheduling in asyncio eliminates many race conditions because code cannot be interrupted between two await expressions. However, a race condition can still occur if two coroutines read and write the same shared state across an await boundary:
balance = 100
async def withdraw(amount):
global balance
current = balance # 读
await asyncio.sleep(0) # 这里让出控制权！另一个协程可能插进来
balance = current - amount # 写——但 current 可能已经过期了
# 如果两个协程同时 withdraw(80)，可能都读到 balance=100
# 然后各自写回 20，最终 balance=20 而不是应该的 -60 或拒绝
Therefore, even in asyncio, a pattern that reads before await and writes afterward must be protected with an asyncio.Lock.
Chapter 8: The Future of Python Concurrency
Python’s concurrency ecosystem is undergoing its largest transformation in nearly a decade. Three directions are advancing.
Direction 1: Free-Threaded Python (A Build Without the GIL)
This is the change attracting the most attention. Python 3.13 first introduced an experimental free-threaded build through PEP 703. In Python 3.14, it has left experimental status and become an officially supported build option, although it is not yet the default build.
The central change in free-threaded Python is removing the GIL, allowing multiple threads to truly execute Python bytecode in parallel. To keep reference counting safe without the GIL, the CPython team implemented a scheme called biased reference counting. The thread that creates an object updates its reference count with fast, non-atomic operations, while other threads use atomic operations. Built-in types such as dict, list, and set also use fine-grained internal locks to protect concurrent modifications.
The current state is that a free-threaded build must be enabled with a specific installation option. The resulting executable usually has a t suffix, such as python3.14t; it is not the default behavior. In Python 3.14, the performance penalty for single-threaded code has fallen to roughly 5–10%, compared with about 40% in 3.13, a substantial improvement.
For you as an AI application engineer, the impact is that once free-threaded Python matures, you will be able to use multithreading directly for true parallel CPU computation in Python instead of detouring through multiprocessing. This may provide substantial benefits for CPU-bound stages such as data preprocessing and feature engineering in AI applications. For now, however, many third-party libraries—especially those containing C extensions—have not fully adapted to free-threaded builds, so production use requires caution.
Direction 2: Subinterpreters
Through PEP 734, Python 3.14 officially introduced the concurrent.interpreters module into the standard library. Subinterpreters are multiple independent Python interpreter instances within one process. Each subinterpreter has its own GIL, module state, and __main__ namespace, while sharing the memory space of the same process.
You can think of a subinterpreter as a parallelism option that is somewhat heavier than a thread but lighter than a process. Like multiprocessing, it can achieve true multicore parallelism. Because it remains within one process, however, it has a lower creation cost and more efficient communication.
# Python 3.14+ 的子解释器 API
from concurrent.futures import InterpreterPoolExecutor
def compute(n):
return sum(i * i for i in range(n))
# 类似 ThreadPoolExecutor 的 API，但每个 worker 跑在独立的子解释器里
with InterpreterPoolExecutor(max_workers=4) as pool:
results = list(pool.map(compute, [10_000_000] * 4))
# 四个子解释器各有自己的 GIL，可以真正并行
The design is inspired by Erlang’s process model and Go’s goroutines: isolated execution environments communicate through message passing rather than sharing mutable state.
Direction 3: The Continued Evolution of asyncio
asyncio itself continues to improve. Python 3.12 introduced TaskGroup for structured concurrency, while Python 3.11 introduced asyncio.TaskGroup and exception groups (ExceptionGroup) to handle error propagation across concurrent tasks more effectively. A future direction is closer integration between asyncio and subinterpreters—for example, distributing CPU-bound subtasks from an asyncio event loop to subinterpreters for parallel execution.
How the Three Directions Relate
They do not replace one another. More accurately, Python’s future concurrency system will look like this:
asyncio will remain the preferred choice for I/O concurrency. Its advantage in managing large numbers of network connections will not change.
For CPU-bound parallelism, developers will have three options: multithreading in free-threaded Python, which is simplest but requires managing thread safety; subinterpreters, which provide strong isolation and a friendly API but impose some communication constraints; and multiprocessing, which is the most mature but has the greatest interprocess communication overhead.
The pattern most likely to emerge in real applications is a hybrid approach: asyncio manages I/O, while subinterpreters or free-threaded multithreading handles CPU-bound work.
Chapter 9: Summary—A Decision Map
Concise Rules for Choosing an Approach
I/O-bound with moderate concurrency, such as dozens of tasks: Use threading or ThreadPoolExecutor. They are simple, direct, and supported by most libraries.
I/O-bound with high concurrency, such as hundreds or thousands of tasks: Use asyncio. It consumes fewer resources but requires libraries from the asynchronous ecosystem.
CPU-bound and requiring parallelism: Use multiprocessing or ProcessPoolExecutor. These are currently the most mature options for genuine parallelism.
Synchronous blocking code encountered in asyncio: Use asyncio.to_thread() or loop.run_in_executor() to bridge it to a thread pool or process pool.
Python 3.14+ with adapted dependencies: Begin evaluating subinterpreters through InterpreterPoolExecutor as a lighter alternative to ProcessPoolExecutor.
Core Concepts at a Glance
The GIL prevents Python threads from executing Python bytecode in parallel, but it does not interfere with I/O waits. Free-threaded Python is gradually removing this limitation.
Concurrency means that several tasks make alternating progress, like one chef preparing three dishes. Parallelism means that several tasks execute simultaneously, like three chefs each preparing one dish.
threading provides concurrency but not parallelism for Python code. multiprocessing provides genuine parallelism. asyncio provides efficient I/O concurrency within one thread. concurrent.futures is a unified, high-level interface for thread pools and process pools.
Thread safety is not the same thing as the GIL. Multithreaded modifications to shared state still require locks. Cooperative scheduling in asyncio reduces the risk of races, but the pattern “read before await, write after await” remains unsafe.
async provides the scheduling protocol, while thread pools and process pools isolate blocking code. They cooperate rather than replace one another.