Python multiprocessing vs threading vs asyncio decision guide

Need to choose between multiprocessing, threading, and asyncio for concurrent Python code.

Decision guide based on workload type:

CPU-bound (computation): Use multiprocessing

from multiprocessing import Pool

def heavy_math(n):
    return sum(i*i for i in range(n))

with Pool(processes=4) as pool:
    results = pool.map(heavy_math, [10**7]*8)
# True parallelism - bypasses GIL

I/O-bound with many connections: Use asyncio

import asyncio
import aiohttp

async def fetch_all(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [session.get(url) for url in urls]
        return await asyncio.gather(*tasks)
# Best for: 100s-1000s of network requests

I/O-bound with blocking libraries: Use threading

from concurrent.futures import ThreadPoolExecutor

def download(url):
    return requests.get(url)  # Blocking library

with ThreadPoolExecutor(max_workers=10) as pool:
    results = list(pool.map(download, urls))
# Good when async library isn't available

Decision tree:

1. CPU-bound? -> multiprocessing (or ProcessPoolExecutor)
2. I/O-bound + async library exists? -> asyncio
3. I/O-bound + blocking library? -> threading
4. Simple parallelism? -> concurrent.futures (unified API)
5. Need shared state? -> threading (with locks) or multiprocessing.Manager

Performance characteristics:

• asyncio: 1 thread, 10K+ concurrent I/O ops, low memory
• threading: N threads, GIL limits CPU parallelism, moderate memory
• multiprocessing: N processes, true parallelism, high memory (copies per process)

Python's GIL prevents true CPU parallelism with threads. Choosing the wrong concurrency model results in either no speedup (threading for CPU) or unnecessary complexity (multiprocessing for I/O).

Python applications needing concurrent execution