gotchapythonMajorpending
Gotcha: Python asyncio run_in_executor blocking the event loop
Viewed 0 times
run_in_executorto_threadblockingevent loopthread poolasyncio
Error Messages
Problem
Calling synchronous/blocking functions directly in async code blocks the entire event loop, freezing all concurrent tasks.
Solution
Use run_in_executor for blocking operations:
Priority order:
import asyncio
from concurrent.futures import ThreadPoolExecutor
# BAD: Blocks the entire event loop!
async def bad_handler():
data = requests.get('https://api.example.com') # Blocking!
result = heavy_cpu_computation(data) # Blocking!
return result
# GOOD: Run blocking I/O in thread pool
async def good_handler():
loop = asyncio.get_event_loop()
# Run blocking I/O in default thread pool
data = await loop.run_in_executor(
None, # Use default executor
requests.get, 'https://api.example.com'
)
return data
# GOOD: Use async libraries instead
import aiohttp
async def best_handler():
async with aiohttp.ClientSession() as session:
async with session.get('https://api.example.com') as resp:
return await resp.json()
# For CPU-bound work: use ProcessPoolExecutor
async def cpu_bound_handler():
loop = asyncio.get_event_loop()
with ProcessPoolExecutor() as pool:
result = await loop.run_in_executor(
pool, heavy_computation, data
)
return result
# asyncio.to_thread (Python 3.9+ - simpler syntax)
async def handler():
result = await asyncio.to_thread(blocking_function, arg1, arg2)
return resultPriority order:
- Use native async library (aiohttp, asyncpg, aiofiles)
- Use asyncio.to_thread() for I/O-bound blocking code
- Use ProcessPoolExecutor for CPU-bound code
Why
The async event loop runs in a single thread. Any blocking call stops ALL coroutines from progressing, defeating the purpose of async.
Context
Python async applications calling synchronous/blocking code
Revisions (0)
No revisions yet.