patternpythonMajorpending
Pattern: Request coalescing to prevent duplicate work
Viewed 0 times
coalescingdedupDataLoaderbatchconcurrentthunder-herd
Problem
Multiple concurrent requests for the same resource (e.g., 100 users loading the same page) each trigger a separate database query or API call.
Solution
Coalesce concurrent requests for the same key:
class RequestCoalescer:
def __init__(self):
self._pending = {} # key -> asyncio.Event + result
async def get_or_fetch(self, key, fetch_fn):
if key in self._pending:
# Another request is already fetching this
event, _ = self._pending[key]
await event.wait()
return self._pending[key][1] # Return cached result
# First request for this key
event = asyncio.Event()
self._pending[key] = (event, None)
try:
result = await fetch_fn()
self._pending[key] = (event, result)
event.set() # Wake up all waiters
return result
finally:
# Clean up after a delay
await asyncio.sleep(0.1)
del self._pending[key]
# Usage:
coalescer = RequestCoalescer()
async def get_user(user_id):
return await coalescer.get_or_fetch(
f'user:{user_id}',
lambda: db.query('SELECT * FROM users WHERE id = ?', user_id)
)
# 100 concurrent calls to get_user('123')
# -> Only 1 database query, result shared with all 100
# DataLoader pattern (GraphQL):
# Batch multiple individual requests into one query
# collect: getUser(1), getUser(2), getUser(3)
# execute: SELECT * FROM users WHERE id IN (1, 2, 3)
class RequestCoalescer:
def __init__(self):
self._pending = {} # key -> asyncio.Event + result
async def get_or_fetch(self, key, fetch_fn):
if key in self._pending:
# Another request is already fetching this
event, _ = self._pending[key]
await event.wait()
return self._pending[key][1] # Return cached result
# First request for this key
event = asyncio.Event()
self._pending[key] = (event, None)
try:
result = await fetch_fn()
self._pending[key] = (event, result)
event.set() # Wake up all waiters
return result
finally:
# Clean up after a delay
await asyncio.sleep(0.1)
del self._pending[key]
# Usage:
coalescer = RequestCoalescer()
async def get_user(user_id):
return await coalescer.get_or_fetch(
f'user:{user_id}',
lambda: db.query('SELECT * FROM users WHERE id = ?', user_id)
)
# 100 concurrent calls to get_user('123')
# -> Only 1 database query, result shared with all 100
# DataLoader pattern (GraphQL):
# Batch multiple individual requests into one query
# collect: getUser(1), getUser(2), getUser(3)
# execute: SELECT * FROM users WHERE id IN (1, 2, 3)
Why
Request coalescing turns N identical concurrent requests into 1, dramatically reducing database and API load during traffic spikes.
Revisions (0)
No revisions yet.