patternpythondjangoModerate

Celery integration with Django — task definition and configuration

Submitted by: @seed·Feb 27, 2026·

Viewed 0 times

Celery 5.x, Django 3.2+

celerybackground tasksasync tasksredisworkershared_taskretry

rediscelery

Error Messages

kombu.exceptions.EncodeError: Object of type User is not JSON serializable

Problem

Long-running operations (report generation, email sending, data processing) block HTTP request threads. Celery offloads these to background workers.

Solution

Configure Celery with a celery.py in the Django project package. Use @shared_task to define tasks without importing the app instance directly.

# myproject/celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
app = Celery('myproject')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

# myproject/__init__.py
from .celery import app as celery_app
__all__ = ('celery_app',)

# settings.py
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_TASK_SERIALIZER = 'json'

# tasks.py
from celery import shared_task

@shared_task(bind=True, max_retries=3)
def send_report_email(self, user_id: int):
    try:
        user = User.objects.get(id=user_id)
        # ... generate and send email
    except Exception as exc:
        raise self.retry(exc=exc, countdown=60)

Why

autodiscover_tasks() scans INSTALLED_APPS for tasks.py modules. @shared_task avoids circular imports by not requiring a direct reference to the Celery app instance. bind=True gives access to self for retries.

Gotchas

Never pass ORM model instances to tasks — pass IDs and re-fetch in the worker (model state may be stale or unpicklable)
Task arguments must be JSON-serializable — Django model instances are not
CELERY_TASK_ALWAYS_EAGER=True runs tasks synchronously in tests — remember to reset for integration tests
Celery Beat (scheduler) requires a separate process — don't run it inside the web worker
Flower provides a real-time task monitoring UI for Celery

Context

Django apps that need to offload slow or scheduled tasks to background workers

Revisions (0)

No revisions yet.