django-structlog

django-structlog wires structlog into Django’s request/response cycle and Celery task lifecycle, automatically binding contextual metadata — request IDs, user IDs, correlation IDs, and IP addresses — to every log line emitted during a request or task, so operators can grep or query logs by request_id across an entire call chain without manually threading context through every view and task.

Beyond middleware, it exposes Django signals (bind_extra_request_metadata, bind_extra_request_finished_metadata, bind_extra_request_failed_metadata, update_failure_response) for extending the bound context per project, and a Celery integration (CeleryReceiver, DjangoStructLogInitStep boot step) that propagates parent-task context to child tasks and emits enqueued/started/succeeded/failed/retried/revoked events with duration tracking. It supports Django REST framework and django-ninja out of the box, and can optionally log management command invocations.

What You Get

• A drop-in RequestMiddleware (sync and async capable) that binds request_id, user_id, correlation_id, and client IP to structlog’s contextvars for the duration of a request
• Automatic request_started / request_finished / request_failed log events with configurable log levels per status code range (2xx/4xx/5xx)
• A Celery integration (CeleryReceiver + DjangoStructLogInitStep boot step) that propagates request/parent-task context into child tasks and logs enqueue/start/success/failure/retry/revoke events with task duration
• Four Django signals (bind_extra_request_metadata, bind_extra_request_finished_metadata, bind_extra_request_failed_metadata, update_failure_response) for project-specific context extension
• Optional structured logging of Django management command invocations via DJANGO_STRUCTLOG_COMMAND_LOGGING_ENABLED
• Out-of-the-box support for Django REST framework and django-ninja views

Common Use Cases

• Correlating every log line produced by a single HTTP request (including logs emitted deep in business logic) via a shared request_id
• Tracing a background job back to the HTTP request that triggered it by propagating request context into Celery tasks
• Feeding structured JSON logs into an ELK/Loki/Datadog pipeline where fields like request_id, user_id, and duration_ms are queried directly instead of regex-parsed from plain text
• Auditing failed requests and task failures with full exception context alongside the same correlation IDs used for successful requests
• Logging Django management command runs (e.g. scheduled maintenance scripts) with the same structured format as web requests

Under The Hood

Architecture The middleware wraps Django’s request/response cycle with dual sync/async code paths (RequestMiddleware.__call__/__acall__), binding context via structlog.contextvars, while separate modules isolate concerns: middlewares/ for HTTP, celery/ for task lifecycle (CeleryReceiver plus the DjangoStructLogInitStep worker boot step), commands.py for management-command logging, and signals.py for four extension points. AppConfig.ready() conditionally wires the Celery and command receivers based on app_settings flags, and context propagates from parent task to child task by smuggling a __django_structlog__ header through Celery’s before_task_publish/after_task_publish signal pair — a layered design where the core abstraction (a contextvars-bound structlog logger) stays decoupled from the two integration surfaces that consume it.

Tech Stack A pure-Python library targeting Django 4.2 through 5.2, built on structlog, asgiref for sync/async bridging, and django-ipware for client IP extraction, with optional extras for celery and django-extensions (command logging). Packaged with setuptools using dynamic versioning, linted with ruff and black, and validated across a wide Python/Django/Celery version matrix via tox.

Code Quality A dedicated test suite covers app settings, app config, management commands, Celery receivers and signals, and middleware behavior, run with pytest-django and tracked via codecov. mypy runs in strict mode across the library and its tests, and a GitHub Actions workflow runs the full compatibility matrix plus pre-commit checks on every push. Error handling is deliberate rather than swallowed — exceptions are explicitly re-raised after logging in streaming response wrappers, and expected exceptions (404s, permission denials) are distinguished from genuine failures so they don’t pollute failure logs.

API Design The public surface is small and consistent: adding structured, request-scoped logging to an existing Django project needs only two settings.py changes (an installed app plus a middleware entry), with Celery support added via a single boot-step registration line and no changes to task code. Event naming mirrors itself across surfaces (task_started/task_succeeded/task_failed echo request_started/request_finished/request_failed), and every public signal ships a runnable usage example in its docstring.