Dramatiq

Dramatiq is a Python library for running background jobs on RabbitMQ or Redis, built as a lighter-weight, more strictly-scoped alternative to Celery. You define an @dramatiq.actor-decorated function, call .send() to enqueue work, and run one or more dramatiq worker processes to consume it — with retries, rate limiting, and message middleware built in rather than bolted on.

The library emphasizes reliability by default: message acknowledgement, automatic retries with backoff, and at-least-once delivery semantics are core behaviors rather than opt-in configuration, and a pluggable middleware system lets you customize prioritization, rate limiting, or result storage without forking the broker layer itself.

What You Get

• An @dramatiq.actor decorator that turns a plain function into a distributed background task with .send()/.send_with_options()
• RabbitMQ and Redis broker backends, selectable via install extras (dramatiq[rabbitmq] / dramatiq[redis])
• Built-in retry, rate-limiting, and prioritization middleware, plus a pluggable middleware system for custom behavior
• A dramatiq CLI worker runner with an optional watch mode for auto-reloading during development
• Result backends for retrieving actor return values, and composition helpers (composition.py) for chaining and grouping tasks

Common Use Cases

• Offloading slow operations (image processing, email sending, report generation) from a web request into a background worker
• Running scheduled or retryable jobs against RabbitMQ or Redis with reliable at-least-once delivery
• Rate-limiting calls to a downstream API or resource shared across many worker processes via Dramatiq’s rate-limit middleware
• Composing multi-step workflows (fan-out/fan-in task pipelines) using Dramatiq’s task composition helpers

Under The Hood

Architecture - actor.py and broker.py form the core: an actor wraps a function and a Broker implementation (under brokers/, covering RabbitMQ and Redis) handles enqueueing and delivery, while worker.py implements the process/thread pool that pulls messages and executes actor callables. middleware/ implements retries, rate limiting, prioritization, and other cross-cutting behavior as composable hooks into the actor lifecycle rather than hardcoded broker logic, and results/ provides a pluggable result-backend abstraction for actors that return values. composition.py adds higher-level pipeline/group primitives on top of the base send/receive model, and cli.py/__main__.py implement the dramatiq worker-runner command with watch.py-based auto-reload support.

Tech Stack - Python 3.10+, with broker-specific dependencies isolated behind install extras so a RabbitMQ-only deployment doesn’t pull in Redis client libraries or vice versa. rate_limits/ implements distributed rate limiting and locking primitives (reflected in the distributed-lock GitHub topic) directly rather than depending on an external library, and asyncio.py/threading.py provide separate concurrency-model support for actors that need async or thread-based execution.

Code Quality - tests/ contains 35 test files covering brokers, middleware, rate limits, composition, and the CLI, and a dedicated pytest-gevent.py runner variant indicates the maintainers test compatibility with gevent-based deployments specifically. The library ships a py.typed marker for type-checker support, and errors.py centralizes actor/broker-specific exceptions rather than leaking raw AMQP/Redis client errors to user code.

API Design - The core loop — decorate a function with @dramatiq.actor, call .send(...) to enqueue, run dramatiq module:app to start workers — requires very little boilerplate to get a first background job running, and middleware (retries, rate limits) attach declaratively via actor options rather than requiring separate configuration objects. The one-time setup cost is picking and installing the right broker extra (RabbitMQ vs. Redis) up front, since the two aren’t interchangeable at runtime without reconfiguring the broker.