arq

arq is an async job queue for Python, built on asyncio and Redis, that lets you enqueue functions to run in a separate worker process (or many, for high throughput) rather than blocking the calling process. It supports cron-like scheduled jobs, job retries, timeouts, and result storage, all using Redis as the sole broker without needing a separate message broker like RabbitMQ.

Inspired by (and API-compatible in spirit with) Celery but designed specifically for asyncio, arq is lightweight and easy to deploy for teams already running Redis, offering a simple decorator-based API for defining and enqueuing background tasks.

What You Get

• An asyncio-native worker (arq.worker.Worker) that pulls and executes jobs from Redis
• A simple async def function + decorator pattern for defining background tasks
• Cron-style scheduled job support via arq.cron
• Automatic job retries, timeouts, and result storage in Redis
• A CLI (arq) for running worker processes against a settings module

Common Use Cases

• Offloading slow operations (emails, image processing, webhooks) from a web request to a background worker
• Running scheduled/cron-style jobs (nightly reports, periodic cleanup) from Python asyncio apps
• Adding a lightweight task queue to a FastAPI or other asyncio-based service without introducing RabbitMQ
• Fan-out job processing across multiple worker processes reading from the same Redis queue
• Building simple RPC-style call patterns where a function call is dispatched to and executed by a separate process

Under The Hood

Architecture - arq’s core is compact: connections.py manages the Redis connection pool, worker.py implements the polling/dispatch loop that pulls jobs and executes registered async functions, jobs.py models job state/results, and cron.py layers scheduled-job support on top of the same worker loop, with cli.py providing the arq command to launch a worker against a settings module. Tech Stack - Built directly on redis-py’s asyncio client with no additional broker dependency, targeting Python 3.9+ and using hatchling as its build backend. Code Quality - The tests/ directory (8 files) covers worker execution, cron scheduling, connection handling, and CLI behavior with CI-enforced coverage tracked via Codecov, though the project is explicitly marked ‘in maintenance only mode’ by its maintainers, meaning new feature work has slowed. API Design - Defining a job is a plain async def function registered in a functions list, and enqueuing is a single await pool.enqueue_job(...) call, which keeps the API surface small and easy to pick up for anyone already comfortable with asyncio, at the cost of fewer built-in features (routing, chaining, flower-style monitoring) than heavier frameworks like Celery.