RQ

RQ (Redis Queue) is a lightweight Python library for creating background job queues and processing them asynchronously with worker processes. It is backed by Redis (or Valkey) and designed to have a low barrier to entry: any Python function can be enqueued without special decorators or a running message broker beyond Redis itself.

RQ has become a popular alternative to heavier task-queue systems like Celery for teams that want background job processing without the operational complexity of an AMQP broker. It supports job prioritization via multiple queues, scheduling, cron-style recurring jobs, retries, rate limiting, and webhooks for job lifecycle events, all while keeping the core API small enough to learn in minutes.

What You Get

• A Queue class for enqueueing arbitrary Python functions as background jobs backed by Redis/Valkey
• Worker processes (Worker, SimpleWorker, SpawnWorker, and worker-pool) that pull and execute jobs, with per-job process isolation and crash containment
• Built-in job scheduling (enqueue_at, enqueue_in) and a cron-style scheduler (rq cron) for recurring tasks
• Retry, repeat, and concurrency-based rate-limiting primitives for controlling how and how often jobs run
• Webhook support that posts job status updates to an HTTP endpoint without writing a callback function
• A CLI (rq worker, rq info, rq cron) for running and inspecting workers and queues

Common Use Cases

• Offloading slow operations (sending email, generating reports, processing images) from a web request/response cycle
• Running scheduled or cron-style maintenance jobs such as database cleanup or nightly report generation
• Building a lightweight task-queue layer for a Flask/Django app without adopting Celery’s broker and configuration surface
• Prioritizing urgent jobs ahead of bulk/low-priority work using multiple named queues

Under The Hood

Architecture - RQ’s core flow is Queue.enqueue() serializing a function reference plus arguments into a Job object stored in Redis, which a Worker process (or SimpleWorker/SpawnWorker) later pops via BRPOP-style blocking reads and executes, writing results and status transitions (queued -> started -> finished/failed) back into Redis-backed registries (registry.py). Scheduling and cron support (scheduler.py, cron.py) layer on top by pushing due jobs into the normal queue at the right time, and rate_limit.py/repeat.py add concurrency and re-execution semantics without changing this base model. Tech Stack - Pure Python 3.10+ with a deliberately small dependency set: redis-py for the Redis/Valkey client, click for the CLI, and croniter for cron expression parsing; packaging uses hatchling with dynamic versioning. Code Quality - The tests/ directory contains 41 test modules with roughly 700 test functions covering CLI behavior, connections, cron, callbacks, and worker lifecycle, indicating broad functional coverage; the codebase uses py.typed to expose type hints to consumers, and Ruff is used for linting per the CI badge in the README. API Design - The public API favors minimal ceremony: any plain function can be enqueued with no decorator or base class, and higher-level behaviors (retry, repeat, rate limiting, webhooks) are opt-in constructor objects (Retry(...), RateLimit(...), Webhook(...)) passed straight into enqueue(), keeping the common path (queue.enqueue(fn, arg)) a single line while still supporting advanced configuration.