rudder-sdk-python

rudder-sdk-python is the server-side Python client for RudderStack, the open-source, warehouse-first customer data platform. It exposes a Segment-analytics.js-style API (track, identify, page, screen, group, alias) that queues events in memory and ships them in the background to a configurable RudderStack data plane URL, which then routes them to whichever destinations (warehouses, SaaS tools, webhooks) the source is wired to. Because it targets your own data plane rather than a fixed vendor endpoint, it works identically against RudderStack Cloud or a self-hosted rudder-server deployment.

Under the hood it is a thin, dependency-light wrapper: a background daemon thread batches queued events and posts them (optionally gzip-compressed) with exponential backoff via backoff, while a synchronous mode is available for scripts and short-lived processes that need blocking delivery. It also supports HTTP/HTTPS proxy configuration for environments that require egress through a corporate proxy. The package has shipped as a stable, low-churn dependency for RudderStack customers since 2020 and is maintained directly by the RudderStack team.

What You Get

• Full event API - track, identify, page, screen, group, and alias calls matching the familiar Segment-style analytics API shape.
• Background delivery with batching - a daemon consumer thread accumulates events and flushes them in size- and time-bounded batches so calling code never blocks on network I/O.
• Synchronous mode - sync_mode=True sends each event inline via a blocking HTTP POST, useful for short-lived scripts, Lambda functions, or tests where a background thread would be killed before flushing.
• Automatic retry with backoff - failed uploads retry with exponential backoff (via the backoff library) up to a configurable max_retries, giving up early only on non-retryable 4xx errors.
• Gzip payload compression - request bodies are gzip-compressed by default to reduce bandwidth, toggleable via the gzip flag.
• Proxy support - accepts either a requests-style proxy mapping or a legacy host:port string, applied to both HTTP and HTTPS traffic.
• Configurable data plane URL - points at any RudderStack data plane (cloud or self-hosted), not a hardcoded vendor endpoint.
• Graceful shutdown hooks - registers an atexit handler plus explicit flush()/join()/shutdown() methods so queued events aren’t silently dropped on process exit.

Common Use Cases

• Server-side event tracking - emit track calls from application backends (Django, Flask, FastAPI) for signups, purchases, and feature usage without adding a client-side script.
• User and account identification - call identify/group when a user logs in or an account is created so downstream destinations get enriched trait data alongside events.
• Batch or cron job instrumentation - use sync_mode in short-lived scripts and scheduled jobs where a background thread wouldn’t have time to flush before the process exits.
• Migrating off Segment - drop-in replacement for analytics-python/Segment’s Python SDK for teams moving to RudderStack’s open-source data pipeline while keeping the same call signatures.
• Multi-destination fan-out - send one track/identify call and have RudderStack route the event to a warehouse, a CDP, and a marketing tool simultaneously, without separate SDK integrations for each.

Under The Hood

Architecture The SDK is organized as four small, single-responsibility modules: client.py exposes the public Client class and validates/normalizes each call (track, identify, page, screen, group, alias) into a common message dict before enqueuing it on a bounded queue.Queue; consumer.py defines a daemon Thread subclass that drains the queue in time- and size-bounded batches (upload_interval, upload_size, a 4MB BATCH_SIZE_LIMIT) and hands batches to request.py, which performs the actual HTTP POST (gzip-optional, proxy-aware) against the configured data plane URL and raises a typed APIError on non-200 responses; utils.py holds small helpers for timezone normalization and recursive dict/message cleaning. The module-level __init__.py layers a Segment-compatible functional API (analytics.track(...), module-level config attributes) on top of a lazily-created singleton Client, so callers can either use the module directly or instantiate their own Client for finer control (custom queue size, sync mode, multiple write keys). Retries are handled declaratively via the backoff library’s exponential-backoff decorator in Consumer.request, with a fatal_exception predicate that stops retrying on 4xx errors (except 429) but always retries 5xx and network failures — a clean, easy-to-follow separation between transport, batching, and retry policy.

Tech Stack Pure Python 3.8+ with a deliberately minimal dependency set: requests for HTTP transport (via a module-level requests.sessions.Session for connection reuse), backoff for retry/backoff policy, python-dateutil for timezone-aware timestamp handling, python-dotenv for optional .env-based configuration in tests/examples, and deprecation to mark the legacy host property as deprecated in favor of dataPlaneUrl. There is no async/await support — concurrency comes entirely from a background threading.Thread consumer, which keeps the implementation simple but means there’s no native asyncio integration for async web frameworks. Packaging uses classic setuptools/setup.py (no pyproject.toml), with pip-compile-generated requirements.txt pinning transitive versions for reproducible CI runs across five supported Python versions.

Code Quality The repo has a dedicated rudderstack/analytics/test/ package with nine test modules (client, consumer, request, request options, utils, module-level API, public API contract, constants, and a release-validation test) totalling over a thousand lines, run via pytest and mocking the transport layer (mock.patch('rudderstack.analytics.consumer.post')) so tests never hit the network. CI (GitHub Actions) runs the full suite across Python 3.8 through 3.12 on every PR, plus a flake8 lint pass (first a strict syntax-error-only pass, then a broader complexity/line-length pass in warn-only mode). Naming and error handling are straightforward and explicit (a small require() assertion helper enforces argument types at the API boundary rather than failing silently downstream), though the codebase has no static type annotations or mypy checking, so type errors surface only at runtime or in tests.

What Makes It Unique Unlike vendor-locked SDKs (e.g. Segment’s own client, which only ever talks to Segment’s endpoint), this SDK’s entire value proposition is that the data plane URL is a first-class, user-supplied configuration value — the identical client code works against RudderStack Cloud or a self-hosted rudder-server instance with no code changes, which is the core promise of RudderStack’s “warehouse-first, open-source” positioning. Its close API compatibility with Segment’s analytics-python (same method names and call signatures) is also a deliberate migration aid rather than a coincidence, letting teams swap SDKs with minimal application-code churn.