pytest-randomly

pytest-randomly is a Pytest plugin that shuffles the order of your test suite and resets Python’s global random seed before every test, exposing hidden inter-test dependencies and flaky ordering assumptions that a fixed run order would silently hide.

It prints a base seed at the start of each run so any failure can be reproduced exactly with —randomly-seed, and it automatically reseeds popular randomness sources such as factory_boy, Faker, Model Bakery, and NumPy so data-generating tests stay deterministic yet varied.

What You Get

• Automatic shuffling of test order at the module, class, and function levels to surface inter-test dependencies
• A printed base seed plus —randomly-seed=last for exact reproduction of any failing run
• Automatic reseeding of factory_boy, Faker, Model Bakery, and NumPy when they are installed
• A faker_seed fixture and a pytest_randomly.random_seeder entry point for registering custom generators
• Fine-grained opt-out flags and full pytest-xdist compatibility

Common Use Cases

• Detecting tests that secretly depend on execution order or leaked global state
• Making randomized test data (factory_boy fuzzy values, Faker output) reproducible across runs
• Reproducing a CI failure locally by replaying the exact seed reported in the logs

Under The Hood

Architecture - The entire plugin is a single ~280-line module (src/pytest_randomly/init.py) that hooks into Pytest’s lifecycle. It registers command-line options via pytest_addoption, computes a base seed with make_seed (random.Random().getrandbits(32)), derives a per-item seed from the base seed, the test node id, and a setup/run/teardown offset via zlib.crc32, and reorders collected items in pytest_collection_modifyitems using itertools.groupby to shuffle by module, then class, then function.

Tech Stack - Pure Python (>=3.10), depending only on pytest at runtime. It integrates with factory_boy, Faker, Model Bakery, and NumPy through guarded optional imports, resetting each generator’s state when installed. Built with setuptools, linted with Ruff, type-checked with strict mypy (ships a py.typed marker), and tested under tox across Python 3.10-3.14.

Code Quality - Advertises and enforces 100% coverage; the 279-line implementation is matched by a ~800-line test suite (tests/test_pytest_randomly.py). Optional integrations are wrapped in try/except ImportError blocks with pragma-no-cover annotations, and lru_cache is used to memoize entry-point discovery. Strict typing and required future imports keep the surface disciplined.

API Design - Developer experience is excellent: install and it works with zero configuration. Every behavior is toggleable through discoverable flags (—randomly-seed, —randomly-dont-reset-seed, —randomly-dont-reorganize, -p no:randomly), the —randomly-seed=last shortcut streamlines reproduction, and the pytest_randomly.random_seeder entry point offers a clean extension path for custom generators.