Gymnasium

Gymnasium is an open-source Python library that defines a standard API for developing and comparing reinforcement learning algorithms. It provides the Env interface (reset, step, render, close) that RL agents and environments communicate through, plus a large collection of reference environments spanning Classic Control, Box2D, Toy Text, MuJoCo, and Atari (via a separate package). Originally forked from OpenAI’s Gym after OpenAI handed maintenance to an outside team, it is now maintained by the Farama Foundation as the de facto standard interface for single-agent RL research and tooling.

Beyond the core interface, Gymnasium ships a registration system for naming and versioning environments (gym.make("CartPole-v1")), a composable wrapper system for modifying observations/actions/rewards without touching the underlying environment, and both synchronous and asynchronous vectorized environments for running many environment instances in parallel. It also includes conversion utilities for JAX, PyTorch, and NumPy array backends, making it usable across the major deep learning frameworks used in RL research.

What You Get

• A standard Env base class with step, reset, render, and close, plus typed action_space/observation_space attributes for describing valid inputs and outputs
• A registry and gym.make() function for creating versioned environments by string id (e.g. CartPole-v1), with strict versioning so behavior changes bump the version number
• Built-in environment families: Classic Control, Box2D, Toy Text, and MuJoCo, with Atari support available via the separate ale_py integration
• A composable Wrapper system (ActionWrapper, ObservationWrapper, RewardWrapper) for modifying environment behavior without subclassing the environment itself
• Synchronous and asynchronous vectorized environments (SyncVectorEnv, AsyncVectorEnv) for running many environment copies in parallel to speed up training
• Array-backend conversion utilities for JAX, PyTorch, and NumPy, plus a Space type system (Box, Discrete, MultiDiscrete, Dict, Tuple, Graph, Text, Sequence) for describing observation and action spaces

Common Use Cases

• Training and benchmarking reinforcement learning agents against a consistent, versioned set of environments
• Implementing a custom RL environment by subclassing gymnasium.Env so it works with any RL library that targets the Gymnasium API
• Wrapping an existing environment to reshape rewards, stack/normalize observations, or clip actions without modifying the base environment
• Running large-scale RL experiments with vectorized environments to parallelize rollouts across CPU cores
• Using Gymnasium as the environment layer under a separate training library (e.g. Stable-Baselines3, CleanRL) that implements the actual learning algorithms

Under The Hood

Architecture Gymnasium is organized around a small core (gymnasium/core.py) defining the Env and Wrapper generic base classes, with gymnasium/envs/registration.py providing the id-based registry, versioning, and make()/make_vec() factory functions that instantiate environments by string name. Concrete environments live under gymnasium/envs/{classic_control,box2d,toy_text,mujoco,phys2d,tabular,functional_jax_env}, each implementing the Env interface independently, while gymnasium/wrappers/ layers optional behavior (action/observation/reward transforms, JAX/Torch/NumPy conversion, Atari preprocessing) on top without touching the wrapped environment. gymnasium/vector/ provides SyncVectorEnv and AsyncVectorEnv, which batch multiple environment instances (the latter using subprocesses) behind the same Env-shaped API. This separation — a thin core interface, an independent registry, environments that only depend on the interface, and wrappers that compose over any environment — means new environments or wrappers can be added without changes to the core, and the same training code works against a single environment or a vectorized batch.

Tech Stack Gymnasium is pure Python (99.9% of the codebase) targeting Python 3.10+, with a deliberately small required dependency set: NumPy for array operations, cloudpickle for environment/wrapper serialization, typing-extensions for generic typing support, and farama-notifications for maintainer announcements. Everything else — pygame-ce for Classic Control/Box2D/Toy Text rendering, box2d/box2d-py for Box2D physics, mujoco for MuJoCo environments, ale-py for Atari, and jax/torch for array-backend conversion — is gated behind optional extras (gymnasium[atari], gymnasium[mujoco], gymnasium[all], etc.) so consumers only install what their environments actually need. The package builds with setuptools and is published to PyPI on tagged releases via a dedicated GitHub Actions workflow.

Code Quality The repository ships an extensive test suite (over a hundred test files under tests/, covering envs, spaces, wrappers, vector, and functional subsystems) run via pytest across a matrix of Python versions and NumPy major versions in CI, including doctest execution against the library’s own docstrings. Code style is enforced through a pre-commit configuration (Ruff-based linting/formatting, doc-style checks) and the codebase is fully type-hinted, using Generic/TypeVar for parameterizing Env and Wrapper over observation and action types, which lets static type checkers like mypy or pyright verify environment implementations against the declared spaces. Public classes and methods carry Sphinx-style docstrings that are built into a documentation site on every push to main.

What Makes It Unique Gymnasium’s distinguishing choice is treating the RL environment interface itself as the product, deliberately kept minimal and stable, rather than bundling an opinionated set of training algorithms — the strict environment versioning scheme (environments end in -vN, bumped whenever behavior changes) exists specifically so published RL results stay reproducible even as the library evolves. This interface-first design is why it has become the shared substrate that many independent RL and multi-agent libraries (CleanRL, PettingZoo, Stable-Baselines3, and others) build against instead of each defining their own environment contract.