diffusers

Diffusers is Hugging Face’s library for state-of-the-art diffusion models in PyTorch, covering everything from Stable Diffusion and Flux text-to-image pipelines to video, audio, and molecule-structure generation. Rather than shipping one monolithic model, it exposes three composable building blocks — pipelines, models, and schedulers — so developers can run inference in a few lines of code or assemble a fully custom diffusion system from the same parts researchers use.

The library is built around from_pretrained, which pulls any of the 30,000+ diffusion checkpoints hosted on the Hugging Face Hub straight into a runnable pipeline, with quantization, LoRA, ControlNet, and multi-GPU/parallelism support layered in for production use. Its explicit design philosophy — usability over performance, simple over easy, customizability over abstraction — keeps the codebase approachable even as it tracks the fast-moving diffusion research landscape.

With over 30,000 GitHub stars, 1,100+ contributors, and more than 90 releases, diffusers has become the de facto standard for running and fine-tuning diffusion models in Python, used both directly by practitioners and as the inference backend inside countless downstream apps and research projects.

What You Get

• Ready-to-run diffusion pipelines (Stable Diffusion, Flux, Kandinsky, ControlNet, DeepFloyd IF, and dozens more) invoked via DiffusionPipeline.from_pretrained
• A library of interchangeable noise schedulers (DDPM, DDIM, DPMSolver, and others) that trade off inference speed against output quality
• Pretrained model building blocks (UNets, transformers, autoencoders/VAEs) that can be recombined into custom diffusion systems
• Training utilities and example scripts covering DreamBooth, textual inversion, ControlNet training, and LoRA fine-tuning
• Built-in support for quantization backends (bitsandbytes, GGUF, TorchAO, Optimum Quanto) and memory/performance optimization for consumer GPUs and Apple Silicon
• A diffusers-cli console tool and a modular pipeline system for composing custom generation flows without forking pipeline code

Common Use Cases

• Generating images from text prompts using Stable Diffusion, Flux, or other Hub-hosted checkpoints in production or research code
• Fine-tuning a diffusion model on a custom dataset with DreamBooth, textual inversion, or LoRA using the provided example scripts
• Building guided image editing features (inpainting, ControlNet-conditioned generation, instruction-based image-to-image) inside an application
• Assembling a custom diffusion pipeline from individual schedulers and models for research experimentation
• Serving diffusion inference at scale with quantized or memory-optimized model loading on limited GPU hardware

Under The Hood

Architecture diffusers organizes around three interchangeable layers under src/diffusers/: pipelines/ (one directory per pipeline family — flux, controlnet, cogvideo, hunyuan_video, and dozens more — each wiring together a DiffusionPipeline subclass), models/ (building blocks like UNets, transformers, autoencoders, and modeling_utils.py’s shared ModelMixin for from_pretrained loading, sharding, and device placement), and schedulers/ (interchangeable noise schedulers implementing a common step/scale interface). configuration_utils.py and modeling_utils.py centralize how any component loads from a Hugging Face Hub repo id, so swapping a scheduler or a checkpoint never requires touching pipeline logic. This separation is what lets the same DDPM UNet be driven by a DDIM, DPMSolver, or custom scheduler, and it is the abstraction that would break the whole system if it were removed — pipelines depend on models and schedulers exposing a stable, interchangeable interface, not on any single model’s internals.

Tech Stack The library targets Python 3.10+ and PyTorch as its core tensor backend, with a deliberately lean set of hard dependencies (huggingface-hub, safetensors, numpy, Pillow, httpx, regex, filelock, importlib_metadata) — everything else (torch itself, accelerate, transformers, bitsandbytes, gguf, torchao, optimum-quanto, sentencepiece, torchvision) is gated behind pip extras like diffusers[torch] so installs stay minimal by default. Packaging is classic setup.py plus pyproject.toml (ruff-configured), checkpoints load via safetensors/Hub integration rather than pickled .bin files, and a diffusers-cli entry point ships as a console script. CI on GitHub Actions runs style, unit, nightly, and GPU-backed test suites separately.

Code Quality Testing is extensive and non-negotiable here — 445+ test_*.py files under tests/, organized to mirror pipelines/, models/, and schedulers/, run through dedicated PR, nightly, and GPU-tests workflows. Code style is enforced with ruff (configured in pyproject.toml, both linting and formatting) and a make quality/make style workflow referenced throughout CONTRIBUTING.md. Core modules make consistent use of type hints (Optional, Callable, ContextManager, tensor types) and centralized logging via diffusers.utils.logging, and error handling favors explicit validation with deprecate() warnings for API transitions rather than silent failure.

What Makes It Unique What distinguishes diffusers from a typical model-serving library is that it treats diffusion models as decomposable systems rather than fixed checkpoints: any pipeline’s scheduler, UNet, or VAE can be swapped independently, and the same from_pretrained convention that loads a full pipeline also loads its individual parts for custom composition — a design pulled directly from Hugging Face’s broader ecosystem philosophy (mirroring transformers). Its ‘modular diffusers’ system takes this further, letting developers assemble pipelines from reusable blocks instead of subclassing. Combined with day-one support for nearly every major open diffusion model family and quantization/parallelism options built in rather than bolted on, it functions as the shared substrate most of the open diffusion ecosystem is built against, rather than a single opinionated tool.